Mohammad Dilshad

Background

About Mohammad Dilshad

I am a Lead Data Engineer with over 10 years of dedicated experience in designing, architecting, and managing data engineering platforms across FinTech, AdTech, and Telecom industries. My professional journey has been entirely focused on data engineering, transforming raw, complex data into scalable, automated, and insightful systems. I have architected and built 100+ data pipelines, handling data from small event-driven files to large-scale batch workloads processing terabytes of information daily. My work covers data lakes, ETL pipelines, and data warehouses, enabling efficient, reliable, and business-ready data delivery. Currently, I lead AWS Cloud–based data engineering initiatives, designing and orchestrating event-driven, automated pipelines using AWS Glue, Lambda, MWAA (Apache Airflow), Step Functions, SNS, SQS, EventBridge, and CloudWatch. I specialize in data lake architectures using Glue Catalog, Iceberg, Delta Lake, and Parquet, ensuring data consistency, scalability, and high performance. I have deep expertise in Change Data Capture (CDC) and Slowly Changing Dimension (SCD Type 0/1/2) design patterns, implementing Terraform-based infrastructure automation to support CI/CD deployments and reproducible environments. In Snowflake, I’ve led several migration projects from on-prem and Hadoop-based data warehouses, as well as integrated Snowflake with AWS pipelines to enable both batch and real-time analytics. These efforts have modernized legacy data systems, optimized performance, and reduced operational costs. Technically, I’m highly skilled in Python, PySpark, and SQL, developing distributed ETL workflows, automating data transformations, and fine-tuning complex analytical queries for performance and scalability. Using Airflow (MWAA), I orchestrate DAGs with dataset and schema dependencies to ensure reliable and fully monitored pipeline execution. My early career working with Hadoop, Hive, Tez, and Presto gave me strong fundamentals in big data processing, which I’ve evolved into cloud-native solutions built for flexibility and scale. I’m passionate about data architecture, automation, and orchestration, and take pride in building robust, future-ready data platforms that seamlessly handle everything—from small incremental files to massive data streams—empowering organizations to make smarter, faster, data-driven decisions.