Jay Panchal

Background

About Jay Panchal

I’m a Data Engineer with 9+ years of experience in designing, building, and optimizing end-to-end data solutions—4+ years specializing in Microsoft Azure cloud technologies, including Azure Data Factory (ADF), Azure Databricks (ADB), and Data Lake Storage Gen2. I’ve worked across industries such as retail, finance, healthcare, and e-commerce, helping businesses turn raw data into trusted, high-quality insights. I build scalable, reliable ETL/ELT pipelines and automate complex data workflows using PySpark, SQL, and Python. My approach combines deep technical expertise with a strong understanding of business goals, allowing me to deliver data solutions that are fast, maintainable, and cost-effective. I focus heavily on data quality, performance tuning, reusability, and governance. Core Competencies: Azure Data Factory | Azure Databricks | Data Lake Gen2 PySpark | SQL | Python | Delta Lake ETL/ELT Pipelines | Batch & Streaming Data Data Modeling | Data Quality & Monitoring CI/CD for Data Pipelines | Git | DevOps Career Highlights: Delivered 20+ enterprise-grade pipelines using ADF and ADB, processing TBs of structured and semi-structured data. Reduced data latency by 60% through Spark job optimizations and partitioning strategies. Designed reusable pipeline frameworks supporting 10+ business domains with minimal maintenance. Enabled business teams to build self-service dashboards using curated datasets and Power BI. Spain Work Visa Holder – No sponsorship required.