Kuldeep Kushwaha

Background

About Kuldeep Kushwaha

With a Master degree in Computer Science, I am a passionate and proficient Data Engineer who specializes in developing and managing data processing pipelines using Airflow,ADF, Spark, and Databricks. As a Data Engineer at Infosys, I spearheaded the creation and monitoring of Airflow Dags, ensuring seamless data processing for diverse clients. I also acted as the primary liaison for resolving complex data engineering issues related to large-scale datasets, leveraging my expertise in S3, NSP, Python, and orchestration tools. I collaborated cross-functionally to troubleshoot and resolve batch and real-time data processing challenges, and optimized data processing workflows, handling and migrating hundreds of TB of data. I contributed to slashing processing time by a remarkable 30% by implementing incremental data processing pipelines using Apache Spark. My goal is to enhance organizational efficiency through technology-driven, data-centric approaches. I offer comprehensive guidance to clients on data engineering best practices and optimizations, and empower businesses to make informed decisions through data insights. I am proficient in Big Data tools like Airflow, Spark, Kafka, and Hive, along with databases such as MySQL and Snowflake. I am always eager to learn new technologies and skills to advance my data engineering career.