Arpit Gupta

Background

About Arpit Gupta

Thanks for visiting my profile. • 5+ years of experience in developing applications that perform large scale Distributed Data Processing using Big Data ecosystem tools Hadoop, Spark, Airflow, Cloud Services - Azure, SparkSQL. • Passionate, innovative and disciplined Software Engineer with good analytical, inter-personal, organizational & mentoring skills. • Excellent experience in following the agile methodologies. • Efficient in writing Python using Apache Spark API for analyzing the structured and unstructured data. • Hands on experience in Spark processing Framework using Spark APIs such as Spark RDD/Data Frames/Datasets in Python languages and Spark SQL. • Expertise in implementing queries using SQL windowing functions, optimization techniques and troubleshooting techniques. • Hands on experience in setting up workflow using Apache Airflow workflow engine for managing and scheduling Hadoop jobs using Airflow-Coordinator. • Proficient in UNIX shell scripting. • Good experience working with AWS, Azure CLOUD. • Hands on experience in setting up DevOps process(Git). • Proficient knowledge of CI/CD pipeline setup.