Arpitha Shibu

Background

About Arpitha Shibu

Hands-on experience in modernizing stored procedure jobs, specialize in leveraging cutting-edge technologies like PySpark, Azure Data Factory, and Azure Databricks to drive performance and scalability. Transforming legacy systems, optimizing processing workflows, and ensuring data readiness for advanced analytics. Key responsibilities include: • Modernizing stored procedure jobs into PySpark-based solutions, significantly enhancing scalability and efficiency across the data pipeline. • Designing and developing high-performance ETL pipelines and optimizing PySpark jobs for reducing processing times by over 50% using Azure Data Factory and Databricks. • Leveraging PySpark and Pandas for data cleansing, transformation, and aggregation, ensuring clean and actionable data for analytics. • Optimizing the performance of notebooks and data pipelines through efficient logic, functions, and design to align with business requirements. • Automating data workflows with Python scripting to minimize manual intervention and accelerate throughput. • Translating complex business requirements into scalable and efficient data solutions, ensuring seamless integration and transformation of business-critical data. • Collaborating with cross-functional teams to align technical specifications with business objectives, enhancing overall data infrastructure and reporting capabilities. In addition to my current role, I have gained valuable experience as a Data and Analytics Intern at Aditya Birla Management Corporation Pvt Ltd, where I built ML models for anomaly detection and process failure prediction. Holds Master's degree in Big Data Analytics from St. Joseph's College (Autonomous) and a Bachelor's degree in Mathematics from Stella Maris College. With a passion for continuous learning and problem-solving, I thrive in dynamic environments, consistently delivering high-quality data solutions that drive business success.