Arvind Pal

Background

About Arvind Pal

Microsoft certified Azure Data Engineer(Azure DP-900 & DP-203 Certified). 15+ Years of IT experience, out of which 3+ years of experience as Data Engineer. Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and Load data from different sources like Azure Data Lake gen 2, Azure SQL, Blob storage, Azure SQL Data warehouse and Azure Synapse. Developed Spark applications using PySpark and Spark-SQL for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns. Responsible for estimating the cluster size, monitoring and troubleshooting of the Spark data bricks cluster. Analyze, design and build modern data solutions using Azure PaaS service to support visualization of data. Understand current production state of application and determine the impact of new implementation on existing business processes. Design & implement migration strategies for traditional systems on Azure. Scheduling the jobs and pipelines using triggers on regular interval of time. Deriving valuable insights from the data and prepare the reports. Domain Knowledge of Banking, Insurance, SCM, Retail and Health Care