Deshmanth Reddy

Background

About Deshmanth Reddy

Having sound working experience on ETL/ELT projects with skills as Google Cloud Platform, Databricks, Azure Data Factories, Cloud Storages. Understanding business requirements, analysis, design, development and implementation into the system. Having experience in using the Cloud Storages like Flat namespace & Hirearchy to read the data as input to pipeline Having expertise in creating the datasets as a reference data points to pipelines to orchestrate Pyspark, python and sql were used extensively to use perform the various transformations. Extensively used and created multiple linked services to access the data and input them to pipelines Extensively implemented the Archival process to have a clear tracking on processed files. Good understanding in Data warehouse concepts like Dimensions, Facts, different schemas and all SCD’s. Have experience in creating pipeline jobs, schedule triggers using azure data factory and airflow Automated the execution of ADF pipelines using triggers and airflow DAGs Experience with SNOWFLAKE DATA WAREHOUSE, deep understanding ofsnowflake architecture. Having experience in connecting to different sources like Azure, SQL database, data lake storage and blob storage in ADF and GCS, Bigquery and Dataproc in GCP Good understanding on Spark architecture including spark core, PySpark,spark-SQL using data frames. Strong experience in coding the PySpark programming on core and functional Azure Data bricks Extensively used joins, unions, control flows and also plain SQL implementation using spark SQL Sound exposure in using python variables list, tuples, files operations, functions, control flow statements from python programming.