Rahim Sheik

Background

About Rahim Sheik

Experienced Data Engineer Lead: Over 12 years of proven expertise in leveraging both on-premise and cloud-based solutions to drive business outcomes effectively. • Designing and Maintaining Big Data Architectures: Spearheaded the design, development, and maintenance of robust Big Data architectures, including data pipelines and ETL processes, to facilitate data-driven decision-making. • Specialization in Data Management and BI Technologies: Proficient in data architecture, ETL development, data warehousing, and Data Modeling, with a focus on delivering comprehensive solutions. • Business Requirement Translation to Data Models: Proficient in translating complex business requirements into conceptual and logical data models to support decision-making processes effectively. • Implementation of Configuration-Driven Data Processing Frameworks: Developed and deployed configuration-driven data processing frameworks using cutting-edge technologies like Spark (Cloudera, Control M, Airflow), Azure Data Bricks Notebook (PY Spark with Delta Lake), and Azure Data Factory. • Expertise in Big Data Technologies: Extensive experience (over 6 years) in Big Data technologies including Hive, Hadoop, Impala, NiFi, Spark, Scala, KUDU, PIG, HBase, Kafka, JAVA, and Python. • Proficiency in Azure and AWS Data Technologies: Skilled in Azure Data technologies such as Data Bricks, Py Spark, Azure Functions, Azure Delta Lake, Azure Synapse DW, and AWS Data technologies including EC2, EMR, S3, and Glue crawlers. • Knowledgeable in Airflow and Snowflake DW: Well-versed in Airflow for workflow management and Snowflake Data Warehouse for modern data analytics needs. • Extensive Experience in Data Bricks and Delta Lake: Over 5 years of relevant work experience in Data Bricks with Python, Py Spark, and Azure Synapse DW, and 4+ years in Data Factory and Delta Lake. • Developed NiFi flows to extract data from diverse source systems such as MSDB, Yuga Byte, Dynamo DB, Postgres, Maria DB, MySql, db2, Oracle, and Hive, streamlining data ingestion workflows. • Proficient in CI/CD with Bitbucket and GitHub: Skilled in implementing Continuous Integration and Continuous Deployment (CI/CD) pipelines using Git platforms like Bitbucket and GitHub, ensuring efficient software delivery. • Development Expertise in Control-M: Experienced in the development of job drafts in Control-M for efficient job scheduling and workflow automation.