Mithun Dama

Background

About Mithun Dama

PROFESSIONAL SUMMARY: Over 12+ years of IT experience across various industries, with a strong focus on Big Data, Hadoop, and Python development in cloud environments. Expertise in the Azure ecosystem, including services like Azure HDInsight, Azure Databricks, Azure Data Lake, Azure Synapse Analytics, Azure Blob Storage, Azure Data Factory, and Azure Data Explorer (ADX). Comprehensive knowledge of the Hadoop Ecosystem tools within Azure, including Spark, Hive, HDFS, and MapReduce, along with Azure-specific tools like Azure Synapse and Azure Data Lake Analytics. Proficient in designing, developing, and automating data engineering pipelines using Azure Data Factory, ensuring best practices in partitioning, file formats, and incremental loads. Extensive experience in developing production-ready data processing applications in Spark using Scala/Python on Azure Databricks. Skilled in working with Snowflake on Azure, leveraging multi-cluster and virtual warehouses for optimized data processing. Strong ability to create efficient Spark applications for various data transformations such as data cleansing, de-normalization, joins, and data aggregation on Azure. Experienced in fine-tuning Spark applications on Azure Databricks using techniques like broadcasting, shuffle parallelism, caching/persisting DataFrames, and executor sizing. Proficient in writing Python applications using libraries such as Pandas, NumPy, SciPy, and Matplotlib for data analysis on Azure. Knowledgeable in productionizing Machine Learning pipelines on Azure Databricks, including featurization, model training, scoring, and evaluation using Spark ML libraries. Familiar with the Agile software development process, particularly within cloud-based environments. Experienced in analyzing large datasets on Azure, discovering patterns and insights within both structured and unstructured data. Extensive experience with Hadoop distributions like Cloudera and Hortonworks, and specifically with Azure Databricks. Hands-on experience with NoSQL databases on Azure, including Azure Cosmos DB, HBase, Cassandra, and MongoDB. Skilled in writing complex MapReduce programs and working with various file formats, including Text, Sequence, XML, Parquet, and Avro, within Azure HDInsight. Proficient in using Azure Data Factory for orchestrating data workflows and managing Hadoop jobs using Azure's scheduling capabilities. Experienced in migrating data using Azure Data Factory and Azure Synapse from HDFS to Relational Database Systems and vice versa.