Akash Baidya

Background

About Akash Baidya

I specialize in designing and delivering data-intensive applications across the Hadoop ecosystem, cloud data engineering, data warehouses, and analytics platforms. My experience spans end-to-end data solutions, from ingestion and transformation to visualization, reporting, and data quality management. I have a strong foundation in Hadoop architecture, including YARN, HDFS, and the core components that drive distributed computing. Over the years, I’ve built enterprise-level solutions using technologies such as Apache Spark, Hive, HBase, Sqoop, Pig, Oozie, Flume, NiFi, Kafka, Zookeeper, and more. My expertise extends to data ingestion pipelines, enrichment, and aggregation processes, along with performance optimization. I’ve leveraged Spark (PySpark and Scala) to improve the efficiency of existing algorithms, applying tools like Spark SQL, DataFrames API, Spark Streaming, MLlib, and Pair RDDs to handle complex workloads at scale. I’m passionate about turning raw data into actionable insights and continuously improving performance in large-scale data environments.