Dipanjan Bhui

Background

About Dipanjan Bhui

Competent and dynamic Software Professional, offering a progressive career of 12 years across Software and Database Development • Worked with Hadoop Ecosystem components like Hadoop Map-Reduce, HDFS, HBase, Hive, Spark, Sqoop, Zookeeper, Airflow. •. Developed extensive Pipelines using Azure Synapse and ADLS to load TB's of data leading to a 30% increase in workflow efficiency. • Created E2E Data Pipelines starting from Ingestion into HDFS from Oracle DB via Sqoop, processing the data using Apache Spark (Transform, aggregate, join) and loading the processed data into Redshift/S3. • Hands on Experience in AWS (EC2, EMR, Athena, S3, Redshift, Glue). • Hands on Experience in Python and Scala. • Worked with Collections, Packages, Procedures, Functions, Triggers, Indexes, hints and other performance enhancements features. • Proficient in writing high performing SQL Queries. • Worked on Elastic Search with Logstash Configuration. Includes creation of Indexes, Logstash Configuration file, Incremental Run for delta records and loading data into Elastic server. • Acquired hands-on on basic commands and scripting in UNIX. • Acquired working knowledge on VB and VB.NET with LinQ and other.NET Concepts. • Basic knowledge on PostgreSQL and MySQL.