Dhanush Reddy

Background

About Dhanush Reddy

5 + years of experience in IT data analytics as a Google Certified Professional Data Engineer. Practical knowledge of all the big data solutions offered by Google Cloud Platform (GCP), including Composer (Air Flow as a service), Cloud DataProc, Bigquery, and Google Cloud Storage. experience utilising bash scripting to write continuous integration tests (CI tests) for the Python Dags while publishing patches in BuildKite/Azure Devops. To keep up with the growing amount of data, SQL principles, Presto SQL, Hive SQL, Python (Pandas, NumPy, SciPy, Matplotlib), Scala, and Spark are used. Practical experience writing Shell/Bash scripts and constructing data pipelines on Linux and Unix platforms. Familiarity in managing source control systems, Subversion or GIT (Bitbucket/Gitlab). Knowledge of Agile methodology, JIRA, Confluence, and working in sprints. Expertise using Google Cloud Platform's orchestration tools, such as Kubernetes. Knowledge of utilising SQOOP in oozie scripts Many Hive SQL codes have been converted, if needed, into SPARK SQL and PySpark code. PL/SQL code was converted to a Python bigquery architecture, which was then used in DataProc for Azure Data Bricks and PySpark. Practical experience creating CI/CD Azure pipelines with the Microsoft agent, executing tests pertaining to Python files, and managing task scheduling and monitoring with Oozie and Zookeeper. Excellent knowledge of the Apache Hadoop ecosystem, including Spark, HDFS, MapReduce, Hive, Pig, Sqoop, and Presto.