Nandan G.

Background

About Nandan G.

Experience doing ELT operations on large datasets of varied structure and type by using Hadoop, Hive, Spark and Kafka.* Have done both batch processing(Hadoop and Spark) and real-time stream processing of data(Spark).* Experience with querying structured data from relational databases using SQL. Experience working with structured, semi-structured and unstructured data in NoSQL databases like MongoDB, Cassandra, HBase and Redis.* Experience with Linux Administration, Cloud architectures and fundamentals, processing data on AWS EC2 and storing data on AWS S3.* Proficient with core Python and frameworks such as numpy, pandas, Tensorflow, matplotlib, scipy.* Proficient with core Java, and features such as Generics, Collections, Multi-threading, Serialization, I/O, Design patterns, Object-Oriented Design.* Pursuing master degree from Stockholm University focusing on data science. Doing thesis in the field of vehicular prognostics using generative deep learning models.* Have 4 years of fintech industry experience in identifying relevant financial data from sources on the web and gathering them for ingestion into the application, using web scraping tools like Selenium Web-driver API on Java.* I have a growing curiosity in the fields of big data, analytics and generative AI. Experience has given me the virtues of perseverance, attention to detail and a quality-first approach to work.* Areas of experience in the data pipeline->Data fetch: Gathering structured data through web-scraping, validating and verifying the quality, and then inserting into local RDBMS->Data ingestion: Ingesting data from a local RDBMS or Hive into Spark using relevant PySparkSQL API->Data load: Loading data into HDFS for Hadoop based processing, or HDFS+Spark processing->Data transform: Transforming large structured/ unstructured datasets of varied types and sizes into smaller structured datasets by either using MapReduce API / Hive (for Hadoop processing) or standalone PySpark / PySparkSQL API (for Spark processing)->Data analysis: Statistical data analysis and visualization using Python numpy, pandas, matplotlib, scipy libraries, performing descriptive, inferential and predictive analysis on structured data. Applying machine learning algorithms on structured datasets in order to perform un-supervised/supervised learning operations and measuring the efficiency of the trained models by calculating accuracy.