Saravanan Madhavan

Background

About Saravanan Madhavan

Around 9 years of experience in software industry having worked in various technologies and Hadoop systems including Unix shell scripting,Informatica,Oracle SQL, Sqoop,Hive,Spark,Scala,Autosys,Pig. Experience in both Banking/insurance domain (Have worked with BNP Corporate Banking/ State Farm Inc.). Good hands on experience on linux systems and data ingestion. Good understanding of storage layer and processing layer of Hadoop framework. Good knowledge of data processing engines like Spark. Good working knowledge of Spark core (RDD) and Spark SQL (Data frames). Developed spark scripts to process the datasets in HDFS as storage layer. Developed Sqoop jobs to transfer data between Oracle SQL and HDFS. Developed managed and external Hive tables improvising the tuning options using functions such as Partitioning, Bucketing,Index,Joins. Good working knowledge on ETL/Data warehousing Solutions. Deep understanding on various methods to tune and optimize Hive performance. Developed UDF for various business requirement. Good understanding of Autosys scheduling and monitoring tools. Experienced working in Agile and Waterfall delivery methodologies.