Rahul Shivhare

Background

About Rahul Shivhare

With over 10 years of experience in the IT industry, I specialize in DevOps, Big Data, Machine Learning, and LLMOps technologies. I have a proven track record of architecting scalable, secure, and production-grade platforms on AWS, leading cross-functional teams, and delivering end-to-end solutions across data engineering, MLOps, and GenAI initiatives. Professional Journey 10+ Years in IT Hands-on delivery across DevOps, Big Data, ML, and GenAI, building data platforms, AI pipelines, and container-native services. 5+ Years in Leadership Led cross-functional teams across DataOps, MLOps, and DevOps, bridging engineering, data science, and platform operations. 7+ Years in Data Pipeline Management Built and maintained production pipelines handling 5–10 TB of daily data via Hadoop, Spark, Kafka, Airflow, Athena, Glue, and Redshift. 5+ Years in ML & GenAI Implementation Designed ML platforms on SageMaker, Kubeflow, MLflow, and DVC; built LLMOps pipelines using LangChain, Bedrock, Qdrant, and LangGraph for RAG and multi-agent orchestration. 9+ Years in Kubernetes & Automation Architected Kubernetes-based microservices on EKS with Helm, Istio, and GitOps practices using ArgoCD, Terraform, and GitHub Actions. 9+ years in Observability & Governance Enabled full-stack observability with Prometheus, Grafana, Loki, Jaeger, and Elastic; enforced data governance with OpenMetadata, OpenLineage, and Lake Formation. Featured Projects DataGPT: Built a GenAI assistant using LangChain, Bedrock (Claude), Titan Embeddings, and Qdrant with Streamlit/Gradio chat UI. Athena Optimizer: Developed a LangChain-powered Athena interface using embeddings and Streamlit for query optimization. Technologies & Tools Big Data: Hadoop, Spark, Hive, Presto, Flink, Kafka, Airflow, Nifi, EMR, Athena, Glue, Redshift, Airbyte, Tableau DevOps & IaC: Terraform, Jenkins, GitHub Actions, ArgoCD, Ansible, Docker, Helm, GitOps, Linux, ECS, EKS ML & GenAI: SageMaker, Bedrock, Kubeflow, MLflow, TensorFlow, PyTorch, Hugging Face, DVC, Triton, KServe, Evidently AI LLMOps: LangChain, LangGraph, Amazon Titan, Qdrant, FAISS, Claude Models Observability & Security: Prometheus, Grafana, Jaeger, Loki, Elastic, Quickwit, Kyverno, Falco, Okta, AD Visualization & UI: Streamlit, Gradio, Jupyter Certifications: CKS + CKA 7x AWS Certification 1x Azure Certification