Ren Yang

Background

About Ren Yang

With over a decade of experience in data engineering, I specialize in designing and implementing scalable, cloud-native data solutions on AWS. My expertise covers the entire data engineering ecosystem, including data ingestion, transformation, DataOps, and data serving, ensuring robust and efficient data architectures for public sector organizations. AWS-Centric Data Engineering Expertise Data Ingestion & ETL on AWS I have extensive experience in real-time and batch data ingestion using AWS-native tools: • Streaming: Amazon Kinesis (Data Streams, Firehose) and AWS MSK (Managed Kafka) • Batch Processing: AWS Glue, AWS Data Pipeline, AWS Lambda-based ingestion • Database Migration: AWS DMS (Database Migration Service) for seamless cloud transitions Data Transformation & Processing As an AWS Certified Solutions Architect & Data Analytics Professional, I design and optimize high-performance ETL pipelines leveraging: • AWS Glue (PySpark, Scala) and AWS EMR (Apache Spark, Hive, Presto) for large-scale data transformations • Athena & Redshift Spectrum for efficient query execution on S3-based data lakes • Amazon Redshift for high-performance analytics with materialized views, RA3 nodes, and Spectrum DataOps & Automation To ensure scalability, reliability, and security, I focus on DataOps best practices, including: • CI/CD for data pipelines using AWS CodePipeline, AWS CodeBuild, and Terraform • Monitoring & Observability with AWS CloudWatch, AWS X-Ray, and AWS OpenSearch • Data Governance & Security using AWS Lake Formation, AWS Glue Data Catalog, and IAM best practices Data Serving & Analytics I design AWS-based data serving architectures tailored for public service organizations, enabling seamless data access and analytics: • Data Warehousing with Amazon Redshift and Redshift Serverless • BI & Reporting using Amazon QuickSight and Athena for self-service analytics • Data APIs & ML Integration via AWS Lambda, API Gateway, and SageMaker