We are looking for a skilled
Data Engineer with 4+ years of experience in designing, developing, and maintaining scalable data pipelines using AWS data services. The ideal candidate should have hands-on expertise in
Amazon Redshift, Redshift Spectrum, Athena, AWS Glue, EMR, Kinesis, Lambda, and
JSON data processing. The role involves building efficient ETL/ELT solutions, optimizing data warehouses, and enabling analytics through cloud-native data platforms.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using AWS services.
- Develop and optimize data warehouse solutions using Amazon Redshift and Redshift Spectrum.
- Build serverless data processing workflows using AWS Lambda and Glue.
- Process and transform structured and semi-structured (JSON) data.
- Use Amazon Athena for interactive querying and analytics on data stored in Amazon S3.
- Develop real-time data ingestion pipelines using Amazon Kinesis.
- Work with Amazon EMR for large-scale data processing using Spark/Hadoop.
- Monitor, troubleshoot, and optimize data pipeline performance.
- Collaborate with data analysts, architects, and business teams to deliver high-quality data solutions.
- Ensure data quality, security, and governance best practices.
Required Skills
- 4+ years of experience in Data Engineering.
- Strong expertise in Amazon Redshift and Redshift Spectrum.
- Hands-on experience with AWS Glue, Athena, EMR, Kinesis, and Lambda.
- Strong SQL programming skills.
- Experience working with JSON and semi-structured data.
- Knowledge of ETL/ELT concepts and data warehousing.
- Experience with Amazon S3 and AWS IAM.
- Understanding of performance tuning and query optimization.
- Familiarity with Git and Agile methodologies.
Good to Have
- Experience with Python or PySpark.
- Knowledge of Apache Spark/Hadoop ecosystem.
- Exposure to CI/CD pipelines.
- AWS Certification is an added advantage.
Education
- Bachelor's degree in Computer Science, Information Technology, or a related field.
Mandatory Skills
- Amazon Redshift
- Redshift Spectrum
- AWS Athena
- AWS Glue
- Amazon EMR
- Amazon Kinesis
- AWS Lambda
- SQL
- JSON
- Amazon S3
Skills: kinesis,spectrum,athena,glue,lambda,amazon redshift
