Job Description
Data pipeline Automation Engineer(Test module lead with experience 5-9 yrs)
Build Python based AWS data pipeline validation framework. Should be able to automate the below operations using the framework -
Execution of SQL queries on AWS Athena .
Uploading json files into AWS S3.
Verify job status of AWS lambda function and glue jobs.
Compare the data in json files and AWS Athena tables.
Mandatory Skill sets - AWS Data pipeline, Python Automation scripting, SQL
AWS Data Lake, Python scripting, SQL
