I
Aws Developer
I
- Posted 3 days ago
- Be among the first 10 applicants
Job Description
- Design, develop, and maintain ETL/ELT pipelines using AWS Glue.
- Develop AWS Glue jobs using PySpark, and SQL for large-scale data processing.
- Build data ingestion and transformation pipelines across structured and semi-structured data sources.
- Configure and manage AWS Glue Data Catalog, Jobs, Workflows, and Triggers.
- Work with AWS services such as Amazon S3, Redshift, and IAM.
- Implement data transformations, validations, data-quality checks, and reconciliation processes.
- Optimize Glue/Spark jobs for performance, scalability, reliability, and cost efficiency.
- Implement incremental loads, partitioning strategies, error handling, restart/recovery, and audit frameworks.
- Troubleshoot ETL pipeline failures and production performance issues.
- Implement appropriate security controls including IAM roles, encryption, and access management.
- Support migration of legacy/mainframe or on-premises ETL workloads to the AWS cloud data platform.
- Develop automated deployment processes using CI/CD and Infrastructure-as-Code practices where applicable.
- Collaborate with data architects, analysts, business teams, and cloud/platform teams to translate requirements into scalable data solutions.
- Prepare technical design documents, mapping specifications, operational documentation, and deployment/runbook documentation.
- AWS Glue
- Python/PySpark / Apache Spark
- SQL
- Amazon S3
- Data Lake / Cloud Data Warehouse concepts
- ETL/ELT design and development
- Performance optimization and troubleshooting Experience
- 7+ years of overall Data Engineering / ETL development experience.
- Strong hands-on experience developing production-grade AWS Glue,Python and PySpark solutions.
- Experience handling high-volume datasets and complex transformations in AWS environments. Preferred Certification
- AWS Certified Data Engineer – Associate or other relevant AWS certification.
More Info
Key Skills
ELT design and development
Cloud Data Warehouse concepts
About Company
Skills:
Amazon Redshift, Pyspark, AWS Glue, Python, Sql, ELT, Etl, Distribution Keys, RA3 nodes, Redshift Spectrum, Sort Keys
Skills:
amazon dynamodb , Java, Aws Lambda, CSS, PostgreSQL, Npm, Nosql, Jenkins, React, Terraform, AWS CloudFormation, Python, Api Gateway, Third-party API integrations, Event-driven patterns, AWS Cognito, boto3, Serverless Software Architecture, AWS Secrets Manager
Skills:
snowflake , Java, RDS, PostgreSQL, Spring Boot, J2EE, Sql, Angular, Microservices, Quicksight, React, Mockito, JUnit, Python, AWS
Skills:
snowflake , Java, Integration Testing, Agile Methodologies, PostgreSQL, Dynamodb, Spring Boot, Unit Testing, Microservices, Automated Testing, Restful Web Services, Docker, MySQL, Swagger, Software Development Life Cycle, Postman, Troubleshooting, Modern AI-assisted development tools, Application performance monitoring, CI/CD pipelines, GitHub branching strategies
Skills:
Java, Devops, AWS CloudFormation, Amazon CloudWatch, AWS CodePipeline, Python, Go, AWS Systems Manager, Application performance monitoring, Container orchestration, Chaos engineering, Observability solutions