Search by job, company or skills

AWS, Pyspark

Cognizant Consulting
6-12 Years
Chennai, India
Early Applicant
  • Posted 12 days ago
  • Be among the first 10 applicants

Job Description

Skill: Pyspark, AWS

Exp: 6 to 12 years

Location: Pune

Job Summary

We are seeking an experienced AWS PySpark Developer to design, develop, and optimize large-scale data processing pipelines in the cloud. The ideal candidate will have strong expertise in PySpark, AWS services, and ETL workflows, with a focus on building scalable, high-performance data solutions for analytics and reporting.

Key Responsibilities

  • Design, develop, and maintain ETL pipelines using PySpark on AWS.
  • Work with large datasets to perform data cleansing, transformation, and aggregation.
  • Optimize Spark jobs for performance and cost efficiency.
  • Integrate data from multiple sources including AWS S3, RDS, Redshift, DynamoDB, and APIs.
  • Implement data quality checks and ensure compliance with governance policies.
  • Collaborate with data analysts, data scientists, and business stakeholders to deliver data solutions.
  • Deploy and monitor jobs using AWS Glue, EMR, Step Functions, or Lambda.
  • Troubleshoot and resolve issues in production data pipelines.
  • Maintain documentation for data flows, transformations, and architecture.

Required Skills & Qualifications

  • 3–7 years of experience in data engineering or big data development.
  • Strong proficiency in PySpark and Spark SQL.
  • Hands-on experience with AWS services such as:
    • S3, Glue, EMR, Lambda, Athena, Redshift, IAM
  • Solid understanding of ETL concepts and data warehousing.
  • Proficiency in Python for scripting and automation.
  • Experience with version control (Git) and CI/CD pipelines.
  • Strong problem-solving skills and ability to work in an agile environment.
Preferred Skills

  • Experience with AWS Glue Studio and Glue Catalog.
  • Knowledge of Airflow or other orchestration tools.
  • Familiarity with streaming data using Kafka or Kinesis.
  • Exposure to Terraform or CloudFormation for infrastructure as code.
  • Understanding of data security and encryption in AWS.

More Info

Job Type:
Permanent Job
Industry:
Other
Function:
Data Engineering
Employment Type:
Full time

Job ID: 151729367

Last Updated: 29-07-2026 05:53:10 PM
Homejobs in ChennaiAWS, Pyspark

Similar Jobs

AWS, Pyspark

Cognizant Softvision
Chennai, India

Skills:

Spark SQL, Git, Pyspark, AWS Glue, Python, Aws S3, AWS, Etl

Python, Pyspark, AWS

Cognizant Consulting
Chennai, India

Skills:

S3, Pyspark, Emr, Redshift, Sql, Lambda, Git, Kinesis, Python, AWS, Glue, Athena

Lead Software Engineer (AWS, Snowflake & ML Backend)

caterpillar inc.
Chennai, India

Skills:

snowflake , Python, AWS

Senior Pyspark Developer

P.R.Glolinks Consulting Private Limited
P
Bengaluru, Chennai

Skills:

Pyspark, Python

Early Applicant

Java FSD + AWS

Virtusa
V
Chennai, India

Skills:

Java, Spring Boot, AWS, Full Stack Development

Beware of Scammers

We don’t charge money for job offers

Know More