Senior Data Engineer

Required Skills:

• Minimum 8+ years of experience in Data Engineering
• Strong hands-on experience with Apache Spark
• Proficiency in Scala and/or Python (PySpark)
• Good understanding of Spark internals (RDD, Data Frame, Dataset APIs)
• Experience working with data formats such as Parquet, Avro, and JSON
• Solid understanding of distributed systems and big data concepts
• Strong SQL skills
• Experience with AWS cloud services (S3, EMR, Glue, Kinesis, Firehose, Hive preferred)
• Knowledge of performance tuning and optimization techniques
• Experience with CI/CD pipelines
• Exposure to streaming frameworks such as Spark Streaming
• Familiarity with workflow orchestration tools like Apache Airflow

Preferred Skills:

• Familiarity with Agile development practices
• Strong analytical and problem-solving abilities
• Good communication and stakeholder collaboration skills
• Ability to work independently in a fast-paced environment
• Strong ownership mindset with focus on data quality and reliability