Senior Data Engineer

Job Description:

• Analyze complex business requirements and design scalable, high-quality data solutions that align with long-term data platform strategy.
• Design and implement highly scalable, efficient data pipelines using data lake/data lakehouse architectures.
• Define, develop, and maintain data validation and data quality frameworks across technical and business rules.
• Lead the design and implementation of key data management services (data quality, monitoring, security, governance) within the DataOps framework.
• Monitor and ensure reliability, scalability, and performance of data pipelines and data platform clusters (e.g., Kubernetes, container orchestration).
• Collaborate effectively with cross-functional teams (SRE, Security, Data Science, AI/ML Engineering, Business teams, and leadership).
• Lead large-scale or complex projects end-to-end, either as a technical expert or as a project lead managing multiple contributors.
• Define engineering standards, best practices, design patterns, and development guidelines for the data engineering team.
• Review designs and solutions from Mid-level and Junior team members to ensure quality and alignment with standards.
• Mentor, coach, and guide team members to support their technical and professional growth.
• Influence key stakeholders and communicate complex technical concepts to both technical and non-technical audiences, including executives.

Qualifications:

• 5+ years of experience designing and implementing large-scale or complex data pipelines.
• Extensive experience with cloud platforms (GCP preferred).
• Strong expertise in Apache Spark, Apache Airflow, and distributed data processing.
• Proven experience leading or managing multiple concurrent projects.
• Strong proficiency in SQL and Python with deep understanding of software engineering best practices.
• Experience with containerization and orchestration technologies (Docker, Kubernetes, Helm).
• Strong understanding of coding standards, testing methodologies, and code review processes.
• Growth-oriented mindset and ability to adopt new tools, frameworks, and technologies.
• Understanding of data management concepts (data lineage, observability, data mesh, data governance, data security).
• Excellent communication, documentation, and stakeholder management skills.