Senior Data Engineer (TILDI)

Job Description:

- Analyze complex business requirements and design scalable, high-quality data solutions that align with long-term data platform strategy.

- Design and implement highly scalable, efficient data pipelines using data lake/data lakehouse architectures.

- Define, develop, and maintain data validation and data quality frameworks across technical and business rules.

- Lead the design and implementation of key data management services (data quality, monitoring, security, governance) within the DataOps framework.

- Monitor and ensure reliability, scalability, and performance of data pipelines and data platform clusters (e.g., Kubernetes, container orchestration).

- Collaborate effectively with cross-functional teams (SRE, Security, Data Science, AI/ML Engineering, Business teams, and leadership).

- Lead large-scale or complex projects end-to-end, either as a technical expert or as a project lead managing multiple contributors.

- Define engineering standards, best practices, design patterns, and development guidelines for the data engineering team.

- Review designs and solutions from Mid-level and Junior team members to ensure quality and alignment with standards.

- Mentor, coach, and guide team members to support their technical and professional growth.

- Influence key stakeholders and communicate complex technical concepts to both technical and non-technical audiences, including executives.

Qualifications:

- 5+ years of experience designing and implementing large-scale or complex data pipelines.

- Extensive experience with cloud platforms (GCP preferred).

- Strong expertise in Apache Spark, Apache Airflow, and distributed data processing.

- Proven experience leading or managing multiple concurrent projects.

- Strong proficiency in SQL and Python with deep understanding of software engineering best practices.

- Experience with containerization and orchestration technologies (Docker, Kubernetes, Helm).

- Strong understanding of coding standards, testing methodologies, and code review processes.

- Growth-oriented mindset and ability to adopt new tools, frameworks, and technologies.

- Understanding of data management concepts (data lineage, observability, data mesh, data governance, data security).

- Excellent communication, documentation, and stakeholder management skills.