Responsibilities
- Analyze complex business requirements and design scalable, high-quality data solutions that align with long-term data platform strategy.
- Design and implement highly scalable, efficient data pipelines using data lake/data lakehouse architectures.
- Define, develop, and maintain data validation and data quality frameworks across technical and business rules.
- Lead the design and implementation of key data management services (data quality, monitoring, security, governance) within the DataOps framework.
- Monitor and ensure reliability, scalability, and performance of data pipelines and data platform clusters (e.g., Kubernetes, container orchestration).
- Collaborate effectively with cross-functional teams (SRE, Security, Data Science, AI/ML Engineering, Business teams, and leadership).
- Lead large-scale or complex projects end-to-end, either as a technical expert or as a project lead managing multiple contributors.
- Define engineering standards, best practices, design patterns, and development guidelines for the data engineering team.
- Review designs and solutions from Mid-level and Junior team members to ensure quality and alignment with standards.
- Mentor, coach, and guide team members to support their technical and professional growth.
- Influence key stakeholders and communicate complex technical concepts to both technical and non-technical audiences, including executives.
Qualifications
- 5+ years of experience designing and implementing large-scale or complex data pipelines.
- Extensive experience with cloud platforms (GCP preferred).
- Strong expertise in Apache Spark, Apache Airflow, and distributed data processing.
- Proven experience leading or managing multiple concurrent projects.
- Strong proficiency in SQL and Python with deep understanding of software engineering best practices.
- Experience with containerization and orchestration technologies (Docker, Kubernetes, Helm).
- Strong understanding of coding standards, testing methodologies, and code review processes.
- Growth-oriented mindset and ability to adopt new tools, frameworks, and technologies.
- Understanding of data management concepts (data lineage, observability, data mesh, data governance, data security).
- Excellent communication, documentation, and stakeholder management skills.
Technology Innovation Lab & Decision Intelligence by CJ Express is the CEO Office at CJ Express Group Co., Ltd. for enabling innovation team for the betterment of SMEs and society.
- TILDI stands for Technology Innovation Lab and Decision Intelligence.
- We are building a technology & business platform of the future for the equality and betterment of the community & society.
