Responsibilities:
- Analyze business or analytical requirements and design effective data solutions to meet user needs.
- Design and implement scalable and reliable data pipelines using data lake/data lakehouse architectures.
- Develop and maintain robust data validation processes covering both technical and business rules.
- Independently own and deliver small to medium-scope data engineering projects with minimal supervision.
- Directly communicate with business users, Data Analysts, or Data Scientists to gather requirements and validate solution design.
- Contribute to the design and implementation of self-service tools within the data platform as part of the DataOps framework.
- Monitor, troubleshoot, and maintain the performance and stability of data pipelines and data platform clusters (e.g., Kubernetes).
- Participate in design reviews and collaborate with Senior DEs on architecture decisions.
- Support and guide junior team members through code reviews, pair programming, and knowledge sharing.
Qualifications:
- 12 years of experience designing and implementing data pipelines.
- Strong proficiency in SQL and Python (understanding of OOP concepts preferred).
- Experience with data warehouse, data lake, or data lakehouse architectures.
- Experience with cloud platforms (GCP preferred).
- Hands-on experience or strong understanding of Apache Spark and Apache Airflow.
- Good understanding of coding standards, testing practices, and version control (Git).
- Able to work independently, take ownership, and drive work forward with minimal supervision.
- Effective communication skills, especially in explaining technical concepts to less technical users.
- Experience with Kafka, Docker, or Kubernetes is a plus.
- Familiarity with data management concepts (data quality, lineage, observability, data mesh) is a plus.
Technology Innovation Lab & Decision Intelligence by CJ Express is the CEO Office at CJ Express Group Co., Ltd. for enabling innovation team for the betterment of SMEs and society.
- TILDI stands for Technology Innovation Lab and Decision Intelligence.
- We are building a technology & business platform of the future for the equality and betterment of the community & society.
