We&aposre looking for Data Engineer who helps implementing methods to establish data model and improve data reliability and quality, combining raw information from different source to create consistent and machine-readable formats. and also developing and testing architectures that enable data extraction and transformation for analytics and modeling.
- Design, create, maintain and improve ETL and data pipeline to support entire cycle of data usage for data analytics team
- Create and maintain data catalog, data linkage and data life cycle for data analyst team
- Create and improve automation entire data process from data injection, ETL orchestration to model deployment
- Analyze problem and find resolution, as well as work closely with administration to monitor performance and advise any necessary infrastructure changes
- Work with business domain experts, data scientist and developers to identify relevant data for analysis
Qualification
- At least 4 years experience for Data Engineer in Data warehouse
- Strong skill i.e. SQL, Python,
- Experience working with varies forms of data infrastructure inclusive of relational database such as SQL, Oracle
- Experience with orchestration tools e.g. batch and real-time data processing
- Experience with CICD pipeline data and machine learning model deployment
- Knowledge in ETL and data pipeline framework, data patterns such as structured, semi structured, understand data in various type of database and data store such as Hive, HBase, Impala, MongoDB, Delta Lake, graph database using batch and streaming mechanism leveraging on-premise and on cloud Big data architecture
- Knowledge in data science fundamental such as database, operation system, algorithms, data structures, etc
- Interest leaning and working with new technology stack
