Senior Data Engineer

Responsibility

1. Develop and maintain ETL processes to extract, transform, and load data from various sources to target data repositories.
2. Work closely with the Data Architect to understand data models and ensure that ETL processes align with the overall data architecture.
3. Monitor and maintain data infrastructure to ensure data quality and accuracy, including implementing data security and privacy policies.
4. Provide technical support to end-users to help them access and utilize data effectively.
5. Work collaboratively with other members of the data team to ensure the effective and efficient use of data within the organization.
6. Develop and maintain documentation of ETL processes and data infrastructure

Qualification

1. Experience working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working with a variety of databases.
2. Experience building and optimizing big data data pipelines, architectures and data sets.
3. Build processes supporting data transformation, data structures, metadata, dependency and workload management.
4. A successful history of manipulating, processing and extracting value from large disconnected datasets.
5. Experience supporting and working with cross-functional teams in a dynamic environment.
6. Experience with object-oriented/object function scripting languages: Python, Java, Scala, C#, etc.
7. Experience in using data platform tools and their components: Databricks, Pyspark, Apache Airflow
8. Previous work designing/managing data structures within a data warehouse
9. Experience in data stream processing and its processor is a plus
10. Experience developing application eg. web application, mobile application, REST web services is a plus