This job has expired

Data Pipeline Engineer (Bangkok based, relocation provided)

Not Specified
Not Specified

Job Description

Get to Know our Team
The Data department oversees all of Agoda's data-related requirements. Our ultimate goal is to enable and increase the use of data in the company through creative approaches and the implementation of powerful resources such as operational and analytical databases, queue systems, BI tools, and data science technology. We hire the brightest minds from around the world to take on this challenge and equip them with the knowledge and tools that contribute to their personal growth and success while supporting our company's culture of diversity and experimentation. The role the Data team plays at Agoda is critical as business users, product managers, engineers, and many others rely on us to empower their decision making. We are equally dedicated to our customers by improving their search experience with faster results and protecting them from any fraudulent activities. Data is interesting only when you have enough of it, and we have plenty. This is what drives up the challenge as part of the Data department, but also the reward.
The Opportunity
As a data engineer, you will work on distributed systems spanning multiple data centers, thousands of servers, and hundreds of billions of messages a day. Ensuring data quality, data integrity, and data accuracy is a core part of our identity. Design for the new scalable system in which data is increasing day by day, including auditing and monitoring systems. You will be eager to solve problems that come from managing and making sense of large amounts of data. Some of the things you'll get to work on include schema registry, real-time data-ingestion, cross data center replication, enrichment, storage, and analytics of the data flow.
We are a small passionate team, and we are looking for exceptional individuals to be a part of designing, building, deploying (and probably debugging) our Data Pipeline.
In This Role, You'll Get to
  • You will build, administerand scale data pipelines that process hundreds of billions of messages a day spanning over multiple data centers
  • You will develop and expand upon existing frameworks that is used by Teams throughout Agoda to produce messages to the data pipeline
  • You will build and manage data ingestion into multiple systems (Hadoop, ElasticSearch, other Distributed Systems)
  • You will build tools that monitor high data accuracy SLAs for the data pipeline
  • You will explore available new technologies that improve upon our quality of data, processes and data flow
  • You will develop quality software through design review, code reviews and test driven development

What You'll Need to Succeed
  • Bachelor's degree in Computer Science / Information Systems / Computer Engineering or related field
  • 2 plus years of industry experience, preferred at a tech company
  • Good knowledge of data architecture principles
  • Have operational experience debugging production issues
  • An experienced coder, who can stand your ground with experience building systems with purpose that are flexible, well-tested, maintainable and scale
  • You're detail oriented considering every outcome of a particular decision
  • You can communicate in technical English with fluidity, both verbal and written

It's Great If You Have
  • Good understanding of how Kafka works
  • Kafka Administrator Experience
  • Understands Concepts relating to Schema Registry and Schema Evolution
  • Experience working with Serialization Formats either with ProtocolBuffers, Avro or Thrift
  • Knows how to use ElasticSearch proficiently
  • Experience with data ingestion from Kafka into Hadoop, ElasticSearch, other Distributed Systems
  • Strongsystems administration skills in Linux
  • Worked on or contributed to Open Source Project

Job Details

Employment Types:




About Agoda

Job Source :

Similar Jobs

People Also Considered

Simple body text this will replace with orginal content