Data Engineer

Responsibilities:

• Work with stakeholders throughout the organisation to identify opportunities for leveraging company data to drive business solutions
• Assess the effectiveness and accuracy of new data sources and data gathering techniques
• Create and maintain optimal data pipeline architecture
• Assemble large, complex data sets that meet functional / non-functional business requirements
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability etc
• Develop custom data models and algorithms to apply to data sets

• Bachelor's degree in Computer Science, Applied Mathematics, Information Technology, Information Systems, Statistics, Engineering, or any other technology related field
• Experience working in Banking & Financial industry more than 5 years
• Knowledge of software architectural and development Oracle, SQL, Big data Technology and Python Programming
• Core understanding of EDW and Data Lake architecture
• Understanding of data flows, data architecture, ETL and processing of structured and unstructured data.
• Knowledge in Hire purchase application
• Experience processing large amounts of structured and unstructured data.
• Understanding of programming languages such as Java, SQL, SAS, Python.
• Experience with as many tools as possible in Hadoop eco system and platforms (Apache, Hadoo, Sqoop, Air Flow, Flume, Kafka, Pig, Hive, HBase, Spark, Ambari, ZooKeeper, Oozie, HAWQ, Mahout, MapReduce, HDFS, YARN)
• Programming knowledge to clean datasets
• Working knowledge of message queuing, stream processing, and highly scalable big data data stores
• Strong project management and organizational skill
• Experience supporting and working with cross-functional teams in a dynamic environment
• Strong coding skills in Python (preferred)