Azure - Lead Data Engineer

We are looking for a highly skilled Lead Data Engineer with expertise in Azure, Python, Databricks, and ETL to drive data engineering initiatives. The ideal candidate will have strong leadership experience, hands-on technical skills, and a deep understanding of cloud-based data solutions.

Key Responsibilities:

• Lead the design, development, and implementation of scalable data pipelines using Azure Data Services, Databricks, and Python.
• Architect and optimize ETL workflows to ensure data integrity and efficiency.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and provide robust solutions.
• Manage and mentor a team of data engineers, providing guidance on best practices and performance optimization.
• Implement and enforce data governance, security, and compliance standards.
• Troubleshoot and resolve data pipeline issues, ensuring high availability and reliability.
• Optimize data storage, transformation, and retrieval for large datasets in a cloud environment.

Required Skills & Experience:

• 8+ years of experience in Data Engineering, with at least 2+ years in a lead role.
• Strong hands-on experience with Azure Data Services (Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake).
• Expertise in Databricks and Spark for big data processing.
• Proficiency in Python for data manipulation and pipeline development.
• Deep understanding of ETL/ELT processes and best practices.
• Experience with CI/CD pipelines and DevOps practices for data engineering.
• Strong knowledge of data modeling, warehousing concepts, and performance tuning.
• Excellent problem-solving, analytical, and communication skills.