Role:
Kanerika is building out a dedicated Databricks delivery practice and is looking for a Practice Lead / Architect to be the founding technical leader of this group. This is a high-ownership, build-from-scratch role: you will define delivery standards, lead solution architecture for client engagements, support pre-sales and proposal efforts, and mentor a growing team of engineers. You'll be the senior-most technical voice for Databricks both internally and in front of clients.
Key Responsibilities
- Define and own the technical architecture, standards, and best practices for Databricks-based solutions across client engagements (data engineering, lakehouse, ML/AI workloads, and GenAI solutions including RAG pipelines and LLM-based applications).
- Lead solution design for greenfield implementations and migrations to Databricks, including Unity Catalog, Delta Lake (including Change Data Feed, liquid clustering, and deletion vectors), workflows, and governance models.
- Act as the primary technical lead in pre-sales: scope client requirements, build estimates, design proposed architectures, and present solutions in client-facing meetings and demos.
- Support RFP/RFI responses and SOW creation with accurate technical scoping and effort estimation for Databricks engagements.
- Build and maintain reusable accelerators, reference architectures, and delivery templates to speed up future client engagements.
- Provide technical leadership and mentorship to the delivery team, including code/architecture reviews and unblocking technical challenges.
- Partner with practice leadership to define hiring needs, skill development plans, and the long-term roadmap for the Databricks practice.
- Stay current with the Databricks platform roadmap (Unity Catalog, Lakeflow Declarative Pipelines (formerly Delta Live Tables) and Lakeflow Connect for managed ingestion, MLflow, Databricks SQL) and proactively bring new capabilities into client conversations.
- Serve as primary point of escalation for technical risk and delivery quality across active engagements.
Required Skills & Experience
- 8–14 years in data engineering/architecture roles, with at least 4 years of hands-on Databricks and Apache Spark experience in production environments.
- Deep expertise in PySpark/Spark SQL, Delta Lake, Unity Catalog, and Databricks workflow orchestration.
- Hands-on experience with Auto Loader and structured streaming pipelines in production environments.
- Strong experience designing lakehouse architectures, including ingestion, transformation (ETL/ELT), and serving layers.
- Hands-on experience with at least one major cloud platform (Azure, AWS, or GCP); Azure Databricks experience strongly preferred.
- Cluster design and performance tuning: instance pools, cluster policies, Adaptive Query Execution, and data skew handling.
- Proven experience in client-facing technical roles — solutioning, presenting architectures, and responding to technical objections.
- Experience with pre-sales activities: effort estimation, SOW/proposal input, and technical demos.
- Strong understanding of data governance and security on Databricks, including Unity Catalog privilege model, row-level and column-level security, secrets management, and audit logging.
- Excellent communication skills, with the ability to translate technical concepts for both engineering and business audiences.
- Prior experience setting up or scaling a practice, center of excellence, or technical team is a strong plus.
PEFFERED / NICE TO HAVE
- Databricks certifications (e.g., Databricks Certified Data Engineer Professional, Databricks Certified Solutions Architect).
- Experience with MLflow (experiment tracking, model registry, model serving endpoints), Lakeflow Declarative Pipelines and Lakeflow Connect for managed ingestion, Databricks SQL and AI/BI Genie for analytics use cases, Databricks Vector Search for RAG pipelines, and Mosaic AI for LLM fine-tuning and agent frameworks.
- Exposure to consulting/IT services delivery models and multi-client engagement management.
- Familiarity with complementary tools in the modern data stack (dbt, Airflow, Fivetran, Kafka, etc.).
What Success Looks Like
- Within 3 months: Delivery standards, architecture templates, and a reference solution are in place; actively supporting live pre-sales conversations.
- Within 6 months: First client engagement(s) delivered or in active delivery with the practice's standards applied; hiring plan for the broader team is executing.
- Within 12 months: Kanerika is recognized by clients and prospects as having a credible, mature Databricks delivery capability, with a repeatable pre-sales-to-delivery motion.
