About the Role
We are looking for a highly skilled Senior Python AWS Cloud Engineer with strong expertise in Python, AWS Cloud, REST APIs, and Databricks. The ideal candidate will be responsible for building scalable backend services, developing cloud-native integrations, and enabling Databricks-to-HBase data integration. This role requires hands-on experience in distributed systems, observability, and multi-tenant architectures while ensuring high availability, scalability, and performance.
Key Responsibilities
- Design, develop, and maintain scalable backend applications using Python.
- Build, enhance, and maintain RESTful APIs for platform services and data integration.
- Develop, optimize, and manage Databricks workloads, notebooks, jobs, and clusters.
- Implement and support Databricks-to-HBase (HONI Connector) integrations.
- Design and maintain multi-tenant architectures ensuring secure data isolation and optimal performance.
- Implement observability solutions including monitoring, dashboards, metrics, logging, and alerting.
- Monitor application and infrastructure health, proactively resolving performance, reliability, and availability issues.
- Collaborate with Data Engineering, Platform Engineering, DevOps, and Cloud teams to deliver scalable solutions.
- Follow AWS best practices for security, scalability, reliability, and cost optimization.
- Participate in troubleshooting, performance tuning, and production support.
Required Skills
- 7–9+ years of hands-on experience in Python Backend Development.
- Strong experience with AWS Cloud services including:
- EC2
- S3
- IAM
- Lambda
- CloudWatch
- Other core AWS services
- Strong experience designing and developing RESTful APIs.
- Hands-on experience with Databricks, including:
- Apache Spark
- Notebooks
- Jobs
- Clusters
- Experience integrating Databricks with HBase or other big data platforms.
- Strong understanding of distributed systems and large-scale data processing.
- Experience implementing observability solutions using monitoring, dashboards, metrics, logging, and alerting tools.
- Experience designing and managing multi-tenant architectures.
- Strong debugging, troubleshooting, and performance optimization skills.
- Excellent problem-solving and communication skills.
Preferred Skills
- Experience with Apache HBase.
- Knowledge of cloud-native microservices architecture.
- Experience with CI/CD pipelines and DevOps practices.
- Familiarity with infrastructure automation and cloud monitoring tools.
