Python Developer (PySpark/Spark)

Key Responsibilities

· Design, develop, and optimize scalable data pipelines using Python and Apache Spark.

· Build high-performance data workflows with pushdown optimization, streaming, replication, partitioning, and clustering techniques.

· Evaluate and integrate modern data platforms and tools into our enterprise architecture.

· Engineer and manage feature pipelines to support real-time fraud detection systems.

· Design data models and processing strategies that align with distributed system principles, ensuring scalability, consistency, and performance across large-scale environments.

· Develop solutions that are production-ready, maintainable, and built with observability and operational excellence in mind.

· Apply clean code practices, SOLID principles, and architecture patterns to deliver robust and extensible systems.

· Participate in code reviews, testing, and deployment activities.

· Contribute to architectural decisions and continuous improvement initiatives.

Required Skills

· 5+ years of professional experience in Python development with a strong focus on data engineering.

· Hands-on expertise with Apache Spark and Big Data processing.

· Strong SQL skills and experience with both relational and distributed data systems.

· Solid understanding of software engineering principles, clean code, and design patterns.

· Experience in system design and architecture for scalable, data-intensive applications.

· Ability to evaluate new technologies and recommend solutions aligned with enterprise needs.

· Excellent analytical and problem-solving skills.

· Strong communication and collaboration abilities.