Job Title: Java Lead Developer (Spark, Azure, Redpanda Expert)

Experience: 12+ Years

Location: Remote

Contract Basis

Job Description:

We are seeking a Senior Lead Java Developer with expertise in Apache Spark, Azure Data Services (ADLS, Blob Storage, Databases), and Redpanda. The ideal candidate will lead the design and development of scalable data pipelines using Java and Spark in a cloud-based environment.

Key Responsibilities:

• Architect & develop distributed data processing systems with Apache Spark and Java.
• Design real-time & batch data pipelines integrated with Azure Data Services.
• Implement Redpanda (Kafka-like) integration for streaming data ingestion.
• Optimize Spark jobs for performance, scalability, and fault tolerance.
• Ensure seamless CI/CD, monitoring, and observability using Prometheus, Grafana, Azure Monitor.
• Lead and mentor a team, ensuring best practices in Java, Spark, and cloud-native development.

Required Qualifications:

• 13+ years in software development with strong Java (JVM-based) expertise.
• 5+ years working with Apache Spark for data processing.
• Extensive experience with Azure Data Lake, Blob Storage, and Databases.
• Strong expertise in distributed processing, streaming frameworks, and Redpanda/Kafka.
• Experience with Spark performance tuning, partitioning, and storage optimization.
• Hands-on with CI/CD, Docker, Kubernetes, and DevOps practices.
• Strong understanding of cloud security best practices.

Preferred Qualifications:

• Experience with Iceberg tables, Parquet formats, and real-time streaming.
• Knowledge of Spring Boot, microservices, and cloud deployments.

If you are passionate about data engineering, distributed systems, and cloud technologies, apply now!