JD and company profile Data Consultant - Spark Developer with AWS position Gurgaon location:

•         Position Title : Data Consultant - Spark Developer with AWS
•         Mandatory Skills: Experience working on Big Data Consultant with PySpark/Scala or Java, AWS Glue, EMR and Lambda, Cloudera Data Platform (CDP) Rest as per below JD.
•        Exp – 7 to 12 years
•         Qualification : B Tech/BE
•         Job Location – Gurgaon (Remote during pandemic)
•        Interview procedure:–Video (MS Teams) interviews in weekdays. (Google & MS team )
•         Shift Timing : NA

JOB DESCRIPTION

A passionate Big Data Consultant with PySpark/Scala or Java 

Desired Candidate Profile

- Strong Programming skill in Spark, Scala or Java

- must have implemented solutions on AWS Glue, EMR and Lambda

- worked with s3 based data lakes, crawlers and Athena for data exploration

- integration with various source systems for data ingestion pipelines

- Experience with Impala, Kafka or other streaming solutions is good to have

- JIRA, GitLab, Jenkins is a must

- IntelliJ IDEA, Eclipse

- Strong knowledge in Cloudera Data Platform (CDP)

Roles and Responsibilities

- Hands on Spark/Scala programming experience

- Participating in agile development projects for batch and real time data ingestion.

- Integration with AWS, Real-Time Systems, Data Warehouses, and Analytics solutions.

- Working with business and peers to define, estimate, and deliver functionality.

- Produce unit tests for Spark transformations and helper methods

- Design data processing pipelines

- Good in writing Scaladoc-style documentation with all code