Walk-In Interview (Gurgaon) | Lead Data Engineer | 16th March'24

Job Description:

Big Data Centre of Excellence is building state-of-the-art Data Analytics and Management systems used across various Jio Platforms. Primarily we enable different businesses - ingest, process, manage and derive insights from data. We developed numerous data intelligence capabilities on top of these systems which communicate with each other and help businesses make informed decisions. Big Data COE encourages the usage of Open-Source tools/services to build robust and resilient systems that form the core for various AI/ML capabilities.

Responsibilities:

• Design, build, and maintain robust data pipelines and ETL processes to support data integration, transformation, and storage.
• Develop and implement data architecture solutions using technologies such as Kafka, Hadoop, Hive, PySpark, and Cloudera.
• Optimize and tune data pipelines for performance, reliability, and scalability.
• Collaborate with data scientists, analysts, and software engineers to understand data requirements and deliver actionable insights.
• Work closely with infrastructure teams to manage on-premises data infrastructure and ensure high availability and security.
• Implement best practices for data governance, data quality, and data security.
• Evaluate and recommend new technologies, tools, and frameworks to enhance data engineering capabilities.
• Provide technical leadership and mentorship to junior members of the data engineering team.

Qualifications:

• Bachelor’s or master’s degree in computer science, Engineering, or related field.
• 7+ years of experience in data engineering roles with a focus on building data pipelines and data infrastructure.
• Extensive hands-on experience with Kafka, Hadoop, Hive, PySpark, and Cloudera.
• Experience on On-premise
• Proficiency in Python programming language for data manipulation, scripting, and automation.
• Strong understanding of distributed systems, data modelling, and database concepts.
• Experience working with on-premises data infrastructure and cloud platforms (e.g., AWS, Azure, GCP).
• Streaming experience – Spark streaming, Flink.
• Extensive hands-on experience on coding – Java/Scala/Python
• Need to have experience on Linux/Scripting
• Strong understanding of Data Modelling
• Excellent problem-solving skills and ability to troubleshoot complex data issues.
• Strong communication and collaboration skills with the ability to work effectively in cross-functional teams.
• Proven track record of delivering high-quality solutions on time and within budget.

Address: -

Unitech Commercial Tower 1, Sector 45, Gurgaon https://maps.app.goo.gl/2js38aCM5BsAuGQ9A (Previously HSBC Bank Office)