Job description

5+ years of Data Engineering experience

Mandatory Skills : - Hive, Spark, Scala, Python, Linux, Shell scripting

• Experience in designing, implementation and maintenance of high-performance big data infrastructure, systems & processing pipelines scaling to billion records of structured and unstructured data.

•Technical expertise with data models, data mining, and segmentation techniques

•Must implement data pipelines to automate the ingestion, transformation, and augmentation of data sources, and provide best practices for pipeline operations

• Hands-on experience using Hive, Spark, Scala, Python, Linux, Shell scripting.

• Hands on experience with Spark Streaming or other live stream processing technologies would be a plus including building Flask and Rest APIs

• Any Cloud development (AWS, Azure, GCP) would be a plus and added advantage

• Oversee assigned programs and provide guidance to team members

• Assist with solving technical problems when they arise

• Explore new technologies and learn new techniques to solve business problems creatively.