Job Responsibilities:

 Responsible for designing, developing, and maintaining data pipelines and ETL processes

using Databricks and Azure Data Factory.

 Good experience in using SQL, Python and Spark within Databricks notebooks for data

processing and transformations.

 Integrate and orchestrate data from various sources, ensuring data quality and accuracy.

 Build reusable rules for Data Quality and required transformations.

 Optimize, monitor data pipelines for performance and scalability and debug in case of issues.

 Proficiency in Databricks, Azure Data Factory, Python, SQL is must.

 Familiarity with ETL principles, data warehousing, and big data technologies.

 Evaluate and define technical specs using BRD for BI and DW solutions.

 Build conceptual and logical models based on the functional flow of business in scalable way.

 Work with Business functions/ leadership and Application SMEs to understand the

requirement and analysing the source to fulfil the requirements.

 Be compliant to data quality, governance, compliance, and other legal requirements.

 Act as key P.O.C between data analyst, data scientists, and the business/application teams.

 Must have Azure Databricks experience & Data Integrations.

 Good work experience on Data bricks, ADF, and other Azure and Databricks objects.

 Experience on creating frameworks towards building the optimized data pipelines is a must.

 Working knowledge of different self-hosted and other runtimes along with SSIS.

 Strong work experience of SQL, Scala, Python and Spark.

 Performance Tuning of Spark SQL running on DELTA LAKE storage and Strong Knowledge on

Databricks and Cluster Configurations.

 Working knowledge of Data Warehousing, Modelling, Data Validation, Data Migration.

 Documentation of all project resources and follow best practices for development.

 Knowledge of front-end tools like Qlik Sense, MS PBI

 Minimum bachelor’s degree in computer science or related field is required.

 Strong problem-solving skills, good communication abilities and go-getter attitude.