
Data Engineer (5+ years)

Data Engineer (5+ years)
5
Applications
About the Job
Skills
🔹 1. Data Architecture Design
· Design end-to-end data architectures using Azure services (e.g., Azure Synapse Analytics, Azure Data Lake, Azure SQL Database, Cosmos DB).
· Define data models, data flows, and integration patterns aligned with business and technical requirements.
· Ensure solutions support scalability, reliability, and performance goals.
🔹 2. Data Integration and ETL/ELT
· Design and oversee data ingestion pipelines using tools like Azure Data Factory, Synapse Pipelines, or Databricks.
· Establish processes for data transformation, cleansing, and enrichment.
· Integrate structured, semi-structured, and unstructured data from multiple sources (on-premises and cloud).
🔹 3. Data Storage and Management
· Choose appropriate storage solutions (e.g., Data Lake Gen2, Blob Storage, SQL, Cosmos DB) based on use cases.
· Define data retention policies, partitioning strategies, and lifecycle management.
· Optimize storage performance and cost.
🔹 4. Data Governance and Security
· Implement data governance frameworks, including data lineage, cataloging, and metadata management (e.g., Azure Purview / Microsoft Purview).
· Ensure compliance with GDPR, HIPAA, and other regulations.
· Define and implement data security, encryption, and role-based access control (RBAC) in Azure.
🔹 5. Analytics and BI Enablement
· Architect analytical environments for Power BI, Azure Synapse, and Databricks.
· Enable real-time and batch analytics.
· Ensure that data is accessible and trustworthy for reporting and machine learning.
🔹 6. Collaboration and Stakeholder Management
· Work with data engineers, analysts, scientists, and business stakeholders to understand data requirements.
· Translate business goals into technical data strategies.
· Provide architectural guidance to cross-functional teams.
🔹 7. Performance Optimization
· Monitor and optimize query performance, data pipelines, and storage costs.
· Implement data partitioning, caching, and indexing strategies.
· Use Azure Monitor, Log Analytics, and other tools for system health and performance tracking.
🔹 8. Cloud Strategy and Best Practices
· Develop and maintain a data strategy aligned with overall Azure cloud architecture.
· Stay current on Azure updates and industry trends to recommend improvements.
· Promote DevOps and CI/CD practices for data pipeline deployment and management.
🔹 9. Documentation and Standards
· Create and maintain architecture diagrams, design documents, and data dictionaries.
· Define naming conventions, coding standards, and documentation best practices for data teams.
🔹 10. Mentorship and Leadership
· Guide and mentor data engineers and developers in best practices for Azure data solutions.
· Provide architectural oversight during project execution.
· Conduct code and design reviews to ensure consistency and quality.
About the company
Industry
Staffing and Recruiting
Company Size
2-10 Employees
Headquarter
Bangalore
Other open jobs from Kiash Solutions LLP
