Senior Data Engineer Job Description
In this role, you will be responsible for optimizing and managing data flow architecture, implementing and maintaining data pipelines, and ensuring data integrity and quality. You will also collaborate with cross-functional teams to define and implement data processing systems and build scalable data integration solutions.
Responsibilities
- Designing, building, and maintaining scalable data pipelines and data processing systems.
- Collaborating with data science and analytics teams to understand their data requirements and develop data solutions.
- Implementing and maintaining efficient data storage and retrieval systems.
- Ensuring data integrity, availability, and performance through monitoring and optimization.
- Developing and implementing data governance and security measures.
- Identifying and resolving data quality issues and data-related problems.
- Keeping up to date with the latest trends and technologies in data engineering and data management.
Qualifications
- Bachelor's degree in Computer Science, Engineering, or a related field.
- Proven experience as a Data Engineer or in a similar role.
- Strong programming skills in languages like Python, Java, or Scala.
- Experience with big data technologies and frameworks such as Hadoop, Spark, or Kafka.
- Solid understanding of data modeling and database management systems.
- Proficiency in SQL and data querying.
- Experience with cloud platforms (e.g., AWS, Azure, GCP) and related services.
- Knowledge of data governance and security principles.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration abilities.
Skills
- Data pipeline development
- Big data technologies (Hadoop, Spark)
- Database management systems
- SQL and data querying
- Cloud platforms (AWS, Azure, GCP)
- Data governance and security
- Programming languages (Python, Java, Scala)