Lead Data Engineer
What You Will Deliver
- Lead and Mentor: Lead, manage, and mentor a team of 3-6 data engineers, fostering a collaborative and high-performing environment.
- Technical Vision & Ownership: Own and define the technical vision, providing architectural guidance and best practices for data modelling, ETL/ELT processes, data warehousing, and data lake solutions.
- Collaboration & Stakeholder Management: Work closely with Product Owners, Business Analysts, and other stakeholders to understand requirements, translate them into technical specifications, and ensure successful delivery of data solutions.
- Hands-on Development: Be a hands-on contributor to the design, development, and maintenance of scalable data pipelines and platforms, leveraging a modern data stack.
- Data Quality & Testing: Implement and champion robust testing strategies for data pipelines, including unit testing, integration testing, and data quality checks, to ensure accuracy, reliability, and completeness of data.
- Extensive Data Engineering Experience: Proven experience as a Senior or Lead Data Engineer, with a strong track record of designing, building, and maintaining complex data pipelines and platforms.
- Technical Expertise:
- Languages: Strong proficiency in Python and SQL.
- Cloud Platform: Hands-on experience with AWS services (EMR, Athena, Lambda, S3, Glue, Step Functions).
- Data Warehousing: In-depth experience with Snowflake.
- Big Data Processing: Experience with Spark and familiarity with Scala (even if limited).
- Orchestration: Solid experience with Airflow for workflow orchestration.
- Data Transformation: Hands-on experience with DBT (Data Build Tool).
- Version Control: Proficient with Git and GitHub, including experience with GitHub Actions for CI/CD.
- Leadership & Management:
- Demonstrated experience in leading, mentoring, and managing a team of data engineers.
- Ability to motivate and inspire a team, fostering a culture of continuous learning and excellence.
- Strong communication and interpersonal skills, with the ability to effectively collaborate with technical and non-technical stakeholders.
- Real-time data processing
- Data governance and data quality frameworks
- Data operations and observability