Big Data Engineer

About the Role:

Synechron UK is seeking an experienced Lead Data Engineer to spearhead the development and evolution of our data engineering platforms. This leadership role requires a hands-on professional proficient in designing, building, and optimising enterprise-grade data solutions, with a focus on innovation, resilience, and regulatory compliance.

Key Responsibilities:

• Lead and develop robust data engineering platforms leveraging technologies such as Hadoop, Spark, and Splunk.
• Design, implement, and maintain scalable ETL/ELT data pipelines for diverse data types, including raw, structured, semi-structured, and unstructured data (SQL and NoSQL).
• Integrate large and disparate datasets using modern tools and frameworks to support analytical and operational needs.
• Collaborate effectively with BI and Analytics teams in dynamic environments, providing technical guidance and support.
• Develop, review, and maintain automated test plans, including unit and integration tests, to ensure high-quality, reliable code.
• Drive SRE principles within data engineering practices, ensuring service resilience, sustainability, and adherence to recovery time objectives.
• Support source control practices and implement CI/CD pipelines for continuous delivery of data solutions.
• Stay informed on current industry trends, regulatory requirements (cybersecurity, data privacy, data residency), and incorporate best practices into engineering processes.
• Represent Synechron in industry groups and vendor interactions to influence and adopt emerging technologies and standards.

Qualifications and Skills:

• Extensive experience with big data frameworks such as Hadoop, Spark, and Splunk.
• Strong scripting capabilities in Python, with experience in object-oriented and functional programming paradigms.
• Proven expertise in handling varied data formats and integrating large datasets across multiple platforms.
• Deep understanding of building, optimising, and managing complex ETL/ELT pipelines.
• Familiarity with version control systems and CI/CD tools.
• Experience working closely with BI and analytics teams, supporting data-driven decision-making.
• Excellent problem-solving skills with a data-driven mindset.
• Knowledge of agile methodologies (Scrum, Kanban).
• Ability to contribute in collaborative, fast-paced environments, including pair programming and team standups.
• Strong commitment to quality, test-driven development, and automation.