Big Data Developer

Big Data Engineer

Location: Remote (UK-based)

Rate: £400/day (Inside IR35 – Umbrella)

Duration: Initial 3 months

About the Role

We are looking for a proactive Big Data Engineer to support the operation, maintenance, and enhancement of our critical data pipelines and enterprise datasets.

This contract role blends production support, pipeline engineering, automation, and data quality management. You will play a pivotal role in ensuring the reliable, timely delivery of data products, resolving incidents, and supporting platform modernisation initiatives.

Key Responsibilities

• Production Data Operations: Own, monitor, and execute end-to-end production data pipelines (weekly, monthly, quarterly). Perform data controls, reconciliations, and validations.
• Data Engineering & Automation: Enhance pipelines for schema/source changes using SQL and Python. Investigate failures, automate manual processes, and support data migration projects.
• Incident & Quality Management: Perform root-cause analysis on pipeline failures, resolve data discrepancies, and manage recovery/re-run activities.
• Documentation: Build clear operational runbooks, SOPs, and process maps to close documentation gaps and ensure knowledge transfer.
• Stakeholder Support: Liaise with internal SMEs, vendors, and external data providers to support access, onboarding, and client requests within agreed SLAs.

Required Skills & Experience

• Tech Stack: Strong, hands-on SQL and Python experience.
• Pipeline Support: Solid experience supporting and troubleshooting production ETL/data pipelines.
• Data Quality: Proven expertise in data reconciliation, root-cause analysis, and incident management.
• Process Focus: Excellent operational documentation and process mapping skills.
• Mindset: Able to work independently, manage competing priorities, and communicate clearly with stakeholders.

Nice-to-Haves

• Prior experience in Financial Services, market data, reference data, or enterprise data platforms.
• Exposure to pipeline migration, automation, and data transformation programmes.

Success Measures

• High pipeline reliability and faster issue resolution.
• Measurable reduction in manual workloads through smart automation.
• Clear, robust runbooks and enhanced operational resilience.

How to Apply

If you are available to start on immediately and match the core criteria above, please apply directly with your latest CV or drop me a message!

#BigDataEngineer #DataEngineer #DataEngineering #Python #SQL #ETL