1 to 25 of 580 Data Pipeline Jobs in the UK

leading daily standups, allocating and monitoring tasks. Lead technical delivery for a specific source system or domain, taking ownership of its data pipeline architecture. Promote coding standards, continuous improvement, and a safe, compliant working environment. Ensure a healthy, safe and secure working environment, ensuring compliance with legal … database optimisation) and translate complex information for non-technical audiences. Translate business requirements into technical data solutions and provide regular updates on pipeline status, risks, and improvements. Produce written technical reports to support informed decision-making. Communicate sensitive or complex information clearly, including data load ...

Data Engineer FTC – 12 Months – Coventry University of Warwick £47,389 - £56,535 (D.O.E) 42 Days Holidays (Inc Bank Hols) Excellent Pension Contributions Hybrid Working Great Food, Gym and Leisure Facilities on-site We are seeking a new addition for our Data Labs Team to assist … architects, system owners and end users to ensure quality issues are addressed as needed. Documentation ▪ Create and maintain technical documentation, including data pipeline specifications, data models, and transformation logic, to facilitate knowledge sharing and support future enhancements. Collaboration ▪ Collaborate with cross-functional teams, including data ...

Title: Senior Data Engineer Location: Warwick (Hybrid Working) Rate: £550 - £650 per day (Umbrella) Contract: Initial 6 Months Role Overview Our client is seeking a skilled Senior Data Engineer to lead the design, development, and maintenance of data integration solutions and data … develop, and maintain ETL/ELT processes utilising tools such as Matillion. Create and maintain optimal data flows and data pipeline architectures. Develop and manage data warehouse objects, including tables, views, and security roles in Snowflake. Extract data from diverse sources ...

using Spark and Databricks best practices Develop scalable, reusable data engineering patterns to support multiple data domains Monitor and troubleshoot pipeline failures, performance bottlenecks, and data issues Medallion Architecture Development Build and evolve Bronze, Silver and Gold layers aligned to business and analytical … into robust, maintainable data engineering solutions ·Provide guidance on data availability, structure, and appropriate usage ·Documentation & Knowledge Sharing ·Document pipelines, data models, data quality rules, and architectural decisions ·Contribute to shared standards, templates, and best practices within the Data team ...

practices and the ability to translate customer expectations into applied technical functionality are key to success in this role. Key responsibilities Data pipeline development Build and maintain scalable data pipelines using Apache Spark and PySpark, processing and transforming large datasets across distributed cloud infrastructure. Workflow … monitoring, and execution of data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data ...

Data Sharing Lead Portsmouth | Personal Contract (dependent on skills and qualifications) Full-time Joint-contribution pension from 6% (12% total) – Enhanced maternity & family leave – Life assurance – HolidayPlus – Virtual GP & Employee Assistance Programme plus retail and leisure discounts & many more. REQ5603 The Data Sharing Lead is responsible … developing and overseeing our data sharing activities relating to requests received from external stakeholders but also those data sets we wish to publish proactively. The role covers everything from process, data and elements of technical oversight as well as engaging and collaborating with business ...

Oracle, SQL Server, MySQL, or cloud-native systems) combined with deep expertise in AWS data engineering services and modern data pipeline development. Key Responsibilities: Lead end to end database migrations to AWS from diverse sources, including on-premises systems (e.g., Oracle, SQL Server, MySQL … Redshift (data warehousing) Experience designing and implementing data validation and reconciliation frameworks Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent) Deep proficiency in SQL and data modeling concepts Experience working with S3-based data ...

stringent service level agreements (SLAs) and business requirements. What experience you need to be the successful Data Engineer: Proficiency in Palantir Foundry pipeline development: Extensive experience in building, maintaining, and optimising data pipelines and transformations within Palantir Foundry, leveraging tools such as Foundry Code Workbooks … integration. Strong coding skills in Python (Pandas) and PySpark: Essential experience in using Python (Pandas) and PySpark for data manipulation, transformation, and pipeline development within Palantir Foundry, ensuring high performance and scalability across various data flows. Experience in data modelling and ontology development ...

Title Engineering Data Integrity Lead Grade P3 Function Engineering Location Gloucester Security Clearance Required Baseline BPSS and SC Duration Permanent Hours 37hrs per week Raytheon UK At Raytheon UK, we take immense pride in being a leader in defence and aerospace technology. As an employer, we are dedicated … manage the end-to-end data platforms which control product design change, ensuring all changes are correctly assessed, approved, implemented, and recorded. Pipeline and Infrastructure - drive the creation of scalable and efficient pipelines and processing systems for data ingestion, transformation and access - including engineering change ...

Oracle, SQL Server, MySQL, or cloud-native systems) combined with deep expertise in AWS data engineering services and modern data pipeline development. Key Responsibilities: Lead end to end database migrations to AWS from diverse sources, including on-premises systems (e.g., Oracle, SQL Server, MySQL … cloud to cloud). Experience designing and implementing data validation and reconciliation frameworks. Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent). Deep proficiency in SQL and data modeling concepts. Experience working with S3-based data ...

Description We’re looking for a talented and passionate Contract Data Engineer (Reporting) to join our Group Technology team in Milton Keynes on a 6-month contract. You’ll play a key role in delivering Connells Group Reporting Data, covering medallion architecture, data modelling … Maintain the Common Data Model to ensure organisation-wide consistency. Apply Agile principles for iterative and collaborative development. Ensure data pipeline quality, reliability and performance. Develop, test and implement monitoring to ensure effective operation. Support cross-functional data projects, providing expertise as required. ...

Data Architecture Implementation Translate the defined data architecture and standards into implemented solutions—including ingestion, transformation, storage, and performance optimisation. Pipeline Development & Orchestration Develop robust ELT/ETL pipelines using dbt and workflow/orchestration tools (e.g., Argo Workflows), ensuring reliability, maintainability, and adherence … data engineering solutions, ideally with Snowflake. Strong hands-on proficiency with SQL, Python, and dbt for data transformations, modelling, and pipeline automation. Practical experience with Snowflake and RBAC management. Experience with data ingestion and replication tools such as Airbyte, Fivetran, Hevo, or similar. ...

Title:Snowflake/Dataops Data Engineer Technical Lead Department: Delivery Reporting to: Professional Services Manager About the role We are looking for a Client Technical Lead to join our delivery team ensure the technical standards and project approach are implemented in the best possible way. You will … enable customers to enhance and self-serve further visualisations Support and enable Data Science team(s) Review and resolve data pipeline failures and data test failures Optimise and improve data pipelines Utilise Snowflake best practices and features to optimise credit usage ...

seeking an experienced Data Architect to join our Group Technology team on a 6-month contract. You will help realise the value of the immense Connells Group data landscape for the benefit of customers, colleagues and the overall business. You will use your expertise to define … evolve and communicate the existing Estate Agency enterprise data architecture, to democratise safe access to consolidated and unified data, modelled to help engineering teams develop single customer and property views. The data architecture will also support system migration and integration, to establish effective master ...

Data Engineer £45,000 to 55,000 GBP Bonus Hybrid WORKING Location: Central London, Greater London - United Kingdom Type: Permanent Data Engineer - London Salary: Up to £53,000 + bonus + benefits Location: London (Hybrid) Security Clearance: SC eligible (must be sole UK citizen … seeking a Data Engineer to join a high-performing digital and data team working on secure, complex data environments across government and regulated sectors. This is an opportunity to work on meaningful data platforms that support large-scale transformation programmes, helping organisations ...

Description We are seeking an experienced Data Architect to join our Group Technology team on a 6-month contract.You will help realise the value of the immense Connells Group data landscape for the benefit of customers, colleagues and the overall business. You will use your expertise … define, evolve and communicate the existing Estate Agency enterprise data architecture, to democratise safe access to consolidated and unified data, modelled to help engineering teams develop single customer and property views. The data architecture will also support system migration and integration, to establish effective ...

Description We are seeking an experienced Data Engineer to join our Group Technology team on a 9-month contract. You’ll play a key role in delivering Connells Group Reporting Data, covering medallion architecture, data modelling, design, and pipelines. You’ll work closely with … Maintain the Common Data Model to ensure organisation-wide consistency. Apply Agile principles for iterative and collaborative development. Ensure data pipeline quality, reliability and performance. Develop, test and implement monitoring to ensure effective operation. Support cross-functional data projects, providing expertise as required. ...

from multiple internal and external sources Ensure data quality, integrity, security, and governance standards Monitor and troubleshoot data pipeline performance and failures Collaborate with analysts, data scientists, and software engineers to support business requirements Improve data platform scalability, reliability … roles Strong SQL skills and experience optimizing complex queries Proficiency in Python for data processing and automation Experience with ETL/ELT pipeline development Hands-on experience with cloud platforms such as AWS, Azure, or GCP Experience with data warehousing technologies such as Snowflake, Redshift ...

Lead Data Engineer/Data Tech Lead Permanent, Leeds/Hybrid Who are we? Enablis is a technology delivery consultancy who partner with clients to accelerate digital roadmaps. By combining our deep knowledge and best practice experience with the client’s domain expertise, we are able … open the door to incredible opportunities and introduce you to a network of equally driven, like-minded professionals. Job Brief The Lead Data Engineer role is responsible for owning the technical direction, design and delivery of data platforms and data engineering solutions ...

Data Platform Manager UK/Remote Circa £70k per annum + 15% bonus & benefits Are you ready to take ownership of a data platform and shape how data drives business decisions? This is a rare opportunity to step into a leadership role where … combine hands-on data engineering with strategic product management, all while building and mentoring a high-performing team. You'll play a pivotal role in evolving a modern data platform (Microsoft Fabric/Azure Synapse), ensuring it delivers real business value, scalability, and innovation. ...

Description We’re looking for a talented and passionate Core Data Engineer to join our Group Technology team in Milton Keynes. You’ll play a key role in delivering Connells Group Reporting Data, covering architecture, data modelling, design, and pipelines. You’ll work closely … Maintain the Common Data Model to ensure organisation-wide consistency. Apply Agile principles for iterative and collaborative development. Ensure data pipeline quality, reliability and performance. Develop, test and implement monitoring to ensure effective operation. Support cross-functional data projects, providing expertise as required. ...

automate testing, validation, and deployment of geospatial data workflows. Enhancing observability through structured logging, data-quality checks, lineage tracking, and pipeline health indicators. Key Responsibilities Data Pipeline Delivery: Build and maintain geospatial ETL pipelines using Python, SQL, FME, ArcGIS tools, or similar … quality, validation, and structured workflows. Observability, Quality & Reliability: Add structured logging, validation checks, and lineage tracking into pipelines. Contribute to dashboards monitoring pipeline health, reliability, and data‐quality metrics. Apply testing and engineering discipline to improve predictability and reduce defects. Documentation & Governance: Produce clear documentation ...

Data Engineer | London Hybrid | £70,000–£100,000 + 30–40% Bonus I’m currently working with a well-established and growing business in the insurance market that is looking to hire a Data Engineer into its London team. This is a strong opportunity … environments into a more scalable Azure-based platform. The role will suit someone who is confident across Azure data engineering, pipeline development, data modelling and stakeholder engagement, and who wants to play a hands-on role in shaping a more modern data ...

Senior Data Planner This role owns relationships with external data vendors and is accountable for ensuring externally sourced data is technically fit, well-integrated, governed, and delivers long-term value across the organisation. Client Details Senior Data Planner The employer … services and supporting the community. As a large organisation, it offers a professional and structured environment with opportunities for impactful work. Description Senior Data Planner Define and deliver the external data sourcing strategy, ensuring alignment with enterprise data architecture, analytics ambitions, and business priorities ...

us.Please see below for a detailed job description for the role. Main duties of the job We are looking for a highly motivated Data Delivery Project Manager. This post will be an exciting opportunity to be part of the team supporting the delivery of the Wessex SDE (Secure … Data Environment) and the Southampton Emerging Therapies and Technologies (SETT) Centre, to scale data for research across the region and nationally. These Data Delivery Project Management posts will be responsible for project managing a range projects both establishing data pipelines across ...