1 to 25 of 660 Data Pipeline Jobs in the UK

engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources … effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote ...

Data Engineer FTC – 12 Months – Coventry University of Warwick £47,389 - £56,535 (D.O.E) 42 Days Holidays (Inc Bank Hols) Excellent Pension Contributions Hybrid Working Great Food, Gym and Leisure Facilities on-site We are seeking a new addition for our Data Labs Team to assist … architects, system owners and end users to ensure quality issues are addressed as needed. Documentation ▪ Create and maintain technical documentation, including data pipeline specifications, data models, and transformation logic, to facilitate knowledge sharing and support future enhancements. Collaboration ▪ Collaborate with cross-functional teams, including data ...

IMMEDIATE JOINERS !! 💻 DATA ENGINEER (Python & SQL Focus) 📊 We're looking for an enthusiastic and detail-oriented Junior Big Data Developer to join our data engineering team. This role is ideal for an early-career professional with foundational knowledge in data processing, strong … maintaining data pipelines and ensuring data quality across our scalable Big Data platforms. Key Responsibilities Data Pipeline Development: Assist in the design, construction, and maintenance of robust ETL/ELT pipelines to integrate data from various sources into ...

Title: Senior Data Engineer Location: Warwick (Hybrid Working) Rate: £550 - £650 per day (Umbrella) Contract: Initial 6 Months Role Overview Our client is seeking a skilled Senior Data Engineer to lead the design, development, and maintenance of data integration solutions and data … develop, and maintain ETL/ELT processes utilising tools such as Matillion. Create and maintain optimal data flows and data pipeline architectures. Develop and manage data warehouse objects, including tables, views, and security roles in Snowflake. Extract data from diverse sources ...

Data Sharing Lead Portsmouth | Personal Contract (dependent on skills and qualifications) Full-time Joint-contribution pension from 6% (12% total) – Enhanced maternity & family leave – Life assurance – HolidayPlus – Virtual GP & Employee Assistance Programme plus retail and leisure discounts & many more. REQ5603 The Data Sharing Lead is responsible … developing and overseeing our data sharing activities relating to requests received from external stakeholders but also those data sets we wish to publish proactively. The role covers everything from process, data and elements of technical oversight as well as engaging and collaborating with business ...

Oracle, SQL Server, MySQL, or cloud-native systems) combined with deep expertise in AWS data engineering services and modern data pipeline development. Key Responsibilities: Lead end to end database migrations to AWS from diverse sources, including on-premises systems (e.g., Oracle, SQL Server, MySQL … Redshift (data warehousing) Experience designing and implementing data validation and reconciliation frameworks Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent) Deep proficiency in SQL and data modeling concepts Experience working with S3-based data ...

stringent service level agreements (SLAs) and business requirements. What experience you need to be the successful Data Engineer: Proficiency in Palantir Foundry pipeline development: Extensive experience in building, maintaining, and optimising data pipelines and transformations within Palantir Foundry, leveraging tools such as Foundry Code Workbooks … integration. Strong coding skills in Python (Pandas) and PySpark: Essential experience in using Python (Pandas) and PySpark for data manipulation, transformation, and pipeline development within Palantir Foundry, ensuring high performance and scalability across various data flows. Experience in data modelling and ontology development ...

Title Engineering Data Integrity Lead Grade P3 Function Engineering Location Gloucester Security Clearance Required Baseline BPSS and SC Duration Permanent Hours 37hrs per week Raytheon UK At Raytheon UK, we take immense pride in being a leader in defence and aerospace technology. As an employer, we are dedicated … manage the end-to-end data platforms which control product design change, ensuring all changes are correctly assessed, approved, implemented, and recorded. Pipeline and Infrastructure - drive the creation of scalable and efficient pipelines and processing systems for data ingestion, transformation and access - including engineering change ...

Oracle, SQL Server, MySQL, or cloud-native systems) combined with deep expertise in AWS data engineering services and modern data pipeline development. Key Responsibilities: Lead end to end database migrations to AWS from diverse sources, including on-premises systems (e.g., Oracle, SQL Server, MySQL … cloud to cloud). Experience designing and implementing data validation and reconciliation frameworks. Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent). Deep proficiency in SQL and data modeling concepts. Experience working with S3-based data ...

Contract Data Architect - Senior SME (Modern Data Platform) Contract Length: 12-18 months (strong potential to extend or move permanent) It will be Inside IR35 Day Rate: £600-£700 Location: UK-based, primarily remote Ad hoc travel to Stratford-upon-Avon & London for workshops and occasional … principles Lead the adoption of Medallion Architecture (Bronze/Silver/Gold) Design and oversee robust data ingestion and data pipeline frameworks , including ingestion of poor-quality and complex data Provide architectural leadership for: Data lakes and lakehouse patterns Analytics ...

Senior Data Engineer | FX Trading House | London (Hybrid - 3 Days) | Up to £110,000 + Bonus & Benefits | West London We are partnered with a leading FX trading house, operating at scale across global markets. With a rapidly evolving data landscape and continued investment in modern data … shaping architecture, setting engineering standards, and mentoring a small team, while remaining deeply technical. The role has a strong emphasis on data pipeline development in Python, alongside broader platform ownership including data warehousing, data products, and data governance. You’ll work ...

Data Architecture Implementation Translate the defined data architecture and standards into implemented solutions—including ingestion, transformation, storage, and performance optimisation. Pipeline Development & Orchestration Develop robust ELT/ETL pipelines using dbt and workflow/orchestration tools (e.g., Argo Workflows), ensuring reliability, maintainability, and adherence … data engineering solutions, ideally with Snowflake. Strong hands-on proficiency with SQL, Python, and dbt for data transformations, modelling, and pipeline automation. Practical experience with Snowflake and RBAC management. Experience with data ingestion and replication tools such as Airbyte, Fivetran, Hevo, or similar. ...

Senior Data Engineer – Platform & APIs or Senior Data Engineer – Data Exposure Intelligence Data Platform — Manchester (or Croydon) What You'll Own End-to-End Data Ownership Raw data ingestion through final consumption Architecture decisions that affect downstream users … structure and user needs Manage API performance, versioning, and reliability Own the contract between data and consumers Data Pipeline Architecture Design and optimise ingestion, transformation, and delivery systems Work with complex, relationship-driven datasets (knowledge graphs, entity relationships) Structure data ...

Data Architect - AI and Data (DV Cleared) Location: UK Clearance: DV clearance required You'll work on high impact national programmes, shaping how data is governed, trusted and used across Government and Public Services. You'll be part of a strong Data & AI community, delivering meaningful work at scale. You will play a key role in shaping secure, scalable and standards aligned data architectures that underpin critical national systems. A senior consulting role, combining enterprise architecture, delivery assurance and stakeholder leadership. You will assess existing data ...

seeking an experienced Data Architect to join our Group Technology team on a 6-month contract. You will help realise the value of the immense Connells Group data landscape for the benefit of customers, colleagues and the overall business. You will use your expertise to define … evolve and communicate the existing Estate Agency enterprise data architecture, to democratise safe access to consolidated and unified data, modelled to help engineering teams develop single customer and property views. The data architecture will also support system migration and integration, to establish effective master ...

seeking an experienced Data Engineer to join our Group Technology team on a 9-month contract.Youll play a key role in delivering Connells Group Reporting Data, covering medallion architecture, data modelling, design, and pipelines. Youll work closely with technical specialists to innovate, share ideas … Maintain the Common Data Model to ensure organisation-wide consistency. Apply Agile principles for iterative and collaborative development. Ensure data pipeline quality, reliability and performance. Develop, test and implement monitoring to ensure effective operation. Support cross-functional data projects, providing expertise as required. ...

days/week Flexible working : Variety of flexible work patterns subject to line manager discretion e.g. Compressed 9-day fortnight. Reports to: Lead Data Engineer Deadline Note: We reserve the right to close the advert before the advertised deadline if there are a high volume of applications. Role … Summary: The Senior data engineer role will have a specific focus on data and integration, ensuring that they drive single version of the truth by developing and maintaining reliable solutions to store and manage key data. The role will source, model, and provide data ...

Lead Data Engineer/Data Tech Lead Permanent, Leeds/Hybrid Who are we? Enablis is a technology delivery consultancy who partner with clients to accelerate digital roadmaps. By combining our deep knowledge and best practice experience with the client’s domain expertise, we are able … open the door to incredible opportunities and introduce you to a network of equally driven, like-minded professionals. Job Brief The Lead Data Engineer role is responsible for owning the technical direction, design and delivery of data platforms and data engineering solutions ...

automate testing, validation, and deployment of geospatial data workflows. Enhancing observability through structured logging, data-quality checks, lineage tracking, and pipeline health indicators. Key Responsibilities Data Pipeline Delivery: Build and maintain geospatial ETL pipelines using Python, SQL, FME, ArcGIS tools, or similar … quality, validation, and structured workflows. Observability, Quality & Reliability: Add structured logging, validation checks, and lineage tracking into pipelines. Contribute to dashboards monitoring pipeline health, reliability, and data‐quality metrics. Apply testing and engineering discipline to improve predictability and reduce defects. Documentation & Governance: Produce clear documentation ...

efficient data processing, storage and retrieval. You will work closely with cross-functional teams to analyse requirements, propose new data pipeline architectures, and implement solutions to improve performance and scalability. The tasks for this post include the following: Analyse existing data pipelines … optimisation, and scalability. Work closely with Bioinformaticians and annotators to integrate data pipelines with existing systems and applications. Monitor data pipeline performance, troubleshoot issues, and implement solutions to ensure reliability and efficiency. Stay current with industry trends and best practices in data engineering ...

Senior Data Planner This role owns relationships with external data vendors and is accountable for ensuring externally sourced data is technically fit, well-integrated, governed, and delivers long-term value across the organisation. Client Details Senior Data Planner The employer … services and supporting the community. As a large organisation, it offers a professional and structured environment with opportunities for impactful work. Description Senior Data Planner Define and deliver the external data sourcing strategy, ensuring alignment with enterprise data architecture, analytics ambitions, and business priorities ...

Head of Data Bristol (Hybrid) From £90k Certified B Corp The Opportunity We’re working alongside Ecosurety at a pivotal point in their growth. They’re a strong, established business, now placing increased focus on building out a product range that helps organisations make critical decisions about their … packaging, improving sustainability while reducing costs. At the centre of this is a platform that brings together fragmented packaging data, applies complex regulatory logic, and enables customers to model packaging decisions across multiple markets. As this platform evolves, there’s a clear opportunity to go further - moving beyond ...

years) as a Data Engineer in high-volume, high-transactional environments. Strong Python (or Scala) skills for data processing and pipeline development. Hands-on experience with AWS, particularly Glue, Lambda, and S3. Strong SQL skills and experience working with data warehouses. Solid understanding …/ELT design and data pipeline optimisation. Experience with big data frameworks such as Apache Spark, and exposure to streaming technologies like Kafka. Strong problem-solving skills and the ability to collaborate effectively across teams. What's on offer: Opportunity to influence the architecture ...

deploying top-tier engineers into mission-critical projects, combining deep technical expertise with a strong delivery mindset. We’re now hiring a Data Scientist to design and build scalable data solutions for leading organisations. Why Join Work on high-impact, complex data projects Hybrid … working with flexibility Collaborate with high-performing engineering teams Exposure to modern data stacks and cloud technologies Clear progression in a growing consultancy The Role You’ll design, build, and optimise data pipelines and systems that support critical business decisions. Key Responsibilities Build and maintain data ...

Senior Data Engineer Northwest/Remote (UK-based) £65,000-£75,000 We are partnering with a leading organisation to recruit an experienced Senior Data Engineer to play a key role in the design, development, and optimisation of modern data platforms. This position sits … change initiatives and driving adoption of new technologies Skills & Qualifications Strong expertise in data modelling, database design, and data pipeline development Experience with both SQL and NoSQL technologies Hands-on experience with at least one major cloud platform (Azure, AWS, or GCP) Experience with ...