251 to 275 of 372 Data Pipeline Jobs in the UK excluding London

As an AI Software Engineer Apprentice, you’ll deliver real-world AI and ML solutions working with data pipelines, models, APIs, and microservices while advancing toward a fully funded Level 6 degree. This role is rooted in the Digital and Technology Solutions standard, blending a common core with the Software Engineer specialism. Role Location: Hybrid (UK-based) Hours … Monday – Friday, 09:00–17:00 Salary: £15,000 per annum Apprenticeship Standard: ST0119 v1.2 – Digital and Technology Solutions Degree (Software Engineer pathway) Key Responsibilities Model & Data Pipeline Development Assist in collecting, cleaning, validating, and preparing data for training and evaluation. Support the design, development, and tuning of machine learning and deep learning models. Contribute … to scalable and reusable data pipelines using modern ML workflows. Experimentation & Evaluation Conduct experiments and benchmarking exercises to test model performance. Perform error analysis, feature importance, and other model diagnostics. Track and log training/testing outcomes to support reproducibility and model versioning. Engineering Contributions Help build and integrate AI-powered APIs, scripts, and microservices. Collaborate on backend More ❯

ranks 189 on the Fortune 500. Further information about Lear is available at or on Position overview: As an Application Solution Architect, you will design and implement scalable, data-driven solutions using Palantir Foundry. You'll work closely with business stakeholders, data engineers, and developers to translate business needs into robust technical architectures that leverage Foundry … technical constraints. Design scalable and efficient data models and ontologies within Foundry. Define and oversee the development of data pipelines and workflows using Foundry's Pipeline Builder and Code Repositories. Lead integration of external systems and APIs into Foundry for seamless data flow. Work with cross-functional teams to gather requirements, define architecture … TECHNICAL/PERSONNEL SKILLS : Experience designing scalable data models and ontologies. Understanding of semantic data modelling and entity relationships. Familiarity with Foundry's Ontology SDK, Pipeline Builder, and Code Workspaces. Experience integrating external systems and APIs into Foundry. Knowledge of data transformation, cleansing, and enrichment techniques. Awareness of data privacy, access More ❯

Wood Mackenzie is the global data and analytics business for the renewables, energy, and natural resources industries. Enhanced by technology. Enriched by human intelligence. In an ever-changing world, companies and governments need reliable and actionable insight to lead the transition to a sustainable future. That's why we cover the entire supply chain with unparalleled breadth and … committed - we put customers at the heart of our decisions Future Focused - we accelerate change Curious - we turn knowledge into action Job Description We are looking for a Data Engineer who can leverage Python to build new data pipelines. Your role will sit within an Agile team developing backend data pipelines and processes, aligned … to our data roadmap for delivering new features for Wood Mackenzie's flagship product offering, Lens. You will work alongside other Data Engineers, business SMEs, product and data owners to ensure Wood Mackenzie delivers a world-class experience for its customers - helping them change the way we power our planet. Main Responsibilities As part More ❯

CONTRACT PALANTIR DATA ENGINEER - DV CLEARED NEW OUTSIDE IR35 CONTRACT OPPORTUNITY AVAILABLE WITHIN A HIGHLY SECURE DELIVERY PROGRAMME FOR A DV CLEARED PALANTIR FOUNDRY DATA ENGINEER. Contract opportunity for a Palantir Data Engineer to support cutting-edge National Security projects £600 - £750 per day (Outside IR35) DV clearance is essential Based full-time onsite … using Foundry tools like Workshop Strong communication skills to collaborate with technical and military/non-technical stakeholders Experience integrating data from multiple sources into Foundry via Pipeline Builder and custom code Comfortable working on-site within a secure environment and at pace with end-users TO BE CONSIDERED... . Please either apply by clicking online or … consent for us to process and submit your application to our client in conjunction with this vacancy only. KEY SKILLS: PALANTIR/FOUNDARY/DATA ENGINEER/PIPELINE BUILDER/ONTOLOGY/PYTHON/TYPESCRIPT/FULL STACK/DV CLEARED/LONDON/CONTRACT More ❯

CONTRACT PALANTIR DATA ENGINEER - DV CLEARED NEW OUTSIDE IR35 CONTRACT OPPORTUNITY AVAILABLE WITHIN A HIGHLY SECURE DELIVERY PROGRAMME FOR A DV CLEARED PALANTIR FOUNDRY DATA ENGINEER. Contract opportunity for a Palantir Data Engineer to support cutting-edge National Security projects £600 - £750 per day (Outside IR35) DV clearance is essential Based full-time onsite … using Foundry tools like Workshop Strong communication skills to collaborate with technical and military/non-technical stakeholders Experience integrating data from multiple sources into Foundry via Pipeline Builder and custom code Comfortable working on-site within a secure environment and at pace with end-users TO BE CONSIDERED... . Please either apply by clicking online or … consent for us to process and submit your application to our client in conjunction with this vacancy only. KEY SKILLS: PALANTIR/FOUNDARY/DATA ENGINEER/PIPELINE BUILDER/ONTOLOGY/PYTHON/TYPESCRIPT/FULL STACK/DV CLEARED/LONDON/CONTRACT More ❯

Viya 4. - Support the migration of Legacy SAS platforms to cloud native SAS Viya 4 environments, including namespace separation and Kubernetes orchestration. - Implement and manage configuration objects, service data, and translation objects within SAS environments. - Collaborate with data engineers and DevOps teams to integrate SAS with Oracle, GitLab CI/CD pipelines, and other enterprise platforms. … Monitor and optimise platform performance, ensuring high availability and compliance with security standards. - Provide technical support and troubleshooting for SAS users and developers. - Automate environment provisioning and data transfers using approved EA integration patterns. - Participate in community knowledge sharing and contribute to best practice documentation. Key Experience: - SAS Enterprise Guide (EG) - workflow orchestration and user support. - SAS Data … Viya 4 - platform configuration, migration, and containerisation (Kubernetes). - SAS Visual Analytics (VA) and SAS Visual Investigator (VI) - dashboard and investigation tool support. - Oracle - integration and data pipeline support. - Configuration Management - experience managing service objects, deployment artefacts, and environment baselines. - Testing Lifecycle Knowledge - including test strategy design, environment comparison utilities, and automated testing integration. - Experience with cloud More ❯

The Data Analyst will be responsible for collecting, processing, and analysing data to support strategic decision-making & planning across the business. Role Analyse large datasets to identify trends, patterns, and insights Develop and maintain dashboards and reports using tools like Power BI Perform data cleaning, transformation, and validation to ensure accuracy and quality Collaborate … with stakeholders to gather requirements and deliver data solutions Conduct A/B tests and statistical analysis to inform business decisions Monitor key performance indicators (KPIs) and provide regular performance reports Work with data engineering teams to improve data pipelines and data architecture Present findings to stakeholders through visualisations and presentations Training … All work uploaded to Aptem Monthly college release days (online or face to face) Potential full-time position Apprenticeship Standard Data analyst (level 4) Training Provider BRADFORD COLLEGE Working Week Monday - Thursday 07:30am - 16:15. Friday 07:30am - 12.45pm. Expected Duration 2 Years Positions Available 1 Closing Date Thursday, 14th August 2025 Start Date Wednesday, 20th August More ❯

Data Engineer | £60-65K | London (5 days p/w Onsite) Build a greenfield data platform from scratch! This logistics company is halfway through a massive digital transformation. Legacy monoliths are being dismantled, replaced with a clean, cloud-native, modular stack - and the new data team is central to making it all talk. … You'll be working directly with their Head of Data, building Azure-based pipelines from the ground up, and shaping how the company uses data across 50+ systems. There's real investment, real urgency, and real interest in doing this properly - not endless meetings and PowerPoints. What you'll be doing: Designing, building, and optimising Azure … based data pipelines using Databricks, PySpark, ADF, and Delta Lake Implementing a medallion architecture - from raw to curated Collaborating with analysts to make data business-ready Applying CI/CD and DevOps best practices (Git, Azure DevOps) Supporting the team's growth with clean, well-documented code Exploring real-time logistics datasets What they're looking More ❯

Head of Data Engineering Manchester (Hybrid)Permanent Full-timeSalary: £100,000 - £120,000 (DOE) A major UK financial services organisation is seeking a visionary Head of Data Engineering to lead and scale its enterprise-wide data platforms. Youll drive the design and delivery of modern data solutions across cloud, analytics, and core … banking systems — enabling smarter decisions, customer innovation, and regulatory excellence. Key Responsibilities: Lead and develop high-performing data engineering teams Architect scalable, secure data pipelines and platforms (Azure, Databricks, etc.) Partner with product, risk, and tech leaders to deliver data-driven outcomes Champion best practices in data governance, quality, and DevOps What … You’ll Bring: Proven leadership in large-scale data engineering within complex environments Deep technical expertise in cloud-native data architecture Strong strategic thinking and stakeholder engagement skills Passion for transforming legacy systems into modern, value-driving platforms This is a rare opportunity to shape the future of data in a trusted institution committed More ❯

Head of Data Engineering Manchester (Hybrid) Permanent Full-time Salary: £100,000 - £120,000 (DOE) A major UK financial services organisation is seeking a visionary Head of Data Engineering to lead and scale its enterprise-wide data platforms. Youll drive the design and delivery of modern data solutions across cloud, analytics, and … core banking systems — enabling smarter decisions, customer innovation, and regulatory excellence. Key Responsibilities: Lead and develop high-performing data engineering teams Architect scalable, secure data pipelines and platforms (Azure, Databricks, etc.) Partner with product, risk, and tech leaders to deliver data-driven outcomes Champion best practices in data governance, quality, and DevOps … What You'll Bring: Proven leadership in large-scale data engineering within complex environments Deep technical expertise in cloud-native data architecture Strong strategic thinking and stakeholder engagement skills Passion for transforming legacy systems into modern, value-driving platforms This is a rare opportunity to shape the future of data in a trusted institution More ❯

Position at Everyday Health - Professional The Opportunity We're looking for a Senior Data Quality Analyst to join our fast-paced Data & Analytics team. This team supports internal operations and client-facing work by ensuring the accuracy and reliability of data pipelines, audience segmentation, and reporting. You'll take ownership of quality checks across … the data lifecycle-from ingestion to delivery-and work closely with engineers, analysts, and stakeholders to drive high standards and solve issues quickly. Strong communication and collaboration are key to this role. Key Responsibilities Own and manage QA processes for data pipelines, campaign outputs, and reporting across Databricks, SQL Server, Looker, and QuickPivot Build automated tests … to catch data issues early and ensure clean, reliable data throughout workflows Test all changes and new development before they go live Work closely with engineering, business intelligence, data science and operations teams to identify and resolve issues quickly Maintain clear documentation of QA processes and testing standards Investigate data quality problems More ❯

Senior Data Engineers with a background are required by this major client, who is creating a new start up engineering capability in their Leeds offices, where you will provide best in class Data Engineering services to a wide range of major Public Sector organisations. As a result of the work that they do, this client requires … national or dual UK national. Please note your application will not be taken forward if you cannot fulfil these requirements. In order to secure one of these Senior Data Engineer roles you must be able to demonstrate the following experience: Experience in prominent languages such as Python, Scala, Spark, SQL. Good experience in using Databricks Experience working with … Knowledge and experience using at least one Data Platform Technology such as Quantexa, or Palantir Knowledge of test automation frameworks and ability to automate testing within the pipeline This is an opportunity for significant growth, but with a start-up feel and mentality. In this role you will have the chance to build and be part of More ❯

you the chance to help make the UK a safer place through innovation, working in partnership with government and law enforcement organisations on some of the most challenging data and technology problems out there. We strongly encourage candidates of all different backgrounds and identities to apply. We are committed to building an inclusive, safe and supportive environm ent … Here are some things we've worked on recently that might give you a better sense of the kind of work you could be leading on: Building a data analytics tool using a Neo4J graph database. A DevOps platform to support 3000 users. Processing digital forensic data to enable search and insight generation. A search and … reporting system backed by an ElasticSearch cluster. A custom workflow tool using Camunda, with a React/TS front-end An AI assurance capability Orchestrating a data pipeline and designing specialist metrics for the evaluation of the performance of LLMs in different contexts. Accountabilities: Understands the technical aspects of the project and the wider customer business model. More ❯

on Python-based AI/ML development, automation, and general software engineering. The successful candidate will play a key role in building and deploying machine learning features and data-driven applications. You will work on end-to-end solutions - from writing robust code and unit tests to developing APIs and integrating machine learning models into our product ecosystem. … to hear from you. Experience in the travel or retail industry would be an advantage. Responsibilities Design, implement, and maintain software components that incorporate machine learning algorithms and data processing. Develop clean, efficient Python code for both backend logic and integration of ML models. Understand the business drivers behind each feature. Create and optimise data pipelines … and software architectures Familiarity with CI/CD and automation tools. Experience using Git for version control and platforms like Bitbucket for code collaboration. Knowledge of build tools and pipeline configuration (Jenkins) to automate testing and deployment. Strong problem-solving and analytical skills Presentation and teamwork skills Understanding of both Waterfall and Agile methodologies About Datalex Datalex's purpose More ❯

Senior Data Engineer UST are currently recruiting for a Senior Data Engineer with proficiency in developing data pipelines including coding and testing for ingesting wrangling transforming and joining data from various sources. About the role The ideal candidate should be adept in ETL tools like Informatica Glue Databricks and DataProc with strong … coding skills in Python PySpark and SQL. You will have expertise in data warehousing solutions such as Snowflake BigQuery Lakehouse and Delta Lake is essential including the ability to calculate processing costs and address performance issues. A solid understanding of DevOps and infrastructure needs is also required. Job Description: Work as part of a self-organised Ab Initio … any inconsistencies with the standards (written or best practices) You will be able to estimate the delivery timescales and justify these estimates You will be aware of the Jenkins pipeline principles Knowledge of Agile practices Experience that would be advantageous: You will be aware of AWS interactions and file storage You will have a core understanding of Lambda programming More ❯

Join a mission-driven team shaping the future of geospatial data engineering. We’re looking for a Principal Data Engineer to lead the design and delivery of cutting-edge data solutions across cloud platforms and spatial technologies. If you're passionate about geospatial data, cloud architecture, and building scalable pipelines, this is … your chance to make a real impact. What You’ll Be Doing: Architecting and implementing robust data pipelines using FME and QGIS . Leading cloud-based data engineering projects across AWS and Azure . Collaborating with cross-functional teams to deliver high-impact geospatial solutions. Mentoring junior engineers and setting best practices for data engineering excellence. Driving innovation in spatial data processing, automation, and analytics. What You’ll Bring: Proven experience as a Senior or Principal Data Engineer . Deep expertise in QGIS and FME for geospatial data transformation. Strong hands-on experience with AWS (e.g., S3, Lambda, Glue) and/or Azure (e.g., Data More ❯

Data Manager (BI) A specialty insurer is looking for a Data Manager (BI) to help shape and drive the next phase of their enterprise data platform. This is a hands-on role that also carries leadership responsibilities, sitting right at the intersection of data engineering, BI architecture, and business strategy.You'll work … closely with senior stakeholders and technical teams to deliver scalable data solutions, build and manage data pipelines, and turn complex requirements into meaningful, actionable insights - all within the Microsoft tech stack.What we're looking for:Strong experience across the Microsoft BI stack - SQL Server, Power BI, SSAS, SSIS Good knowledge of data engineering principles More ❯

You'll Do Design and develop robust backend applications (initially in your core language, with support to learn and transition into Go/Golang) Build scalable APIs and data pipelines integrating with third-party platforms Write clean, maintainable code with strong testing practices Collaborate in agile sprints to deliver high-quality features and improvements Contribute to CI/ More ❯

experienced. The User Foundations team plays a pivotal role in delivering key user capabilities shared across Disney+, Hulu, ESPN+, and Star+. We build high-throughput, reliable services and data pipelines that gather user insights and power personalization across all our platforms. We're a dynamic, fast-paced, and collaborative team looking for a skilled Software Engineer who thrives More ❯

experienced. The User Foundations team plays a pivotal role in delivering key user capabilities shared across Disney+, Hulu, ESPN+, and Star+. We build high-throughput, reliable services and data pipelines that gather user insights and power personalization across all our platforms. We're a dynamic, fast-paced, and collaborative team looking for a skilled Software Engineer who thrives More ❯

As a Data Engineer with a focus on NLP, you will play a crucial role in the development of our initial platform. Your primary responsibilities will include designing, implementing, and optimizing data pipelines that support AI-driven call handling solutions. You will work collaboratively with cross-functional teams to enhance our platform's capabilities, ensuring it … and strive for continuous improvement. Ownership: Take pride in your work and reject mediocrity. Empathy: Show genuine care and consideration for others. You will: Develop and maintain scalable data architecture that supports natural language processing tasks. Implement data processing workflows to prepare text data for NLP applications, including text extraction, cleaning, and transformation. Collaborate … with data scientists and software engineers to integrate machine learning models into our platform. Utilize AWS tools and services to enhance data management and processing efficiency. Contribute to the documentation and technical specifications for the data infrastructure supporting the application. Key Requirements: Proven experience in data engineering, with a strong background in More ❯

Senior Software Engineer (Data) Location - Belfast Hybrid Working Salary - Negotiable A unique opportunity to join this Belfast based client to work on their Engineering team in a role that combines both Software and Data Engineering. This role combines development both Software engineering primarily in .Net/C# along with Data Engineering, of which responsibilities … include aspects of Data Modelling, Message & Data Pipelines and Data Infrastructure. Experience required Strong programming experience with C# and .NET Core. Solid understanding of Microsoft SQL Server Experience working with cloud platforms Exposure to enterprise messaging systems (e.g. RabbitMQ etc) If you are interested in this role or would be keen to see what More ❯

We are a Global Recruitment specialist that provides support to the clients across EMEA, APAC, US and Canada. We have an excellent job opportunity for you. Title: Data Visualisation Engineer Clearance required: Must hold an Active SC Clearance Location: Telford/Hybrid 6 months contract Job description: Purpose: This project seeks to streamline that experience by delivering a … a new scrum team within Minerva Platform to develop and deliver the Ingestion and Risking within the SAS Platform including IDP. Key Responsibilities: Design, development, and deployment of data integration and transformation solutions using Pentaho, Denodo, Talend, and SAS. Architect and implement scalable data pipelines and services that support business intelligence and analytics platforms. Collaborate with … cross-functional teams to gather requirements, define technical specifications, and deliver robust data solutions. Champion Agile and Scrum methodologies, ensuring timely delivery of sprints and continuous improvement. Drive DevOps practices for CI/CD, automated testing, and deployment of data services. Mentor and guide junior engineers, fostering a culture of technical excellence and innovation. Ensure data More ❯

Are you a talented Data Scientist looking for a new challenge? Be part of the 'engine room team within an established business with the financial support of a FTSE listed organisation. Our client is a leading IT solutions provider who are going through exciting and promising times, and this Data Scientist role will allow you to … become a key contributor to future business changes. You will receive coaching and mentorship within an established, growing Data team to support your growth, so that you can perform at your best. In this Data Scientist role, you will be analsying and identifying insights working alongside Senior and Lead Data Scientists, developing and deploying … data-driven solutions, and building and maintaining data pipelines. The key skills required for the Data Scientist role are: A strong education foundation from a Bachelors or Masters degree in Engineering/Statistics/Mathematics/Computer Science/Finance Experience of working in a Data Scientist role for 1+ years Experience More ❯

Data Engineer Contract to Permanent Skills: Python, Pandas Hybrid London Rate: c400/day (outside) & 70k Transform Sustainability with Smart Data Simulations. Ready to code for a cleaner future? A cutting-edge sustainability start-up based in Central London is seeking an experienced Data Engineer to help transform how businesses reduce their carbon footprint. … If you're fluent in Python, a master of Pandas, and passionate about building analytical models with customer data, this role offers you purpose and impact. What you’ll be doing Designing and implementing robust data pipelines Building simulations to model complex real-world customer behaviours Working with large and intricate data sets to … uncover actionable insights Collaborating with analysts, scientists, and product teams to turn data into sustainable solutions What we’re looking for Proven experience with Python and Pandas, especially in data analysis and simulation building A keen eye for data accuracy and insight generation Passion for sustainability and applying data to drive environmental More ❯