101 to 125 of 492 Data Pipeline Jobs in the UK

accommodate the delivery of our expanding workload in support of the broader growth of the business. Linked to the steady growth of the business, we are evolving our data architecture to offer a modern data service with new data sources coming online and redeveloping our data warehouse to align with our projected … growth. The current team is small so the role will have exposure to colleagues across Ridge, representing the Data and Analytics team in conversations across the entire business' data estate. Therefore, the role has a broad scope of responsibility, requiring the individual to own projects themselves and deliver end-to-end solutions, with support and discussion … shared across the team. Responsibilities: Integrating new data sources into the data warehouse, and building in best-practice warehouse design Staging & shaping data from multiple data sources into reporting and analytical schemas Experience with data warehouse modelling, designed to accommodate future growth Creation of technical documentation, capturing development, processes & DW More ❯

Data Engineer - London - Databricks - Azure - 75k + bonus + benefits Tech Stack: Azure | Databricks | Power BI | Python Are you ready to shape the future of data? We're looking for a Data Engineer to join a team that thrives on innovation and cutting-edge technology. If you love solving complex data challenges … and building scalable solutions, this is your chance to make an impact. What You'll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards ETL & Data Modelling: T-SQL, metadata-driven pipelines DevOps: CI/CD Bonus: Python What you'll do Design … and implement scalable Azure-based data solutions Build and optimize data pipelines for integration and transformation Develop Power BI dashboards for global stakeholders Ensure data quality, governance, and security Collaborate in an Agile environment with cross-functional teams What We're Looking For Hands-on experience with Azure & Databricks Strong data engineering More ❯

Data Engineer - London - Databricks - Azure - 75k + bonus + benefits Tech Stack: Azure | Databricks | Power BI | Python Are you ready to shape the future of data? We're looking for a Data Engineer to join a team that thrives on innovation and cutting-edge technology. If you love solving complex data challenges … and building scalable solutions, this is your chance to make an impact. What You'll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards ETL & Data Modelling: T-SQL, metadata-driven pipelines DevOps: CI/CD Bonus: Python What you'll do Design … and implement scalable Azure-based data solutions Build and optimize data pipelines for integration and transformation Develop Power BI dashboards for global stakeholders Ensure data quality, governance, and security Collaborate in an Agile environment with cross-functional teams What We're Looking For Hands-on experience with Azure & Databricks Strong data engineering More ❯

architecture you will: Build and orchestrate data pipelines across Snowflake and AWS environments. Apply data modeling, warehousing, and architecture principles (Kimball/Inmon). Develop pipeline programming using … Python, Spark, and SQL; integrate APIs for seamless workflows. Support Machine Learning and AI initiatives, including NLP, Computer Vision, Time Series, and LLMs. Implement MLOps, CI/CD pipelines, data testing, and quality frameworks. Act as an AI super-user, applying prompt engineering and creating AI artifacts. Work independently while providing clear justification for technical decisions. Key Skills … Experience Strong experience in data pipeline development and orchestration. Proficient with cloud platforms (Snowflake, AWS fundamentals). Solid understanding of data architecture, warehousing, and modeling. Programming expertise: Python, Spark, SQL, API integration. Knowledge of ML/AI frameworks, MLOps, and advanced analytics concepts. Experience with CI/CD, data testing frameworks, and versioning More ❯

Job Title: Microsoft Viva Data & Analytics Specialist Contract Type: Inside IR35 Rate: £598 per day Location: Fully Remote Duration: [Insert duration if known, e.g., 6 months] Overview We're looking for a highly skilled data professional to design and develop usage analytics solutions across Microsoft Viva modules- Connections, Engage, Insights, Learning, Goals, and Topics . This … shape how employee engagement and productivity are measured across a global organisation. Key Responsibilities Design and implement analytics solutions to track usage across all Microsoft Viva modules. Integrate data from Microsoft Graph, Viva Insights, and Microsoft 365 usage reports. Build interactive dashboards in Power BI for executive and functional reporting. Automate data extraction and transformation using … with Microsoft Viva, Microsoft Graph, and Microsoft 365 data sources. Strong proficiency in Power BI, including DAX and data modelling. Experience with data pipeline automation tools such as Power Automate and Azure Data Factory. Solid understanding of data governance, privacy, and compliance standards. Ability to translate business requirements into More ❯

Who We Are TetraScience is the Scientific Data and AI company. We are catalyzing the Scientific AI revolution by designing and industrializing AI-native scientific data sets, which we bring to life in a growing suite of next gen lab data management solutions, scientific use cases, and AI-enabled outcomes. TetraScience is the category … leader in this vital new market. In the last year alone, the world's dominant players in compute, cloud, data, and AI infrastructure have converged on TetraScience as the de facto standard, entering into co-innovation and go-to-market partnerships: Latest News and Announcements | TetraScience Newsroom In connection with your candidacy, you will be asked to carefully … EU. The ideal candidate should have expertise in customer and field enablement, adult learning principles, educational program creation, use case architecture and implementation, and a genuine passion for Data and AI technologies. The candidate should be driven by a desire to educate and inspire, and have the technical skills to quickly prototype new ideas and solutions. And finally More ❯

Lead Data Engineer London, UK (Hybrid) Innovative FinTech About the Company: We are partnered with a high-growth FinTech company headquartered in London that is redefining how technology and data can transform financial services. Their mission is to deliver smarter, faster, and more secure financial solutions through cutting-edge platforms powered by data. With rapid expansion … and strong investment backing, they are scaling their engineering and data capabilities to support ambitious growth plans. The Role: Our client is seeking a Lead Data Engineer to take ownership of their data infrastructure and lead the build-out of scalable, reliable, and secure data pipelines. This is a senior hands-on … role, where you'll also provide technical leadership to a small but growing team. You'll work closely with data scientists, analysts, and software engineers to ensure the company's data strategy underpins their innovative financial products. Key Responsibilities: Lead the design, development, and optimisation of data pipelines and ETL processes. Architect scalable data More ❯

Data Scientist Location: Hybrid (Greater Bristol Area) Salary: £54,000 Python - PySpark - Azure - Pandas - Scikit-learn - TensorFlow - PyStats - Data Science - Power BI We're supporting a growing, forward-thinking organisation in their search for an experienced Data Specialist. This is an exciting opportunity to join a dynamic team at a pivotal point in its … growth, helping shape data strategy, deliver insightful analytics, and drive intelligent decision-making across the business. You'll work closely with stakeholders to uncover challenges, identify opportunities, and deliver actionable, measurable solutions through advanced machine learning models, statistical methods, and high-performance data pipelines. Skills & Experience Required To be considered, you will need: Strong proficiency in … real-world performance trade-offs. Experience building and maintaining end-to-end machine learning applications. Hands-on experience with Azure cloud technologies: Synapse, Fabric, AzureML, ADX, ADF, Azure Data Lake Storage, Event Hubs. Experience with visualisation tools such as Power BI and Streamlit. Familiarity with Parquet and Delta Parquet formats. Strong data modelling and architecture knowledge More ❯

We are hiring a senior Data Engineer to lead the development of intelligent, scalable data platforms for Industry 4.0 initiatives. This role will drive integration across OT/IT systems, enable real-time analytics, and ensure robust data governance and quality frameworks. The engineer will collaborate with cross-functional teams to support AI/… ML, GenAI, and IIoT use cases in manufacturing and industrial environments. Key Responsibilities Architect and implement cloud-native data pipelines on AWS or Azure for ingesting, transforming, and storing industrial data. Integrate data coming from OT systems (SCADA, PLC, MES, Historian) and IT systems (ERP, CRM, LIMS) using protocols like OPC UA, MQTT, REST. Design and … implement profiling, cleansing, standardization, and anomaly detection frameworks. Security & Compliance:Knowledge of data privacy, access control, and secure data exchange protocols. Defining and creating MLOPs pipeline Good to Have Skills GenAI Exposure:Experience with LLMs, LangChain, HuggingFace, synthetic data generation, and prompt engineering. Digital Twin Integration:Familiarity with nVidia Omniverse, AWS TwinMaker, Azure More ❯

depending on experience) + benefitsSecurity: Must be able to undergo security clearanceAbout the RoleAs a Lead DataOps Engineer, you'll be at the forefront of delivering automated, scalable data solutions that power modern analytics and applications. This role combines the best of data engineering and DevOps, focusing on containerized environments, orchestration … and continuous delivery. You'll lead initiatives to streamline data workflows, optimize performance, and ensure reliability across cloud platforms-while guiding teams with strong technical leadership.Key Responsibilities* Pipeline Automation: Build and orchestrate data workflows using tools like Airflow, Prefect, or Dagster.* Containerization: Package and deploy data applications using Docker and Kubernetes (including EKS … your application with you before presenting it to any potential employer.Please note we are on the client's supplier list for this position.KeywordsLead DataOps Engineer, DataOps, Data Pipeline Automation, Airflow, Prefect, Dagster, Docker, Kubernetes, EKS, AKS, CI/CD, Terraform, Ansible, Grafana, Prometheus, Spark, Kafka, PostgreSQL, Infrastructure as Code, Cloud Data Engineering, Hybrid Working, Security More ❯

+ benefits Security: Must be able to undergo security clearance About the Role As a Lead DataOps Engineer, you'll be at the forefront of delivering automated, scalable data solutions that power modern analytics and applications. This role combines the best of data engineering and DevOps, focusing on containerized environments, orchestration, and … continuous delivery. You'll lead initiatives to streamline data workflows, optimize performance, and ensure reliability across cloud platforms-while guiding teams with strong technical leadership. Key Responsibilities * Pipeline Automation: Build and orchestrate data workflows using tools like Airflow, Prefect, or Dagster. * Containerization: Package and deploy data applications using Docker and Kubernetes (including EKS … you before presenting it to any potential employer. Please note we are on the client's supplier list for this position. Keywords Lead DataOps Engineer, DataOps, Data Pipeline Automation, Airflow, Prefect, Dagster, Docker, Kubernetes, EKS, AKS, CI/CD, Terraform, Ansible, Grafana, Prometheus, Spark, Kafka, PostgreSQL, Infrastructure as Code, Cloud Data Engineering, Hybrid Working, Security More ❯

Senior Data Engineer - Cardiff/Hybrid - £65,000 - £75,000 + benefits Yolk Recruitment are proud to be supporting a leading global business investing heavily in its data and analytics capabilities. They're looking for a Senior Data Engineer to help shape the next generation of their data platform - leading technical design … mentoring others, and driving best practice. This is a great opportunity for an experienced data professional who enjoys solving complex challenges, optimising large-scale systems, and influencing strategy within a collaborative, forward-thinking team. What you'll be doing: Lead the design and implementation of scalable, high-performance data architectures and pipelines. Define and enforce best … practices for data engineering, including coding standards, testing, and documentation. Mentor and guide engineers, fostering collaboration and technical excellence. Translate complex business requirements into reliable, well-structured data solutions. Optimise data workflows for performance, reliability, and cost efficiency. Drive adoption of modern data tools and technologies across the organisation. Ensure robust data More ❯

Data Engineer - FinTech - Newcastle (Tech stack: Data Engineer, SQL, Python, AWS, Git, Airflow, Data Pipelines, Data Platforms, Programmer, Developer, Architect, Data Engineer) Our client is a trailblazer in the FinTech space, known for delivering innovative technology solutions to global financial markets. They are expanding their engineering capability in Newcastle and … are looking for a talented Data Engineer to join their team. This role will focus on building and optimising systems that make complex datasets accessible, reliable, and valuable for the business. As a Data Engineer, you will take responsibility for the development of high-quality pipelines that process and manage large volumes of data from a range of external and internal sources. You'll play a key role in enhancing and maintaining their central data platform, ensuring the smooth delivery of information that supports investment decision-making. Working closely with stakeholders across the business, you'll help shape how data is accessed, tested, and leveraged to maximise value. More ❯

Data Engineer | Hybrid | London | Databricks | Azure | 85k I'm working with a global powerhouse that's been setting the standard for excellence for over 60 years. With more than 1,000 projects delivered worldwide and a combined value exceeding $150 billion, they've earned a reputation as a trusted leader in high-value, complex projects. Today, their … you work each week. This is an organisation where world-class projects meet an environment that prioritises your well-being and career development. I'm looking for a Data Engineer who thrives on innovation and loves tackling complex data challenges. If building scalable, cloud-based solutions excites you, this is your chance to make a real … impact. You'll work with cutting-edge technology and stay at the forefront of the data engineering field. You'll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards and analytics ETL & Data Modelling: T-SQL, metadata-driven pipelines Design and More ❯

Data Engineer | Hybrid | London | Databricks | Azure | 85k I'm working with a global powerhouse that's been setting the standard for excellence for over 60 years. With more than 1,000 projects delivered worldwide and a combined value exceeding $150 billion, they've earned a reputation as a trusted leader in high-value, complex projects. Today, their … you work each week. This is an organisation where world-class projects meet an environment that prioritises your well-being and career development. I'm looking for a Data Engineer who thrives on innovation and loves tackling complex data challenges. If building scalable, cloud-based solutions excites you, this is your chance to make a real … impact. You'll work with cutting-edge technology and stay at the forefront of the data engineering field. You'll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards and analytics ETL & Data Modelling: T-SQL, metadata-driven pipelines Design and More ❯

Data Engineer Salary: up to £35,000 Location: Home-based with occasional travel to the Newark office/UK Full time: 35 hours per week (Mondays to Fridays) Maternity Cover Contract until April 2027 Closing date for applications: 30 November 2025 First interview: 9/10 December 2025 Second interview: 16 December 2025 About Us The Wildlife Trusts … bold changes in the way The Wildlife Trusts work, not least in how we mobilise others and support them to organise within their own communities. About You As Data Engineer, you will play a crucial role in supporting the work of The Wildlife Trusts by maintaining and enhancing our data pipelines and infrastructure. You will be … responsible for ensuring the effectiveness, efficiency, and scalability of the federation wide data service system, a data lakehouse, providing key insights and solutions to support decision-making across the organisation and federation, responding to the needs of Wildlife Trusts and the delivery of our 2030 Strategy. Working collaboratively with various teams, you will develop data More ❯

Master Data Management (MDM) Analyst Manchester - Hybrid, Up to £50K We're looking for a Master Data Management (MDM) Analyst to play a pivotal role in maintaining and enhancing the organisation's master data assets. This is an exciting opportunity to join a growing data team and directly contribute to building a … single, trusted view across the business. Reporting to the MDM Manager, you'll help ensure that data powering critical business systems is clean, consistent and reliable - enabling smarter decisions and greater efficiency across the organisation. The Role As an MDM Analyst, you'll support the daily operation of the Tamr platform, monitor data pipelines, validate records … and resolve data quality issues. You'll collaborate with Finance, Operations and Analytics teams to align data across systems and ensure it supports accurate reporting and business processes. You'll also help evolve MDM standards, contribute to data governance initiatives and work closely with both technical and business stakeholders to continually improve how data More ❯

committed to building a diverse team that reflects the industries and societies we serve. Overview We are looking for a mission-driven, technology-savvy and delivery-focused Senior Data Engineer. Your role will be dual: you will put in place Cogna's BI infrastructure (data ingestion, data warehouse, and reporting capabilities) and come up … with the data architecture for Cogna's internal performance and productivity metrics; you will architect and implement the scalable ETL layer that will allow us to eventually support many tens of thousands of applications we'll develop for our customers, capable of running hundreds of thousands of ETL jobs daily. What you'll do Work with Product and … Engineering leaders to define the needs for Cogna's BI infrastructure and come up with a clean data architecture. Implement a state of the art data collection, data warehouse and reporting solution that will help us track and optimize internal productivity metrics as well as external metrics for our apps. This will allow us More ❯

Data Engineer Leicester (remote) £65,000 to £75,000 Permanent I am currently looking for a Data Engineer to join a forward-thinking organisation, where you will play a pivotal role in building a modern, enterprise-scale data platform. Data Engineer - What will you be doing? * Designing and implementing modern data architectures including lakehouse and medallion models using Azure and Databricks.* Developing, maintaining, and optimising data pipelines, quality validation tools, and lineage tracking systems to guarantee high-quality data outputs.* Manage MS SQL and Azure SQL databases efficiently while ensuring optimal performance across all environments.* Utilising Azure Data Factory, Python, and other tools … enterprise environments.* Power BI experience.* Deep understanding of SQL.* Experience with Azure and Databricks.* Proficient Python programming skills.* Demonstrated ability to design lakehouse architectures.* Experience implementing CI/CD pipeline management for reliable solution delivery. Robert Walters are proud to specialise in BI & Data recruitment across the UK, offering amazing opportunities on a permanent and interim basis. More ❯

Managing Consultant/Senior Manager - Data & AI Architecture At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our clients' challenges of today and tomorrow. Informed and validated by science and data. … Superpowered by creativity and design. All underpinned by technology created with purpose. Join our Data & AI Strategy team and be part of the future of AI consulting, where we're not just advising clients, we're actively disrupting traditional consulting models by embedding AI into every aspect of how we work. We're looking for changemakers with recent … sovereignty in enterprise architecture, to enabling the AI-empowered workforce of tomorrow, you'll be working at the cutting edge of what's next. We DESIGN next-generation data, analytics and AI platforms – You will lead the architecture and development of scalable, secure and cost-efficient platforms that power both operational and analytical workloads, leveraging leading cloud technologies More ❯

to achieve this for a 7th time across the coming period. Due to these increasing levels of high activity, they are generating more and more customer and performance data than ever before and are embarking on a key project to best manage & utilise this. An agile and robust data management and utilisation plan will be key … e.g., star/snowflake schemas) Administer Microsoft Fabric Lakehouse and Azure services Optimise performance across queries, datasets, and pipelines Apply data validation, cleansing, and standardisation rules Document pipeline logic and contribute to business-wide metadata catalogues Collaborate with BI, Change, and Product teams to align data outputs to needs Support groundwork for future data … requirements clearly across business teams will be key to success in this role. You’ll be a technically strong data professional with hands-on experience in pipeline development, cloud-based data infrastructure, and performance optimisation. You will be at your best when working collaboratively in fast-paced environments and when translating complex business needs More ❯

Snowflake Data Architect FTE London, UK (Hybrid) Roles & Responsibilities: Possess 15 plus years of work experience at a reputable Data & AI services firm. Outstanding written and verbal communication skills, with a flair for compelling storytelling. Analyse complex insurance data to identify trends, patterns, and insights that support strategic business decisions. Design and implement an … enterprise-grade Data Vault 2.0 architecture on Snowflake, ensuring scalability, flexibility, and auditability. Lead the end-to-end data architecture design, from data ingestion through transformation to reporting layers. Collaborate with business stakeholders to understand insurance-specific data requirements (eg, policy, claims, underwriting, premiums, risk, reinsurance). Define data modelling … standards, naming conventions, and governance frameworks aligned with industry best practices. Work with data engineering teams to implement dbt-based data pipelines for data transformation and lineage tracking. Ensure efficient data integration and performance optimization across Snowflake and downstream systems. Partner with BI developers to enable Power BI reporting and semantic modelling More ❯

Data Architect £75,000 - £80,000 - + Benefits & pension Central London - Hybrid working, office working 2 days per week This is a great opportunity for a talented Data Architect/Senior Data Engineer ready to step up into a Data Architect role for a well established, market leading company. Reporting into the … Head of Data Engineering & Data Platforms, you'll join an ambitious Data Engineering team who are designing and building robust, scalable foundations that power data driven decision making - using the latest tech. This is a hands on role with great scope for growth and impact. You will bring solid experience: Designing and … building robust data pipelines to bring internal and external raw data sources into AWS cloud-based storage and processing platforms Defining and designing scalable, future-ready solutions that deliver strong and flexible data architecture that supports across the business Developing and executing a migration plan for legacy data into Snowflake, ensuring a More ❯

Data Engineer Remote (Occasional Travel to Birmingham) Up to £75,000 + Benefits (12 Months FTC) Are you a hands-on Data Engineer who enjoys building, maintaining, and optimising modern data systems? We're working with a well-established UK business currently going through a major transition period, and they're looking for a … Data Engineer to help ensure their data platforms, models, and pipelines continue to run smoothly during this exciting phase of change. This is a home-based role with occasional travel to Birmingham - ideal for someone who enjoys autonomy, technical variety, and the satisfaction of keeping complex data systems reliable and efficient. Why this role … Work across a modern data stack - from ingestion to visualisation - using Python, SQL, Meltano, and Power BI . Join a small, high-impact team supporting pricing, revenue, and market analytics. Play a key role in maintaining and optimising the business's data warehouse and dynamic pricing model . Gain exposure to Azure cloud , containerisation, and DevOps More ❯

We are now starting to invest heavily in market adoption. We build the most accurate and efficient AI solutions by vertically integrating all layers of the stack. From data collection and curation, custom training software, model architectures, multimodal LLMs, pre- and postprocessing, and all the way down to the fastest inference engines. Not only does our team have … can help us develop state of the art AI products. This can involve anything from improving our training algorithms, training and integrating multimodal LLMs, building our data pipeline, designing new model architectures to using tried and tested ML approaches and coming up with clever algorithms. You will help us build new AI features that will be shipped … train and design more accurate models, while also enabling new and more complex AI applications on low-cost and low-power hardware. You will improve our data pipeline, model architectures and training software. Sometimes there is relevant literature available, but novel approaches and clever hacks are often required for the problems that we are working on. You More ❯