26 to 50 of 917 Data Pipeline Jobs in the UK

platforms so they are reliable, secure, and easy to change. You will work across engineering and operations to automate delivery, improve data pipeline observability, and embed good governance so analytics and AI workloads can run at scale. You will be part of the Data Platforms … restricted environments. Delivering data applications in an automated and continuous fashion. Utilising tools like Airflow for orchestration and automating data pipeline deployments using DevOps tools and techniques (Docker, Containers, Terraform, Kubernetes) Building data pipelines using both cloud-native and open-source data ...

Essential Skills Strong SQL skills, including complex joins, aggregations, subqueries and query optimisation Strong understanding of ETL/ELT concepts and data pipeline design Hands-on experience with Apache Kafka or similar streaming platforms Strong knowledge of AWS data services and managed databases Experience with … from internal and third-party APIs Strong data modelling and data warehousing knowledge Ability to troubleshoot data, pipeline, performance and reliability issues Active SC clearance OR BPSS Desirable Skills Experience working in DDaT-aligned digital delivery environments Experience with Swagger/OpenAPI ...

Will Be Working WithThe Data Platforms team is part of the Insights and Data Global Practice and has seen strong growth and continued success across a variety of projects and sectors. Data Platforms is the home of the Data Engineers, Platform Engineers … Solutions Architects and Business Analysts who are focused on driving our customers digital and data transformation journey using the modern cloud platforms.We specialise on using the latest frameworks, reference architectures and technologies using AWS, Azure and GCP along with various data platforms like Databricks, Snowflake, Quantexa ...

Data Architect Shape the future of enterprise data architecture within a modern cloud environment. We are seeking an experienced Data Architect to play a key role in designing, developing, and evolving a modern data platform. This position offers the opportunity to drive … strategic data initiatives, implement Data Mesh principles, and help create scalable, secure, and high-performing data solutions that support business growth. The Role As a Data Architect, you will be responsible for defining and governing data platform architecture, ensuring data ...

Write efficient SQL queries and manage large datasets Ensure data quality, integrity, and security across systems Monitor and troubleshoot data pipeline performance issues Collaborate with data analysts, data scientists, and software engineers Implement data governance and best practices Support … Strong proficiency in SQL and database technologies Experience with programming languages such as Python, Java, or Scala Hands-on experience with data pipeline tools (e.g., Apache Airflow, Kafka) Familiarity with cloud platforms (AWS, Azure, Google Cloud) Understanding of data warehousing solutions (e.g., Snowflake, Redshift, BigQuery ...

observable data pipelines supporting critical Commodities datasets Modernize legacy workflows, reduce technical debt, and improve performance, reliability, and maintainability. Design automated pipeline controls for validation, monitoring, schema change, exception handling, and data integrity. Develop workflow orchestration, alerting, observability, and remediation processes. Translate business and client … Finance or other STEM related field or degree‐equivalent qualifications 4+ years of experience designing and building scalable data solutions, ETL pipelines, data workflows, and monitoring frameworks. Strong hands‐on experience with Python or similar programming/scripting languages. Experience with querying structured, semi‐structured ...

largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Who You Will Be Working With The Data Platforms team is part of the Insights and Data Global Practice and has seen strong growth and continued success across a variety … projects and sectors. Data Platforms is the home of the Data Engineers, Platform Engineers, Solutions Architects and Business Analysts who are focused on driving our customers digital and data transformation journey using the modern cloud platforms. We specialise on using the latest frameworks, reference ...

Venesky-Brown’s client, a public sector organisation in Glasgow, is currently looking to recruit Data Modeller for initial 8 month contracts with potential to extend on a rate of £526/day (Inside IR35). These roles will be hybrid working with a minimum of 2 days … week in their Glasgow office. Responsibilities: - The Data Modeller will be responsible for designing, developing, and maintaining high-quality data models and datamarts that support analytical and reporting needs. They will translate business requirements into scalable, performant, and well-structured data models that enable ...

Director of Supply Chain Data Solutions – London Reporting to the CTO. Qualifications Bachelor’s degree in Data Science, Supply Chain Management, Business Administration, or related field; advanced degree preferred. 10+ years of experience in data analytics, data governance, or enterprise data management with a focus on supply chain operations. Proven experience delivering dashboards, analytics tools, and data solutions that drive business outcomes. Strong knowledge of data governance frameworks, standards, and best practices. Expertise in enterprise data management, including data modeling, integration ...

About the Role We are seeking an experienced Data Engineer with deep expertise in Snowflake to join a growing Data & Analytics function. This is an exciting opportunity to play a pivotal role in designing, developing and optimising an enterprise data platform that underpins business … intelligence, reporting and analytics across the organisation. The successful candidate will be responsible for building scalable, high-performance data solutions within Snowflake , designing robust data architectures, and developing modern data pipelines that transform data into trusted, business-ready information. This role will ...

Principal Data Engineer – Hybrid/Remote – (SC Cleared) THE COMPANY: A leading organisation undergoing a significant digital transformation is looking to appoint a Principal Data Engineer to play a key role in shaping its enterprise data platform and future analytics capabilities. Joining a growing … Data & Integration function, you’ll work alongside architects, product owners, and engineering teams to deliver modern, cloud-based data solutions that support business-critical decision-making. This is an opportunity to influence technical strategy, establish engineering standards, and drive the adoption of modern DataOps and cloud ...

largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Who You'll Be Working With As a Data Solution Architect at Capgemini, you will design and lead the delivery of modern data platforms that enable analytics, AI, GenAI, and data … Fabric experiences to operationalise models and drive business insights. Design, monitor and optimise data pipelines and platform performance: Use Fabric capacity management, pipeline monitoring, logging, and operational best practices to improve reliability, throughput, cost, and scalability. Collaborate with cross-functional teams: Work closely with business analysts, data ...

sends, logistics, offers, and reporting, but also the analytical core that makes it work, building the response model and sourcing the data pipeline and user-data enrichment that let us target the right customer, with the right offer, at the right moment. … modelling, automation) Desirable Experience in financial services, regulated or other consumer categories where offers and eligibility matter Experience standing up a data pipeline or enrichment stack (first-party + third-party/bureau data) Experience building a channel or function from scratch Familiarity with credit ...

employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Senior Data Engineer Position Description At CGI, we're helping clients unlock the full value of their data by building modern, scalable platforms that … modelling activities with Data Analysts •Monitor & improve data quality, lineage and platform reliability •Troubleshoot & resolve data pipeline issues across development and production environments •Apply & promote Agile delivery and DevOps engineering practices •Maintain & document technical solutions and data integration processes ...

Role: Data Engineer Location: Remote but able to attend client sites in Doncaster/London Exp. Start Date: ASAP Contract Duration: 6 Months initially Rate: £500 per day (Umbrella) Job Specification: See attached Experience Oracle experience beneficial Snowflake experience Experience Working with Data teams IDMC, Kafka … data sources. Understanding of Data Architecture patterns/Data lakes/kimbol Practical experience of Platforms Cloud platforms structuring Data Relational database Snowflake going through the cloud platform Informatica experience Change data capture experience Working with various data ...

Principal Data Engineer - Secure Digital Platform (SDP) Data and Integration Location: Reading (hybrid working) Duration: Initially Until Feb '27 - but very likely to extend (new programme) (Inside IR35) Active SC Clearance Required Role details: We are looking for SC cleared Data Engineers to support … defence client on a contract basis. This role is focused on the Secure Digital Platform (SDP) Data and Integration, and will be based in Reading with hybrid working arrangements. The initial contract will run until February 2027, with a strong likelihood of extension as this is a long ...

largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Who You'll Be Working With As a Data Solution Architect at Capgemini, you will design and lead the delivery of modern data platforms that enable analytics, AI, GenAI, and data … ready architectures, taking end to end ownership from vision through to delivery and value realisation You will be part of the Data Platforms team that is part of the Insights and Data Global Practice that has seen strong growth and continued success across a variety ...

Overview This is an exciting opportunity for a technically minded Data Engineer to join a growing and innovative organisation working at the forefront of data-driven solutions. Youll play a key role in building, maintaining, and optimising data systems that support operational performance … reporting. This role offers exposure to modern data tools, cloud technologies, and large-scale datasets, making it ideal for someone looking to develop a career in data engineering. Data Engineer Key Responsibilities Data Integration & ETL Design, develop, and optimise ETL processes ...

Senior Data Engineer Market Data 5 day onsite We are looking for a Senior Data Engineer to build and operate a high-performance market data platform supporting quantitative research and production trading systems. You'll design scalable data pipelines, process … engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience ...

Description We are seeking an experienced Data Engineer to join our Group Technology team on a 9-month contract. You’ll play a key role in delivering Connells Group Reporting Data, covering medallion architecture, data modelling, design, and pipelines. You’ll work closely with … Maintain the Common Data Model to ensure organisation-wide consistency. Apply Agile principles for iterative and collaborative development. Ensure data pipeline quality, reliability and performance. Develop, test and implement monitoring to ensure effective operation. Support cross-functional data projects, providing expertise as required. ...

Director of Data and AI Department: Platform Employment Type: Permanent Location: London Reporting To: James Complin Description We’re seeking a strategic Director of Data and AI to lead, develop, and empower our Data and AI teams. This role balances leadership with technical excellence … define how data, analytics, machine learning, and generative AI enable the wider business. You’ll work closely with senior stakeholders, translating business objectives into a clear Data and AI strategy, and ensuring the team delivers scalable, trusted, secure, and commercially impactful solutions. The ultimate goal will ...

SENIOR DATA ENGINEER Salary up to £100,000 + Equity London-based role - hybrid working with 4 days per week in the office Join a fast-growing technology business building modern, cloud-based data platforms Develop scalable data infrastructure using Python, SQL and cloud … technologies Apply online or contact Chelsea Hackett via ABOUT THE CLIENT Due to continued growth, we're recruiting for a Senior Data Engineer to join an ambitious technology business with exciting expansion plans. You'll work as part of a collaborative engineering team, helping to design and deliver ...

# AWS Data Solution ArchitectBirmingham, Bristol, Newcastle Bank House, Manchester, LondonApply for this job* Permanent* Experienced Professionals* Architecture* ID 466372-en\_GB## **Who You'll Be Working With**As an AWS Data Solution Architect at Capgemini, you will design and lead the delivery of modern data platforms that enable analytics, AI, GenAI, and data driven decision making at scale. You will translate business strategy into robust, governed, and AI ready architectures, taking end to end ownership from vision through to delivery and value realisationYou will be part of the Data ...

Lead Data Engineer/Data Tech Lead Permanent, Leeds/Hybrid Who are we? Enablis is a technology delivery consultancy who partner with clients to accelerate digital roadmaps. By combining our deep knowledge and best practice experience with the client’s domain expertise, we are able … open the door to incredible opportunities and introduce you to a network of equally driven, like-minded professionals. Job Brief The Lead Data Engineer role is responsible for owning the technical direction, design and delivery of data platforms and data engineering solutions ...

automation in data engineering — this is a small team that needs to move fast, and we expect the use of AI‐assisted pipeline development and management to be embedded in how we work Data Governance Lead the Data Governance function from a standing … alongside a technology partner or platform team (similar to our Ingenuity relationship) Demonstrated use of AI in data engineering contexts — AI‐assisted pipeline development, automated quality monitoring, or similar Strong leadership and stakeholder management skills — able to influence at Director and C‐suite level Experience ...