1 to 25 of 6,172 Data Pipeline Jobs in England

Director , Data Governance Location: Global Reporting to: VP - Data Excellence WHO WE ARE Choreograph is WPP's global data products and technology company. Our mission is to transform marketing by building the fastest, most connected data platform that bridges marketing strategy to … scaled activation. We partner with agencies and clients to unlock the full value of data by combining technology, data, and analytics expertise. This is delivered through the Open Media Studio, an AI-enabled media and data platform designed for the future of advertising. Our … to ensure data governance principles are integrated into all aspects of the data platform development and operation. Data Pipeline Governance: Implement governance controls over data pipelines, ensuring data lineage, data quality, and security are maintained throughout the More ❯

Join to apply for the Senior Data Engineers role at Foreign, Commonwealth and Development Office 1 week ago Be among the first 25 applicants Join to apply for the Senior Data Engineers role at Foreign, Commonwealth and Development Office London or East Kilbride - You will be … UK-based staff work in King Charles Street, London, Abercrombie House in East Kilbride and in Milton Keynes. These roles contribute to FCDOs Data Strategy to improve insights to the business by provisioning it with reliable and trustworthy data sources and improving their delivery in a … secure manner in compliance with Data regulations. The team is primarily Azure focused but other Cloud technologies such as Oracle and Amazon Web Services (AWS) are used in FCDO. The successful candidates will build complex data pipelines (both Extract Transform and Load {ETL} and Extract Load More ❯

engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources … effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote … SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality More ❯

engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources … effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote … SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality More ❯

engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources … effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote … SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality More ❯

engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources … effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote … SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality More ❯

Job Description Job Title: Data Architect Location: London - 3 days travel to office SC Cleared: Required Job Type: Full-Time Experience: 10+ years Job Summary : We are seeking a highly experienced and visionary Data Architect to lead the design and implementation of the data … our cutting-edge Azure Databricks platform focused on economic data. This platform is crucial for our Monetary Analysis, Forecasting, and Modelling efforts. The Data Architect will be responsible for defining the overall data strategy, data models, data governance framework, and data integration patterns. This role requires a deep understanding of data warehousing principles, big data technologies, cloud computing (specifically Azure), and a strong grasp of data analysis concepts within the economic domain. Key Experience: Extensive Data Architecture Knowledge: They possess a More ❯

advantage, and its technology-led service provides access to all major exchanges, order-flow management via screen, voice and DMA, plus award-winning data, insights and analytics. The Technology Department delivers differentiation, scalability and security for the business. Reporting to the COO, Technology provides digital tools, software services … our enterprise-wide services to end users and actively manages the firm's infrastructure and data. Within IT, Marex Technology has established a Data team that enables the firm to leverage data assets to increase productivity and improve business decisions, as well as maintain data compliance. The Data Team encompasses Database Administration, Data Engineering, Data Analysis, Data Architecture, Data Intelligence and AI expertise. In recent years, they have developed a Data Lakehouse architecture, that is relied upon by different departments across More ❯

on financial and tax planning, offering home finance and annuities propositions, and providing collective fund solutions to third party customers. Job Summary The Data Engineer is a hands-on technical role responsible for designing, developing, and maintaining data pipelines within the IT department. The pipelines will … be realised in a modern lake environment and the engineer will collaborate in cross-functional teams to gather requirements and develop the conceptual data models. This role plays a crucial part in driving data-driven decision-making across the organisation, ensuring data availability, quality … guiding and upskilling more junior data engineers and setting data standards and guidelines. What you'll do Data Pipeline Development Design, model, develop and maintain data pipelines to ingest, store, process, and present data. Ensure data quality, accuracy, and More ❯

on financial and tax planning, offering home finance and annuities propositions, and providing collective fund solutions to third party customers. Job Purpose The Data Engineer is a hands-on technical role responsible for designing, developing, and maintaining data pipelines within the IT department. The pipelines will … be realised in a modern lake environment and the engineer will collaborate in cross-functional teams to gather requirements and develop the conceptual data models. This role plays a crucial part in driving data-driven decision-making across the organisation, ensuring data availability, quality … guiding and upskilling more junior data engineers and setting data standards and guidelines. What you'll do Data Pipeline Development Design, model, develop and maintain data pipelines to ingest, store, process, and present data. Ensure data quality, accuracy, and More ❯

Central Operations Enterprise Data Architect - SVP Central Operations Enterprise Data Architect - SVP Overview of Citi: Citi is a world-leading global bank. We have approximately 200 million customer accounts and a presence in more than 160 countries and jurisdictions worldwide. We provide consumers, corporations, governments, and … and is responsible for driving process transformation, risk reduction, and efficiencies through best-in-class technology solutions. Including, development of shared services, comprehensive data architecture, automation, end-to-end solution design for large scale transformation initiatives, and adoption of Generative AI solutions. Job Purpose : The Central Operations Enterprise … Data Architect role is a critical hire in the group to become part of the team that drives data strategy, architecture, and design across critical initiatives -payments services, global reconciliations, reg. reporting, liquidity management, and strategic general ledger. The role requires hands on experience in designing More ❯

advantage, and its technology-led service provides access to all major exchanges, order-flow management via screen, voice and DMA, plus award-winning data, insights and analytics. The Technology Department delivers differentiation, scalability and security for the business. Reporting to the COO, Technology provides digital tools, software services … our enterprise-wide services to end users and actively manages the firm’s infrastructure and data. Within IT, Marex Technology has established a Data team that enables the firm to leverage data assets to increase productivity and improve business decisions, as well as maintain data compliance. The Data Team encompasses Data Analysis, Data Architecture, Data Intelligence and Machine Learning expertise. In recent years, they have developed a Data Lakehouse architecture, that is relied upon by different departments across the firm. Marex now seeks More ❯

and is responsible for driving process transformation, risk reduction, and efficiencies through best-in-class technology solutions. Including, development of shared services, comprehensive data architecture, automation, end-to-end solution design for large scale transformation initiatives, and adoption of Generative AI solutions. Job Purpose : The Central Operations Enterprise … Data Architect role is a critical hire in the group to become part of the team that drives data strategy, architecture, and design across critical initiatives -payments services, global reconciliations, reg. reporting, liquidity management, and strategic general ledger. The role requires hands on experience in designing … executing, and managing enterprise-wide data architecture, including design of frameworks to standardize data management, consumption, and data quality controls. Successful candidate will be working cross disciplines in engineering, architecture, and technology teams across Operations, Finance, CDO, and Technology functions to simplify data More ❯

View these similar available jobs instead: Date Added: Yesterday Data Manager Job DescriptionThis is an exciting opportunity with candidates to start and interview as soon as possible!The Data Manager must have minimum 1 year clinical research experience as they need an individual who can work … Day to day responsibilities will involve... Job DescriptionThis is an exciting opportunity with candidates to start and interview as soon as possible!The Data Manager must have minimum 1 year clinical research experience as they need an individual who can work independently as well as part of a … Day to day responsibilities will involve... Job DescriptionThis is an exciting opportunity with candidates to start and interview as soon as possible!The Data Manager must have minimum 1 year clinical research experience as they need an individual who can work independently as well as part of a More ❯

for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that … engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data … You will work closely with cross-functional teams and contribute to the strategic direction of our data initiatives. RESPONSIBILITIES Data Pipeline Development: Lead the design, implement, and maintain scalable data pipelines for ingesting, processing, and transforming large volumes of data from More ❯

Data Pipeline Development: Design and implement end-to-end data pipelines in Azure Databricks, handling ingestion from various data sources, performing complex transformations, and publishing data to Azure Data Lake or other storage services. Write efficient and standardized Spark … SQL and PySpark code for data transformations, ensuring data integrity and accuracy across the pipeline. Automate pipeline orchestration using Databricks Workflows or integration with external tools (e.g., Apache Airflow, Azure Data Factory). Data Ingestion & Transformation: Build scalable data … for performance by tuning configurations, partitioning data, and caching intermediate results to minimize processing time and resource consumption. Continuously monitor and improve pipeline performance, addressing bottlenecks and optimizing for cost efficiency in Azure. Automation & Monitoring: Automate data pipeline deployment and management using tools like More ❯

UK-based staff work in King Charles Street, London, Abercrombie House in East Kilbride and in Milton Keynes. These roles contribute to FCDOs Data Strategy to improve insights to the business by provisioning it with reliable and trustworthy data sources and improving their delivery in a … secure manner in compliance with Data regulations. The team is primarily Azure focused but other Cloud technologies such as Oracle and Amazon Web Services (AWS) are used in FCDO. The successful candidates will build complex data pipelines (both Extract Transform and Load {ETL} and Extract Load … and Transform {ELT}) in the Azure cloud platforms. You will work with structured and unstructured data, data lakes, data warehouses to service operational and analytical business needs. Job description The successful candidates will: Work collaboratively with Enterprise Data Colleagues to develop pipelines More ❯

for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that … engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data … Load) processes, and big data technologies it becomes possible to develop robust and reliable data solutions. RESPONSIBILITIES Data Pipeline Development: Design, implement, and maintain scalable data pipelines for ingesting, processing, and transforming large volumes of data from various sources More ❯

more exclusive features. Direct message the job poster from Tata Consultancy Services Recruiter - Leadership & Strategic Hiring UK&I at Tata Consultancy Services Role : Data Architect Job Type: Permanent (Hybrid) Location: London, United Kingdom Ready to utilize your skills in designing, creating, and managing data architecture? Join … us as a Data Architect. The Role As a Data Architect, you will be responsible for designing, creating, and managing data architecture. You will also ensure that data is efficiently and securely stored, organized, and accessible across the enterprise. Build the foundation … for databases, data warehouses, data lakes, and other data storage solutions, ensuring they meet both business and technical requirements. Key responsibilities: Develop and design the data architecture framework for the organization. Create models for databases, data warehouses, data More ❯

as core platform features, ensuring enterprises have built-in compliance and data integrity controls. Develop native observability and automation tools to monitor pipeline performance, detect anomalies, and proactively enforce data governance policies. Ensure the platform meets enterprise security, compliance, and scalability requirements, making Workato the … classification, transformation recommendations, and self-healing pipelines that minimize operational overhead. Integrate predictive analytics and semantic enrichment to automate data mapping, improve pipeline efficiency, and surface actionable insights. Work with AI research teams to infuse machine learning into Workato’s data services, driving continuous optimization … data analytics, and competitive insights to inform product decisions. Experience analyzing large-scale data platforms to optimize usage trends and pipeline performance. Educational Background Bachelor’s Degree in Computer Science, Engineering, Data Science, or a related field. An MBA or advanced degree is More ❯

Create and maintain optimal data pipeline architecture. Assemble large, complex data sets that meet functional/non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Build the infrastructure … variety of data sources using SQL and AWS 'big data' technologies. Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics. Work with stakeholders including the Executive, Product, Data … data tools: Hadoop, Spark, Kafka, etc. Experience with relational SQL and NoSQL databases, including Postgres and Cassandra. Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc. Experience with AWS cloud services: EC2, EMR, RDS, Redshift. Experience with stream-processing systems: Storm, Spark More ❯

Site Name: UK – London – New Oxford Street Posted Date: Jun 10 2025 As a Business Systems Data Architect, you will collaborate closely with the Regulatory team to drive the architectural vision for regul atory data and its consumption and usage . This role requires a deep … understanding of modern data architectures, cloud technologies, and knowledge of AI/ML frameworks . You will play a critical role in shaping and overseeing the Regulatory data architecture utilising existing cloud data technologies, data pipelines, the management of structured, semi-structured … and unstructured data and data quality which is critical to the success of GSK’s AI initiatives. We create a place where people can grow, be their best, be safe, and feel welcome, valued and included. We offer a competitive salary, an annual bonus based on More ❯

Social network you want to login/join with: Data Analytics Engineer Whitehall Resources are currently looking for a Data Analysis Engineer. This role will be Inside of IR35, so you will be required to use an FCSA Accredited Umbrella Company. Key Requirements: - As an Analytics … Engineer, you'll have the chance to write great SQL, design complex data models, collaborate with a talented team, and deliver robust data products alongside Data Architects, Data Scientists, Product Managers, and Data Engineers. - You will bridge the gap between … data engineering, data architecture and data analysis. - Your focus will be on building robust and scalable data workflows to help business functions use internal data assets in an accessible and repeatable manner. - This role offers a unique opportunity to combine More ❯

Join to apply for the Senior Data Engineer role at Gamingtec 2 weeks ago Be among the first 25 applicants Join to apply for the Senior Data Engineer role at Gamingtec Get AI-powered advice on this job and more exclusive features. Role Overview As a … for real-time events using Kafka and related technologies; Create data validation processes to ensure data quality and integrity; Optimise pipeline performance and resource utilisation. Data Modeling & Transformation Implement dimensional data models following best practices and team standards; Develop standardised transformations … transformation frameworks; Hands-on experience with AWS data services (Redshift, S3, Glue, Lambda); Proficiency with Python for data processing and pipeline development; Experience with workflow orchestration tools like Airflow or similar; Knowledge of streaming data technologies such as Kafka; Familiarity with infrastructure-as More ❯

Join to apply for the Lead Data Engineer/Architect role at Insurance Office of America 1 day ago Be among the first 25 applicants Join to apply for the Lead Data Engineer/Architect role at Insurance Office of America Get AI-powered advice on … of data platforms, including data lakes, data warehouses, and other data processing systems. Data Pipeline Development: Designing, building, and maintaining robust data pipelines for data ingestion, transformation, and loading. Assemble large, complex data … optimizing data delivery, and assisting in re-designing infrastructure for greater scalability, etc. Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics. Work with stakeholders including the IT operations, automation and More ❯