1 to 25 of 830 Data Pipeline Jobs in England

Job Description Job Title: Data Architect Location: London - 3 days travel to office SC Cleared: Required Job Type: Full-Time Experience: 10+ years Job Summary : We are seeking a highly experienced and visionary Data Architect to lead the design and implementation of the data architecture for our cutting-edge Azure Databricks platform focused on … economic data. This platform is crucial for our Monetary Analysis, Forecasting, and Modelling efforts. The Data Architect will be responsible for defining the overall data strategy, data models, data governance framework, and data integration patterns. This role requires a deep understanding of data warehousing principles, big data technologies, cloud computing (specifically Azure), and a strong grasp of data analysis concepts within the economic domain. Key Experience: Extensive Data Architecture Knowledge: They possess a deep understanding of data architecture principles, including data modeling, data warehousing, data integration, and data governance. Databricks Expertise More ❯

About Us: ATREIDES is a leader in advanced geospatial technologies, big data solutions, and data analytics. We specialize in providing cutting-edge software platforms that enable our clients to process, analyze, and visualize large-scale geospatial and sensor data for critical decision-making. Our mission is to deliver innovative, scalable, and high-performance platforms … that help organizations unlock the full potential of their data for various industries, including defense, national security, environmental monitoring, and urban planning. We are seeking a highly skilled Senior Data Engineer with a strong focus on big data, data analytics, and geospatial data intelligence to join our engineering team. In … and technologies (AWS, Azure, GCP) for big data processing and platform deployment. Strong knowledge of data warehousing, data lakes, and data pipeline design for large-scale data integration and storage. Familiarity with machine learning and AI techniques for data analytics (e.g., classification, regression, clustering, anomaly detection). More ❯

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data … modelling, ETL (Extract, Transform, Load) processes, and big data technologies it becomes possible to develop robust and reliable data solutions. RESPONSIBILITIES Data Pipeline Development: Design, implement, and maintain scalable data pipelines for ingesting, processing, and transforming large volumes of data from various sources using tools such as databricks More ❯

consistency, and proactively collaborating with upstream and downstream teams to enable seamless data flow across the organization. In this role, you will not only troubleshoot and resolve pipeline issues but also contribute to enhancing data architecture, implementing best practices in data governance and security, and ensuring the scalability and performance of data … stakeholders This position requires client presence between 25%-50% of the time per month at the client's office, which is located in London. Key Responsibilities: Data Pipeline Development & Maintenance Build, maintain, and optimize scalable ETL/ELT pipelines using tools such as Dagster, or similar. Ensure high data availability, reliability, and consistency through rigorous … data validation and monitoring practices. Collaborate with cross-functional teams to align data pipeline requirements with business objectives and technical feasibility. Automate data workflows to improve operational efficiency and reduce manual intervention. Data Integrity & Monitoring Perform regular data consistency checks, identifying and resolvinganomalies or discrepancies. Implement robust monitoring frameworks More ❯

best platform, people, and partners together to create limitless opportunities for growth. For more information, visit wppmedia.com About Choreograph: A Leading WPP Media Brand Choreograph is WPPs global data products and technology company. Were on a mission to transform marketing by building the fastest, most connected data platform that bridges marketing strategy to scaled activation. We … work with agencies and clients to transform the value of data by bringing together technology, data and analytics capabilities. We deliver this through the Open Media Studio, an AI-enabled media and data platform for the next era of advertising. Were endlessly curious. Our team of thinkers, builders, creators and problem solvers are over … strong, across 20 markets around the world. Role Summary and Impact We are seeking a meticulous and experienced Data Architect to play a crucial role in designing, documenting, and ensuring the quality of our next-generation media data lake on the Microsoft Azure platform. This role will focus on developing and upholding the architectural integrity of More ❯

best platform, people, and partners together to create limitless opportunities for growth. For more information, visit About Choreograph: A Leading WPP Media Brand Choreograph is WPP's global data products and technology company. We're on a mission to transform marketing by building the fastest, most connected data platform that bridges marketing strategy to scaled activation. … We work with agencies and clients to transform the value of data by bringing together technology, data and analytics capabilities. We deliver this through the Open Media Studio, an AI-enabled media and data platform for the next era of advertising. We're endlessly curious. Our team of thinkers, builders, creators and problem solvers … are over 1,000 strong, across 20 markets around the world. Role Summary and Impact We are seeking a meticulous and experienced Data Architect to play a crucial role in designing, documenting, and ensuring the quality of our next-generation media data lake on the Microsoft Azure platform. This role will focus on developing and upholding More ❯

Join to apply for the Data Architect (Trading) role at Keyrock Join to apply for the Data Architect (Trading) role at Keyrock About Keyrock Since our beginnings in 2017, we've grown to be a leading change-maker in the digital asset space, renowned for our partnerships and innovation. Data Architect About Keyrock Since … s progress with our research and governance initiatives. At Keyrock, we're not just envisioning the future of digital assets. We're actively building it. Position Overview The Data Architect is responsible for designing, implementing, and maintaining an organization's data architecture and strategy, ensuring that data is collected, stored, and processed efficiently and … securely to support business intelligence, data analytics, and machine learning operations (MLOps) practices. Key Responsibilities Designing Data Architecture: Plan and implement a robust, scalable data architecture that integrates data from various sources and supports diverse analytical needs, while optimizing costs and meeting business requirements. Implementing Data Engineering Pipelines: Design and More ❯

Senior Data Engineer - (Azure/Databricks) page is loaded Senior Data Engineer - (Azure/Databricks) Apply locations London - Scalpel time type Full time posted on Posted 7 Days Ago job requisition id REQ05851 This is your opportunity to join AXIS Capital - a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our … civil union status, family or parental status, or any other characteristic protected by law. Accommodation is available upon request for candidates taking part in the selection process. Senior Data Engineer (Azure/Databricks Job Family Grouping: Chief Underwriting Officer Job Family: Data & Analytics Location: London How does this role contribute to our collective success? The Data … services. Implement end-to-end data pipelines, ensuring data quality, data integrity and data security. Troubleshoot and resolve data pipeline issues while ensuring data integrity and quality. Implement and enforce data security best practices, including role-based access control (RBAC), encryption, and compliance with industry More ❯

Position : Senior Data Engineer Salary: £55,000 - £65,000 Location : Hybrid - Bristol, Manchester, or London Security Requirement : SC Clearance to start and be willing/able to obtain DV Clearance We are seeking Data Engineers, with a keen interest/experience in AI/ML and strong proficiency in Python Scripting. To join our Consultancy client … working on meaningful projects in the Defence and Security sector. Required Experience: End-to-End Data Development : Strong experience with data pipelines, ETL processes, and workflow orchestration, demonstrating best practices across tech stacks. Strong Python Skills Diverse Data Handling : Familiarity with batch, streaming, real-time, and unstructured data sources. Architectural Design & Systems … AI Integration : Apply data engineering tools, integration frameworks, and query engines to create high-quality, standardised data for AI applications and reporting. Data Pipeline Development : Design and implement robust data pipelines and stores in collaboration with other engineers and developers. Innovative Problem Solving : Bring fresh approaches to challenging data More ❯

throughout all levels of the client organization, generating results that allow our clients to thrive. What You'll Bring Join a dynamic team transforming our organization into a data-driven enterprise! The Data Layer Team is where we build essential data connectors, data pipelines, and capabilities to empower our clients and colleagues … with high-quality and reusable data. Our Data Layer Team focuses on creating scalable data solutions and advancing our data infrastructure that are the foundation of AI products that will drive informed decision-making across the company. As the Data Layer Connectors & Ingestion Delivery Owner, you will oversee data engineering … practices, data connectors, integrations, while driving continuous improvement, and ensuring alignment with data governance, security, and compliance standards. You will lead the Platform Build & Connectors squad compose of data engineers, architects, and you will closely collaborate with cross-functional teams to ensure timely delivery and operational excellence. Your responsibilities include defining the data More ❯

Data Architect (Contract) London, England, United Kingdom It has been determined that this role falls outside of IR35 THE ALLSAINTS TEAM At AllSaints we are in the business of feelings - making our customers feel cool and confident. If, like us, you believe in the power of a great outfit, we'd love to talk to you. We're … are a brand embarking on a very exciting plan and we are seeking new members of the team who want to contribute to this. THE ROLE As a Data Architect (contract) for Project Nova, you will be instrumental in shaping AllSaints' journey to become a truly data-driven, AI-powered organisation. You will lead the design … techniques and tools Strong hands-on experience with cloud data platforms, specifically Google Cloud Platform - BigQuery Experience with leading ETL/ELT tools and data pipeline orchestration (e.g., Dataflow, Apache Airflow, Talend, Informatica) Advanced SQL skills and deep knowledge of various database technologies (relational, columnar, NoSQL) Practical experience in establishing data governance frameworks More ❯

other characteristic protected by law. Accommodation is available upon request for candidates taking part in the selection process. How does this role contribute to our collective success? The Data & Analytics department transforms raw data into actionable insights to drive informed decision-making and optimize business operations. The Senior Azure Data Engineer will support these … goals by designing, implementing, and managing scalable data solutions on the Azure platform, ensuring efficient data processing, storage, and retrieval. You will play a key role in modernizing our data architecture, ensuring efficient data integration, and enabling advanced analytics to support critical business decisions. This role will enhance the department's ability … services. Implement end-to-end data pipelines, ensuring data quality, data integrity and data security. Troubleshoot and resolve data pipeline issues while ensuring data integrity and quality. Implement and enforce data security best practices, including role-based access control (RBAC), encryption, and compliance with industry More ❯

Meet the team behind this journey DeepL has millions of passionate users around the globe, both consumer and enterprise. These users produce a massive quantity of data. The Data Domain teams increase operational efficiency and drive innovation at DeepL by facilitating the flow of this data from producers to consumers, in order to maximize business impact. … Through the creation of stable and user-friendly analytical data models, we make customer data models and user behavioral data accessible and reusable for data professionals throughout the company. In this role, your main responsibilities and deliverables will be to build and maintain ETL pipelines, create foundational data models in … You may work with revenue, customer, or product usage data and will be expected to gain an analytical understanding of these datasets. Your responsibilities Data pipeline development: design, implement, and maintain scalable data pipelines and architecture in both our on-prem and cloud environments. Build the next generation of DeepL's data More ❯

are a brand embarking on a very exciting plan and we are seeking new members of the team who want to contribute to this. THE ROLE As a Data Architect (contract) for Project Nova, you will be instrumental in shaping AllSaints' journey to become a truly data-driven, AI-powered organisation. You will lead the design … and implementation of our enterprise data architecture. This role is central to the Data & Integration workstream of Project Nova, building on our existing data warehouse and BI capabilities to accelerate our data transformation and enable the use of greater AI-driven insight and actions. You will ensure our data assets … techniques and tools Strong hands-on experience with cloud data platforms, specifically Google Cloud Platform - BigQuery Experience with leading ETL/ELT tools and data pipeline orchestration (e.g., Dataflow, Apache Airflow, Talend, Informatica) Advanced SQL skills and deep knowledge of various database technologies (relational, columnar, NoSQL) Practical experience in establishing data governance frameworks More ❯

are a brand embarking on a very exciting plan and we are seeking new members of the team who want to contribute to this. THE ROLE As a Data Architect (contract) for Project Nova, you will be instrumental in shaping AllSaints' journey to become a truly data-driven, AI-powered organisation. You will lead the design … and implementation of our enterprise data architecture. This role is central to the Data & Integration workstream of Project Nova, building on our existing data warehouse and BI capabilities to accelerate our data transformation and enable the use of greater AI-driven insight and actions. You will ensure our data assets … techniques and tools Strong hands-on experience with cloud data platforms, specifically Google Cloud Platform - BigQuery Experience with leading ETL/ELT tools and data pipeline orchestration (e.g., Dataflow, Apache Airflow, Talend, Informatica) Advanced SQL skills and deep knowledge of various database technologies (relational, columnar, NoSQL) Practical experience in establishing data governance frameworks More ❯

Radius is seeking a highly talented Data Architect and Governance lead for my client going through a digital transformation. Data Strategy Define and lead enterprise-wide data strategy Design and maintain a scalable, secure data architecture Deep understanding of Data Governance Modern data platforms: Cloud Data Quality Management, metadata, lineage, and data modelling Experience with Data integration, master data management (MDM), and designing scalable data pipelines Data Privacy, regulatory compliance, and security best practices BI Tools: Power BI, Tableau, Looker Proficient in data modelling, SQL, and cloud platforms - Azure, AWS, GCP and … their Data services Key Responsibilities Define and implement data architecture strategies in line with the Customer Journey programme Guide the data-driven decision-making process, ensuring data is effectively utilized to inform business strategies and initiatives Develop a data plan to address data quality issues in line with More ❯

Key Responsibilities: Data Pipelines and … ETL Processes ETL Design: Design and implement ETL processes within MaPS architectural patterns to extract, transform, and load data from various source systems into our reporting solutions. Pipeline Development: Develop and configure meta-data driven data pipelines using data orchestration tools such as Azure Data factory and engineering tools … recovery processes to maintain data integrity. Azure Services Utilise Azure Cloud based data technology platforms, Data Lake based storage & Cloud data pipeline/orchestration tools for efficient data storage, processing, and transfer. Ensure the scalability and reliability of data solutions using Azure's suite of services. Collaboration More ❯

Senior Data Ops Data EngineerLeeds 2/3 days per week£55,000-£67,000+ Excellent Benefits18 Months FTC with long term potential Your New Company Our large public sector organisation are seeking a senior Data Platform DataOps Engineer to serve as our clients first DataOps specialist in a growing team of Data Engineers and DevOps professionals.In this pivotal role, you will focus on operationalising and automating their data lifecycle to ensure that data workflows perform with reliability and efficiency. You will integrate CI/CD data pipelines, streamline deployment processes, enforce robust data governance, and optimise operational costs within our Microsoft Azure … APIM. Proficiency in developing CI/CD data pipelines and strong programming skills in Python, SQL, Bash, and PySpark for automation. Strong aptitude for data pipeline monitoring and an understanding of data security practices such as RBAC and encryption. Implemented data and pipeline observability dashboards, ensuring high data More ❯

condition, or are neurodiverse. If you'd like to apply and need adjustments made, you can let us know in your application. About the role As an Azure Data Engineer, your role involves designing, implementing, and managing data solutions on the Microsoft Azure cloud platform. Your responsibilities will include, but not limited to: Data Integration Data Integration and ETL (Extract, Transform, Load): You'll develop and implement data integration processes to extract data from diverse sources and load it into data warehouses or data lakes. Azure Data Factory is a key tool for building efficient ETL pipelines. Your goal is … to ensure that data is effectively cleaned, transformed, and loaded. Your sources will include Rest API from various systems. Being able to onboard data from API's is essential. Use of Postman for testing outside of ADF is a preferred skill. Data Engineering As an Azure Data Engineer, you'll handle and More ❯

Main purpose of job: Responsible for facilitating the implementation of the data strategy, under the strategic direction of the Data Steering Group, through iterative technical delivery of a modern data platform solution that provides the business with the following: Improved data quality and consistency, More readily and reliably accessible MI, Enhanced data … driven business insights, Leveraging of digitalisation and automated solutions to achieve greater resource and process efficiencies. This will include contributing towards the design and building of a modern Data platform solution to handle large and complex data sets. Key tasks: Design Data Lake and Data Warehouse solutions Design Data Models … using Data Vault and Dimensional modelling methods Implement automated, reusable and efficient batch data pipelines and streaming data pipelines Work closely with Governance and Quality teams to ensure metadata, catalogue, lineage and known solutions to data issues are optimised Work closely with other data specialists and technicians in the organisation More ❯

that adventure. Our employees are at the heart of delivering impactful and meaningful work for our clients and helping them to reach for and realise their vision. Snowflake Data Architect Slalom is seeking an experienced Snowflake Data Architect to contribute to its growing Data Capability. The ideal candidate is a professional with experience in … designing, developing, validating and communicating enterprise data solutions using Snowflake. Deep experience in developing enterprise data management strategies including data lake/warehouse implementations, data movement, data services, data acquisition, data conversion, and archive/recovery. What will you do? Lead as a hands-on architect … and governance concepts such as data quality, metadata management, etc. Experience in designing data engineering solutions using open source and proprietary cloud data pipeline tools Experience in leading data platform development teams Knowledge of data science and visualisation Must have Snowflake SnowPro Core Certification and ideally SnowPro Advanced too More ❯

Data Engineer Manager Hybrid - London with 2/3 days WFM Circ £85,000 - £95,000 + Bonus This newly created Data Engineer Managers position is an excellent opportunity for a hands on Senior Data Engineer/Technical Lead/Engineering Team Lead etc looking to move into a Management position. The postholder will … lead and mentor a team of 3 Data Engineers whilst overseeing data platform development and optimisation. Our client is a well-established and rapidly growing global ecommerce business with its headquarters based in London. They've been expanding at pace and have ambitious plans to take the business even further. The Data Engineer Manager … will play a pivotal role at the heart our clients data & analytics operation. Having implemented a new MS Fabric based Data platform, the need is now to scale up to meet the demand to deliver data driven insights and strategies right across the business globally. The role will require someone that's happy to More ❯

Data Engineer Manager Hybrid - London with 2/3 days WFM Circ £85,000 - £95,000 + Bonus This newly created Data Engineer Managers position is an excellent opportunity for a hands on Senior Data Engineer/Technical Lead/Engineering Team Lead etc looking to move into a Management position. The postholder will … lead and mentor a team of 3 Data Engineers whilst overseeing data platform development and optimisation. Our client is a well-established and rapidly growing global E-commerce business with its headquarters based in London. They've been expanding at pace and have ambitious plans to take the business even further. The Data Engineer … Manager will play a pivotal role at the heart our clients data & analytics operation. Having implemented a new MS Fabric based Data platform, the need is now to scale up to meet the demand to deliver data driven insights and strategies right across the business globally. The role will require someone thats happy to More ❯

and building platforms that handle the complexity of modern cruising - all while keeping things collaborative and fun. Your Mission: Are you passionate about unlocking the power of customer data? We're looking for a talented Lead Data Engineer to spearhead the design, development, and optimisation of our critical CRM and customer data transformation. You … ll play a pivotal role in building the data foundations for advanced analytics, personalised customer experiences, and effective marketing activation. As a senior member of the team, you will lead data engineers and champion best practices within our data environment. What You'll Do: Design & Build: Architect, build, test, and deploy robust, scalable, and … Guide: Provide technical leadership to data engineers, fostering their growth and ensuring high standards in coding, testing, and documentation. Optimise: Continuously monitor and improve data pipeline performance, reliability, and cost-efficiency. Document: Create and maintain clear, comprehensive documentation for data models, pipeline architecture, and processes. What You'll Be Responsible For: The More ❯

We believe in better. And we make it happen. Better content. Better products. And better careers. Working in Tech, Product or Data at Sky is about building the next and the new. From broadband to broadcast, streaming to mobile, SkyQ to Sky Glass, we never stand still. We optimise and innovate. We turn big ideas into the products … content and services millions of people love. And we do it all right here at Sky. The team is looking for a Data Architect to join the IoT platform team, supporting technical initiatives aimed at enhancing data architecture and improving the platform's data handling capabilities. This role will focus on designing scalable data models, ensuring data quality, and streamlining the integration of data sources for real-time analytics and platform performance. The Data Architect will also play a key role in implementing data governance policies to ensure compliance, security, and proper management of IoT data. Additionally, the role will involve aligning IoT data More ❯