601 to 625 of 1,433 Data Pipeline Jobs in England

and real work-life balance. That's waiting for you at Sainsbury's. Think about the scale it takes for us to feed the nation. The level of data, transactions and variety it involves. Then you'll realise that ours is a modern software engineering environment because it has to be. We've made serious investment into a … new Artificial Intelligence & Machine Learning platform. What you'll do Own and evolve a newly built ML Ops platform on Microsoft Azure, supporting the full ML lifecycle-from data sourcing & feature engineering to training, deployment, and real-time serving. Architect scalable, secure, and cost-effective solutions, making key build vs. buy decisions to maximise business impact. Drive innovation … capabilities such as graph neural networks and LLM Ops. Oversee the migration & modernisation of existing ML models into a unified strategic stack. Enable multiple cross-functional teams of data scientists and ML engineers, supporting AI-driven automation across the business -from personalisation to logistics optimisation. Own the end-to-end delivery and operations of the platform, balancing feature More ❯

Azure. Collaborate with cross-functional teams to analyze requirements and architect high-quality software solutions. Develop IT solutions using Python and related libraries for AI applications. Build scalable data pipelines utilizing technologies like Pandas, NumPy, or Spark. Apply Agile and DevOps practices to deliver effective solutions. Understand and translate business requirements into technical specifications. Identify opportunities for enhancing … Scrum methodologies and tools (e.g., Jira). Strong knowledge of Salesforce platform features (e.g., Apex, Visualforce) and integration patterns (e.g., REST/SOAP APIs). Proficiency in Salesforce data modeling, database concepts, and security models. Right to work For this role, you will need to provide your own, continuous (i.e. not time limited) right to work in the More ❯

role in architecting and scaling our AI infrastructure. What You’ll Do Lead the design, development, and deployment of scalable Python-based systems for AI applications. Collaborate with data scientists, ML engineers, and product teams to bring AI models into production. Mentor and guide a team of Python developers, fostering a culture of clean code, testing, and continuous … improvement. Own the architecture of backend services, APIs, and data pipelines. Evaluate and integrate new technologies to keep our stack modern and efficient. What We’re Looking For 5+ years of professional Python development experience, with at least 2 years in a leadership role . Strong understanding of software architecture, design patterns, and cloud-native development (preferably AWS … asyncio). Familiarity with containerization (Docker, Kubernetes) and CI/CD pipelines. Excellent communication skills and a passion for mentoring and team leadership. Nice to Have Experience with data engineering tools (Airflow, Kafka, Spark). Knowledge of MLOps practices and tools (MLflow, Kubeflow). Contributions to open-source projects or AI research. Why Join? Work on impactful AI More ❯

Contact email: Job ref: ADE/HH/90 Startdate: ASAP We'e looking for a Senior AWS Data Engineer to join a high-impact team driving a next-generation data platform initiative. This is a greenfield opportunity to shape the architecture and engineering of a cutting-edge data solution from scratch. You'll … be working with modern technologies like Databricks, Snowflake, and the latest in AWS data tooling, helping to solve complex data challenges that have wide-reaching impact across multiple business domains. Key Requirements: Strong experience in AWS data engineering tools (e.g., Glue, Athena, PySpark, Lake Formation) Solid skills in Python and SQL for data processing and analysis Deep understanding of data governance, quality, and security A passion for building scalable, secure, and efficient data pipelines Nice to Have: Broader experience across data platforms or tools Familiarity with analytics, ML, or financial data This is a unique chance to make your mark on a major More ❯

Principal Data Engineer - up to £72,000- Hybrid working (Taunton Based but flexible) The Opportunity Yolk Recruitment Public Sector & Not-for-Profit team are working with a valuable civil service organisation that is relatively unknown. Their specialised focus leads to very unique roles with a large scope for personal learning and development. We are supporting them to recruit … What the Principal Data Engineer will be doing . You will be leading on the design, implementation and maintenance of core data modelling, storage and pipeline solutions; using stakeholder requirements to guide your decisions. Building effective and scalable data pipelines, models and storage - driving the team forward and working closely with stakeholders Generating … Data Engineer will bring to the team You will have a solid coding skills, ideally in Python - with the ability to work in a multicloud environment - demonstrating pipeline tracking, usage monitoring and cost management. Demonstrable skills with Python coding and SQL Awareness of how to work within Azure/AWS/GCP Experience of pipeline building More ❯

About: You will lead a team of data analysts, BI business process experts and BI analysts, who will benefit from your experience in coaching and your knowledge of the data and analytics domain. The Data Fabric team will ensure that your team has access to all relevant information and their skilled data engineers will develop the data pipelines for the projects in your team. Your team can further rely on a central competence and assistance of data science and business intelligence. Do you like new challenges that test your earlier experience in marketing? You will work closely together with the product managers and business development managers … to ensure that the development and analysis roadmap of your team reflects their requirements. Your team will mainly be using SAS and PowerBI, in combination with data pipeline development and analysis on the Azure cloud, using Databricks. The coding language used is Python. An earlier experience with these tools and languages will help you to coach your More ❯

About: You will lead a team of data analysts, BI business process experts and BI analysts, who will benefit from your experience in coaching and your knowledge of the data and analytics domain. The Data Fabric team will ensure that your team has access to all relevant information and their skilled data engineers will develop the data pipelines for the projects in your team. Your team can further rely on a central competence and assistance of data science and business intelligence. Do you like new challenges that test your earlier experience in marketing? You will work closely together with the product managers and business development managers … to ensure that the development and analysis roadmap of your team reflects their requirements. Your team will mainly be using SAS and PowerBI, in combination with data pipeline development and analysis on the Azure cloud, using Databricks. The coding language used is Python. An earlier experience with these tools and languages will help you to coach your More ❯

Data Engineer - Manager x5 There are data teams and then there are data teams! Our customer puts enterprise at a new scale, not only with the size of their client engagements, but also with their own size as one of the big 4 and one of the first ever Microsoft Global Alliance Partners ( all … data technologies will be delivered (GCP/AWS/Big data) although Microsoft Azure is at the forefront of some roles). As a result of insane customer wins, they are now looking to expand the teams by 5 experienced Data Engineering Managers who can be at the level to lead and assist them … As the successful Data Engineering Manager you will be responsible for: Building and maintaining data pipelines Identifying and patching issues and bugs identified in the pipeline/architecture Working as part of a data specialist team to deliver quality service Providing access and identity management to onboard new customers onto the pre-existing More ❯

Role: Data Engineering Manager Salary: £65,000- £70,000 per annum Location: Leeds (One day on site). VIQU have partnered with a leading supply chain organisation who are looking to expand their data teams. The Data Engineering Manager will manage a team of eight to help with an on-going digital transformation. The … the clients Databricks platform (AWS is preferred but open to Azure.GCP experience also), utilising Python and SQL, contribute to CI/CD pipelines, strategy development, cost optimisation and data governance frameworks. Job duties of the Data Engineering Manager: Manage a team of eight engineers, helping to mentor and coach the team. Manage the adoption of automated … experience in a managerial role, coaching similar sized teams. Deep knowledge of the Databricks platform. Hands on Python development experience. SQL optimisation. Experience with large scale data pipeline optimisation. Experience with Streaming and Batch Spark workloads. Strong people management skills. Role: Data Engineering Manager Salary: £65,000- £70,000 per annum Location: Leeds (One day More ❯

Are you passionate about building trust in data and shaping how it’s governed across a global organisation? I’m looking for a Data Quality & Governance Manager to join the newly formed Data Team of a global business that’s building a cutting-edge, AI-ready data platform from the ground up. … is a brand-new role with real influence - perfect for someone who thrives in fast-paced, forward-thinking environments and wants to make a lasting impact on how data is managed, secured, and used across the business. Based in their modern Leeds office, you’ll spend 4 days a week on-site, working closely with the rest of … quality , or data management roles Knowledge of Snowflake , Azure , and modern data platforms Strong understanding of data profiling , data migration , and pipeline integration Familiarity with data classification models , lineage tracking , and metadata frameworks Excellent communication skills and the ability to influence stakeholders at all levels Nice to have: Experience More ❯

Five Sigma Finance is a growing fintech company focused on helping institutional investors unlock scalability and growth in fixed income markets. As we scale, we are building modern data infrastructure to empower smarter decisions across our business. You'll be joining a small but high-impact technology team where your contributions will directly shape our data … office - 1 working from home. After passing probation: 3 days per week from the office, 2 from home Job Purpose You will own and evolve our in-house data platform to support analytics, reporting, and automation for our technology platform and across the business. This role is central to ensuring high data availability, quality, and scalability … problem solver, with an excellent eye for detail, comfortable working in a fast-paced, agile environment. Preferred qualifications include: Highly proficient in Python coding. Experience with data pipeline development and management. Hands-on experience with data processing tools such as Spark or ClickHouse. Proficiency in Pandas or other DataFrame libraries. Familiarity with orchestration tools like More ❯

Data Scientist – Category Management (with ML/AI Focus) About the Role: Quest Search & Selection is working with a leading tech-driven eCommerce platform specializing in consumer goods and rapid delivery. Offering a wide product assortment; including groceries, household essentials, over-the-counter medicine, office supplies, and fresh food. The business leverages data and innovation at … scale to drive customer satisfaction and operational excellence. We are seeking a Data Scientist – Category Management with a strong foundation in machine learning and applied AI to join the Analytics & Insights team. In this role, you'll lead data science initiatives that optimize product availability, buying strategies, and category performance using advanced statistical modeling and machine … Set and tune dynamic targets for availability and expiry rates using continuous learning models. Guide and mentor a Data Analyst to support ongoing analysis, data pipeline development, and dashboarding. Key Requirements: 4–6 years of experience in data science, analytics, or ML engineering , ideally in consumer products, retail tech, or supply chain. Solid More ❯

Data Scientist – Category Management (with ML/AI Focus) About the Role: Quest Search & Selection is working with a leading tech-driven eCommerce platform specializing in consumer goods and rapid delivery. Offering a wide product assortment; including groceries, household essentials, over-the-counter medicine, office supplies, and fresh food. The business leverages data and innovation at … scale to drive customer satisfaction and operational excellence. We are seeking a Data Scientist – Category Management with a strong foundation in machine learning and applied AI to join the Analytics & Insights team. In this role, you'll lead data science initiatives that optimize product availability, buying strategies, and category performance using advanced statistical modeling and machine … Set and tune dynamic targets for availability and expiry rates using continuous learning models. Guide and mentor a Data Analyst to support ongoing analysis, data pipeline development, and dashboarding. Key Requirements: 4–6 years of experience in data science, analytics, or ML engineering , ideally in consumer products, retail tech, or supply chain. Solid More ❯

the atmosphere. With hundreds of active business clients and growing international reach, our impact is expanding daily. The Opportunity We’re looking for an innovative, proactive, Junior Carbon Data Scientist to help us to streamline our data collection and analysis process, transforming small and large datasets into manageable formats for import into our software. You’ll … also be supporting on the design and automation of data pipelines, building predictive models, and piloting new tools. We’ll teach you the environmental context - your superpower is data, code, and a knack for creating reliable, repeatable solutions. We’re looking for you to support: Data driven carbon footprint measurement (Scopes … Key Responsabilities Data Collection and Transformation Ingest and clean CSV datasets (e.g. via Excel Power Query, VBA or comparable ETL tools) Standardise irregular inputs into a repeatable pipeline (“raw → final”) Automate routine data-prep steps where possible Write prompts for AI tools to organise and standardise data Support with developing new tools and More ❯

the atmosphere. With hundreds of active business clients and growing international reach, our impact is expanding daily. The Opportunity We’re looking for an innovative, proactive, Junior Carbon Data Scientist to help us to streamline our data collection and analysis process, transforming small and large datasets into manageable formats for import into our software. You’ll … also be supporting on the design and automation of data pipelines, building predictive models, and piloting new tools. We’ll teach you the environmental context - your superpower is data, code, and a knack for creating reliable, repeatable solutions. We’re looking for you to support: Data driven carbon footprint measurement (Scopes … Key Responsabilities Data Collection and Transformation Ingest and clean CSV datasets (e.g. via Excel Power Query, VBA or comparable ETL tools) Standardise irregular inputs into a repeatable pipeline (“raw → final”) Automate routine data-prep steps where possible Write prompts for AI tools to organise and standardise data Support with developing new tools and More ❯

This fantastic institution located in SW London is seeking a dynamic and versatile Data Engineer/Analytics Engineer to contribute to the development, enhancement, and management of their data systems, pipelines, and reporting infrastructure. This role will focus on designing and maintaining efficient data pipelines while building a scalable data warehouse architecture. … A major aspect of this position will involve integrating new data sources to create a unified Single Customer View. Additionally, collaboration with the Analytics and Insights teams is essential to ensure they have seamless access to the necessary data, as well as assisting in optimizing SQL reporting processes. The ideal candidate will: Possess a strong curiosity … and passion for exploring, transforming, and integrating data from diverse sources. Be adept at troubleshooting and fine-tuning data processes to enhance data quality and stability. Thrive in team settings, with excellent communication skills and the confidence to engage with various stakeholders. Key Responsibilities: Support the development of the data warehouse and More ❯

Senior Data Engineer We are looking for a technical expert with deep data engineering experience, who thrives on solving complex problems, driving business impact, and coaching other engineers. Salary : £70k-£90k Reporting to : François-Xavier Decroix (Engineering Manager) Location : primarily based in our London Office, with flexible and hybrid working (ideally 2 days per week in … the office). About the role As a Senior Data Engineer, you will take responsibility for continuously improving critical systems which transport hundreds of gigabytes of data per day and trillions of messages per year. You will work closely with the broader team to bring innovative new datasets and insights to customers in a reliable, scalable … our mission to empower safe and sustainable transport around the world through data insights. You will be the go-to person for our cloud data pipeline & underlying infrastructure, especially Apache Kafka and our PostgreSQL database . You will also work with tech like Kubernetes, Vault, go microservices and other cloud services, as well as on More ❯

Senior Data Engineer - Hedge Fund - London Company Overview A leading London-based systematic hedge fund managing over $8 billion in client assets. Technology is core to its operations, with a strong focus on data-driven strategies. The team is currently overhauling its data infrastructure to better support quant research and trading. Role Overview Join … a lean, high-impact Data Engineering team responsible for the ingestion, transformation, and delivery of diverse datasets—from tick and time series to reference and alternative data. You’ll build modern, scalable data pipelines and platforms, support the strategic tech stack, and collaborate closely with quants and portfolio managers. This is a hands-on role with … full ownership across the development lifecycle. Qualifications 5+ years in data engineering Strong Python and data wrangling libraries Expert SQL skills and solid RDBMS understanding Experience with cloud-based data solutions (e.g. serverless ETL, data warehouses) Familiarity with Git, Docker, CI/CD pipelines, testing and monitoring Clear communicator, comfortable with cross More ❯

Senior Data Engineer - Hedge Fund - London Company Overview A leading London-based systematic hedge fund managing over $8 billion in client assets. Technology is core to its operations, with a strong focus on data-driven strategies. The team is currently overhauling its data infrastructure to better support quant research and trading. Role Overview Join … a lean, high-impact Data Engineering team responsible for the ingestion, transformation, and delivery of diverse datasets—from tick and time series to reference and alternative data. You’ll build modern, scalable data pipelines and platforms, support the strategic tech stack, and collaborate closely with quants and portfolio managers. This is a hands-on role with … full ownership across the development lifecycle. Qualifications 5+ years in data engineering Strong Python and data wrangling libraries Expert SQL skills and solid RDBMS understanding Experience with cloud-based data solutions (e.g. serverless ETL, data warehouses) Familiarity with Git, Docker, CI/CD pipelines, testing and monitoring Clear communicator, comfortable with cross More ❯

Senior Data Engineer - Hedge Fund - London Company Overview A leading London-based systematic hedge fund managing over $8 billion in client assets. Technology is core to its operations, with a strong focus on data-driven strategies. The team is currently overhauling its data infrastructure to better support quant research and trading. Role Overview Join … a lean, high-impact Data Engineering team responsible for the ingestion, transformation, and delivery of diverse datasets—from tick and time series to reference and alternative data. You’ll build modern, scalable data pipelines and platforms, support the strategic tech stack, and collaborate closely with quants and portfolio managers. This is a hands-on role with … full ownership across the development lifecycle. Qualifications 5+ years in data engineering Strong Python and data wrangling libraries Expert SQL skills and solid RDBMS understanding Experience with cloud-based data solutions (e.g. serverless ETL, data warehouses) Familiarity with Git, Docker, CI/CD pipelines, testing and monitoring Clear communicator, comfortable with cross More ❯

Are you a passionate Data Engineering Manager ready to unlock the power of data? Our client is a leading UK-based consultancy seeking a skilled professional to shape data strategies, mentor dynamic teams, and deliver cutting-edge solutions. With hands-on expertise in Spark, SQL, and cloud platforms like Azure, you’ll lead end … to-end projects, drive innovation, and collaborate with clients across industries. What You’ll Do: Lead complex data engineering solutions with technical excellence. Mentor teams and foster a culture of growth and quality. Showcase your expertise in pre-sales and client workshops. Design data pipelines, lakes, and warehouses with best practices. Skills They're Looking For … Proven Data Engineering experience (5+ years). Consultancy experience is a must. Leadership and multi-project environments experience. Expertise in ETL, data modelling, and Azure Data Services. Experience in designing and implementing data pipelines, data lakes, and data warehouses. Hands-on experience with Apache Spark and bonus points More ❯

A prestigious organization based in London is seeking a dynamic and versatile Data Engineer/Analytics Engineer to contribute to the development, enhancement, and management of their data systems, pipelines, and reporting infrastructure. This role will focus on designing and maintaining efficient data pipelines while building a scalable data warehouse architecture. A … major aspect of this position will involve integrating new data sources to create a unified Single Customer View. Additionally, collaboration with the Analytics and Insights teams is essential to ensure they have seamless access to the necessary data, as well as assisting in optimizing SQL reporting processes. The ideal candidate will: Possess a strong curiosity and … passion for exploring, transforming, and integrating data from diverse sources. Be adept at troubleshooting and fine-tuning data processes to enhance data quality and stability. Thrive in team settings, with excellent communication skills and the confidence to engage with various stakeholders. Key Responsibilities: Support the development of the data warehouse and enhance More ❯

Data Quality & Governance Manager - Leeds - Hybrid - £70k - £75k My client are greenfielding an entire data practice! The chosen candidate for this role will lead efforts to ensure data across the organization is consistent, reliable, and governed effectively. You’ll develop and implement frameworks for data quality, governance, and master data management. You will ensure the effective management of data warehouse architecture, pipelines, and support data migrations using a combination of Snowflake and Microsoft Azure technologies. Salary and Benefits Competitive salary of £70k - £75k (DOE) Hybrid working (4 days in office, 1 from home) 25 days annual leave plus bank holidays (rising to 27 with … service) Discretionary bonus of up to 10% Private medical care And many more Role and Responsibilities Lead Data Governance Strategy : Develop and execute the overall data governance and quality strategy, aligned with business needs and global standards. Data Warehouse Architecture & Pipelines: Work closely with data engineering teams to ensure data More ❯

Role: Principal Data Engineer Type: Permanent Location: Hampshire, London (Hybrid Working) Salary: £80,000 - £90,000 + Benefits Are you an experienced Data Engineer ready to take the next step into a strategic, hands-on leadership role? This is a fantastic opportunity to join a well-established organisation with a long-standing heritage and a strong … They are a household name in general insurance and part of a larger European group, with a UK presence that continues to grow. With significant investment in their data and technology function, theyre now looking to bring on a Principal Data Engineer to help shape and evolve their data infrastructure and engineering capability. The … Opportunity: This is a pivotal role where youll lead on the design, delivery, and optimisation of scalable, secure data systems that support the wider business strategy. Youll be part of a collaborative tech environment, working closely with cross-functional teams and mentoring fellow engineers. Its a great blend of hands-on technical work and strategic influence. What Youll More ❯

We’re working with a leading insurance organisation undergoing a large-scale data transformation, and they’re now seeking a Senior Data Engineer to play a key role in building and improving their data architecture and governance capabilities as part of a major programme of work. Key Responsibilities: Design and implement robust data pipelines and integration frameworks. Support the implementation and enhancement of Master Data Management (MDM) solutions. Work with business and technical stakeholders to ensure data quality, consistency, and governance across systems. Collaborate closely with data architects, analysts, and engineering teams to support strategic data initiatives. Key Skills & Experience: Strong experience as … a Data Engineer in enterprise environments. Proven expertise with MDM tools (e.g. Informatica MDM, Reltio, Semarchy, IBM InfoSphere, etc.). Solid background in data integration, ETL, and data warehousing concepts. Proficiency in SQL and modern data engineering tools/platforms. Experience working in the insurance or financial services sector is highly desirable. More ❯