51 to 75 of 206 Data Ingestion Jobs in the UK

Energy Domain is a must. Under 7 years of experience only. Permanent. Key Responsibilities Design, develop, and maintain data ingestion pipelines using open-source frameworks and tools Build and optimise ETL/ELT processes to handle small to large-scale data processing requirements Develop data models and schemas that support analytics, business intelligence and product needs … Monitor, troubleshoot, and optimise data pipeline performance and reliability Collaborate with stakeholders, analysts and product team to understand data requirements Implement data quality checks and validation processes to ensure data integrity Participate in architecture decisions and contribute to technical roadmap planning Technical Skills: Great SQL skills with experience in complex query optimization Strong Python programming skills … with experience in data processing libraries (pandas, NumPy, Apache Spark) Hands-on experience building and maintaining data ingestion pipelines Proven track record of optimising queries, code, and system performance Experience with open-source data processing frameworks (Apache Spark, Apache Kafka, Apache Airflow) Knowledge of distributed computing concepts and big data technologies Experience with version control More ❯

Lead Data Engineer Salary/Rate : £100,000 - £120,000 per annum + Bonus Location : North London Company : Retelligence About Retelligence Retelligence is partnering with a high-growth, forward-thinking organization that specializes in digital innovation and marketing across international markets. The company is on an exciting journey, rapidly scaling its capabilities and leveraging advanced technology to deliver cutting … edge solutions. Join a dynamic team within a business that values innovation, supports professional development, and offers exceptional career progression. The Role Retelligence is seeking a Lead Data Engineer to take a hands-on role in designing and delivering robust, real-time data pipelines and infrastructure in a Google Cloud Platform (GCP) environment. The company is particularly interested … in candidates with strong expertise in SQL. As the Lead Data Engineer, you’ll play a critical role in shaping their data architecture and driving transformation. You’ll partner closely with engineering, product, and analytics teams to ensure efficient, high-performance data systems that enable the business to thrive in a fast-paced environment. Key Responsibilities: Design More ❯

Global Head of Data Engineering | Base Salary & uncapped Bonus | London | Permanent Industry: Trading/Commodities Location: London – Hybrid (4 days in office) Job Type: Permanent Our client is seeking a Global Head of Data Engineering to oversee the development and optimisation of their enterprise data platform in a front office trading environment. You’ll lead a distributed … team of data engineers, ensuring the seamless ingestion, management, and delivery of high-quality market, economic, and commercial data to power trading, research, and operational decision-making. What You’ll Do: Lead and manage a global team of data engineers, providing mentorship and direction. Oversee the design, build, and ongoing enhancement of an enterprise data platform on AWS. Develop and scale robust ETL/ELT pipelines for the ingestion of financial market, commodity, and commercial datasets. Deliver new data products and services for traders, analysts, and operational teams. Implement and maintain data quality frameworks and best practices. Collaborate with front office users, including traders and research analysts, to understand data More ❯

Global Head of Data Engineering | Base Salary & uncapped Bonus | London | Permanent Industry: Trading/Commodities Location: London – Hybrid (4 days in office) Job Type: Permanent Our client is seeking a Global Head of Data Engineering to oversee the development and optimisation of their enterprise data platform in a front office trading environment. You’ll lead a distributed … team of data engineers, ensuring the seamless ingestion, management, and delivery of high-quality market, economic, and commercial data to power trading, research, and operational decision-making. What You’ll Do: Lead and manage a global team of data engineers, providing mentorship and direction. Oversee the design, build, and ongoing enhancement of an enterprise data platform on AWS. Develop and scale robust ETL/ELT pipelines for the ingestion of financial market, commodity, and commercial datasets. Deliver new data products and services for traders, analysts, and operational teams. Implement and maintain data quality frameworks and best practices. Collaborate with front office users, including traders and research analysts, to understand data More ❯

Global Head of Data Engineering | Base Salary & uncapped Bonus | London | Permanent Industry: Trading/Commodities Location: London – Hybrid (4 days in office) Job Type: Permanent Our client is seeking a Global Head of Data Engineering to oversee the development and optimisation of their enterprise data platform in a front office trading environment. You’ll lead a distributed … team of data engineers, ensuring the seamless ingestion, management, and delivery of high-quality market, economic, and commercial data to power trading, research, and operational decision-making. What You’ll Do: Lead and manage a global team of data engineers, providing mentorship and direction. Oversee the design, build, and ongoing enhancement of an enterprise data platform on AWS. Develop and scale robust ETL/ELT pipelines for the ingestion of financial market, commodity, and commercial datasets. Deliver new data products and services for traders, analysts, and operational teams. Implement and maintain data quality frameworks and best practices. Collaborate with front office users, including traders and research analysts, to understand data More ❯

Global Head of Data Engineering | Base Salary & uncapped Bonus | London | Permanent Industry: Trading/Commodities Location: London – Hybrid (4 days in office) Job Type: Permanent Our client is seeking a Global Head of Data Engineering to oversee the development and optimisation of their enterprise data platform in a front office trading environment. You’ll lead a distributed … team of data engineers, ensuring the seamless ingestion, management, and delivery of high-quality market, economic, and commercial data to power trading, research, and operational decision-making. What You’ll Do: Lead and manage a global team of data engineers, providing mentorship and direction. Oversee the design, build, and ongoing enhancement of an enterprise data platform on AWS. Develop and scale robust ETL/ELT pipelines for the ingestion of financial market, commodity, and commercial datasets. Deliver new data products and services for traders, analysts, and operational teams. Implement and maintain data quality frameworks and best practices. Collaborate with front office users, including traders and research analysts, to understand data More ❯

Global Head of Data Engineering | Base Salary & uncapped Bonus | London | Permanent Industry: Trading/Commodities Location: London – Hybrid (4 days in office) Job Type: Permanent Our client is seeking a Global Head of Data Engineering to oversee the development and optimisation of their enterprise data platform in a front office trading environment. You’ll lead a distributed … team of data engineers, ensuring the seamless ingestion, management, and delivery of high-quality market, economic, and commercial data to power trading, research, and operational decision-making. What You’ll Do: Lead and manage a global team of data engineers, providing mentorship and direction. Oversee the design, build, and ongoing enhancement of an enterprise data platform on AWS. Develop and scale robust ETL/ELT pipelines for the ingestion of financial market, commodity, and commercial datasets. Deliver new data products and services for traders, analysts, and operational teams. Implement and maintain data quality frameworks and best practices. Collaborate with front office users, including traders and research analysts, to understand data More ❯

Data Engineer £500 - £560 per day London - 1 day per week in office We're working with a leading global healthcare technology company who are building out their next-generation data platform, with a strong emphasis on automation, testing, and cloud-native engineering, and are looking for an experienced Data Engineer to join their team. The Role … You'll be part of a modern data engineering function that's implementing best-in-class data practices across ingestion, transformation, and orchestration layers. The environment is highly technical, collaborative, and fast-paced, giving you the opportunity to work across a wide variety of data sources and tools. Day-to-day responsibilities include: Designing and developing … DBT models and Airflow pipelines within a modern data stack. Building robust data ingestion pipelines across multiple sources - including external partners, internal platforms, and APIs. Implementing automated testing and CI/CD pipelines for data workflows. Performing data extraction and enrichment, including web scraping and parsing of unstructured text (e.g., scanned forms and documents). More ❯

About the Opportunity This is a rare opportunity for a Data Engineer to work at the intersection of cutting-edge technology and elite sports performance. You’ll join a collaborative, high-standard engineering team with a culture built around creativity, innovation, and excellence. In this role, you’ll play a key part in designing and developing scalable data … the football world—including performance insights for some of the most well-known footballers in the industry . This is the perfect environment for those passionate about leveraging advanced data engineering to drive real-world results in sports analytics. Key Responsibilities Architect, design, build, test, and maintain scalable and high-performance data pipelines Ensure all systems adhere to … best practices for data quality, integrity, and security Integrate modern data engineering tools and technologies into cloud-native infrastructure Develop tools and solutions to support sports data modeling, analytics, and predictive insights Collaborate with data scientists, analysts, and performance experts to enable end-to-end data workflows Own the architecture and maintenance of a GCP More ❯

About the Opportunity This is a rare opportunity for a Data Engineer to work at the intersection of cutting-edge technology and elite sports performance. You’ll join a collaborative, high-standard engineering team with a culture built around creativity, innovation, and excellence. In this role, you’ll play a key part in designing and developing scalable data … the football world—including performance insights for some of the most well-known footballers in the industry . This is the perfect environment for those passionate about leveraging advanced data engineering to drive real-world results in sports analytics. Key Responsibilities Architect, design, build, test, and maintain scalable and high-performance data pipelines Ensure all systems adhere to … best practices for data quality, integrity, and security Integrate modern data engineering tools and technologies into cloud-native infrastructure Develop tools and solutions to support sports data modeling, analytics, and predictive insights Collaborate with data scientists, analysts, and performance experts to enable end-to-end data workflows Own the architecture and maintenance of a GCP More ❯

About the Opportunity This is a rare opportunity for a Data Engineer to work at the intersection of cutting-edge technology and elite sports performance. You’ll join a collaborative, high-standard engineering team with a culture built around creativity, innovation, and excellence. In this role, you’ll play a key part in designing and developing scalable data … the football world—including performance insights for some of the most well-known footballers in the industry . This is the perfect environment for those passionate about leveraging advanced data engineering to drive real-world results in sports analytics. Key Responsibilities Architect, design, build, test, and maintain scalable and high-performance data pipelines Ensure all systems adhere to … best practices for data quality, integrity, and security Integrate modern data engineering tools and technologies into cloud-native infrastructure Develop tools and solutions to support sports data modeling, analytics, and predictive insights Collaborate with data scientists, analysts, and performance experts to enable end-to-end data workflows Own the architecture and maintenance of a GCP More ❯

About the Opportunity This is a rare opportunity for a Data Engineer to work at the intersection of cutting-edge technology and elite sports performance. You’ll join a collaborative, high-standard engineering team with a culture built around creativity, innovation, and excellence. In this role, you’ll play a key part in designing and developing scalable data … the football world—including performance insights for some of the most well-known footballers in the industry . This is the perfect environment for those passionate about leveraging advanced data engineering to drive real-world results in sports analytics. Key Responsibilities Architect, design, build, test, and maintain scalable and high-performance data pipelines Ensure all systems adhere to … best practices for data quality, integrity, and security Integrate modern data engineering tools and technologies into cloud-native infrastructure Develop tools and solutions to support sports data modeling, analytics, and predictive insights Collaborate with data scientists, analysts, and performance experts to enable end-to-end data workflows Own the architecture and maintenance of a GCP More ❯

Lead Data Engineer Hybrid (London/Reading - 1-2 days per week) Up to £90,000 + Benefits Are you ready to take the lead in shaping world-class data infrastructure at one of the most influential analytics companies on the planet? We're partnering with a global leader in marketing data and insights that's seeking … a Lead Data Engineer to help scale and evolve its cloud-based data ecosystem. This is an opportunity to work on high-impact global datasets , enabling better analytics, stronger fraud prevention, and smarter decision-making across 90+ markets. Why this role? Play a key role in building and optimising global data pipelines supporting over 50 consumer panels … worldwide. Lead the technical strategy for data architecture, quality, and scalability. Collaborate closely with data scientists, analysts, and software engineers to drive innovation in data engineering. Hybrid flexibility: 1-2 days per week in London, Reading, or Southbank office , balance remotely. Join a global organisation with the resources of a world leader and the agility of a More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector we are always in search of the best talent … benefits, you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and onpremises platforms. … You will play a key role in ensuring that all data systems comply with industry regulations and security standards while enabling efficient access for analytics and operational teams. A strong command of Apache NiFi is essential for this role. You will be expected to design, implement, and maintain data flows using NiFi, ensuring accurate, efficient, and secure data More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best talent … benefits, you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and onpremises platforms. … You will play a key role in ensuring that all data systems comply with industry regulations and security standards while enabling efficient access for analytics and operational teams. A strong command of Apache NiFi is essential for this role. You will be expected to design, implement, and maintain data flows using NiFi, ensuring accurate, efficient, and secure data More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best talent … benefits, you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on-premises … platforms. You will play a key role in ensuring that all data systems comply with industry regulations and security standards while enabling efficient access for analytics and operational teams. A strong command of Apache NiFi is essential for this role. You will be expected to design, implement, and maintain data flows using NiFi, ensuring accurate, efficient, and secure More ❯

Test Engineer Data Platforms (FTC-6months Hybrid | London or Manchester) My client is looking for an experienced Test Engineer to take ownership of testing across modern data platforms. This is a hands-on role where you’ll design, implement, and execute test strategies for data pipelines, models, and reporting solutions built on Azure technologies. Key Responsibilities Define … and own the overall testing strategy for data pipelines, data models, and BI solutions. Develop and implement automated testing frameworks integrated with CI/CD pipelines. Design and execute test cases across data ingestion, transformation, storage, and presentation layers. Validate data accuracy, performance, and usability within Azure Databricks, Azure Data Factory, Azure Data Lake Storage, and Power BI. Establish and monitor data quality checks, reconciliation, and compliance processes. Collaborate closely with data engineers, BI developers, architects, and product owners to embed testing best practices across delivery. Essential Skills & Experience Proven experience as a Test Engineer within data engineering or analytics environments. Strong knowledge of Azure data services – Databricks More ❯

we are hiring for Technical Data Architect location: Central London Type : Permanent Hybrid role (2-3 days from client location) We are seeking a highly skilled TechnicalData Architect- with expertise in Databricks, PySpark, and modern data engineering practices. The ideal candidate will lead the design, development, and optimization of scalable data pipelines, while ensuring data accuracy … platform. This role requires strong leadership, technical depth, and the ability to collaborate with cross-functional teams. ________________________________________ Key Responsibilities Lead the design, development, and maintenance of scalable, high-performance data pipelines on Databricks. Architect and implement data ingestion, transformation, and integration workflows using PySpark, SQL, and Delta Lake. Guide the team in migrating legacy ETL processes to … modern cloud-based data pipelines. Ensure data accuracy, schema consistency, row counts, and KPIs during migration and transformation. Collaborate with Data engineer , BI Engineers, and Security teams to define data standards, governance, and compliance. Optimize Spark jobs and Databricks clusters for performance and cost efficiency. Support real-time and batch data processing for downstream systems More ❯

seeking to recruit a DataOps Enginer on an initial 6 month contract based in London. It is hybrid and will require 2/3x days onsite per week. A Data Ops Engineer is a highly technical individual contributor, building modern, cloud-native, DevOps-first systems for standardizing and templatizing biomedical and scientific data engineering, with demonstrable experience across … the following areas:?? They are a full-stack shop consisting of product and portfolio leadership, data engineering, infrastructure and DevOps, data/metadata/knowledge platforms, and AI/ML and analysis platforms, all geared toward: - Building a next-generation, metadata- and automation-driven data experience for scientists, engineers, and decision-makers, increasing productivity and reducing time … spent on "data mechanics"- Providing best-in-class AI/ML and data analysis environments to accelerate our predictive capabilities and attract top-tier talent- Aggressively engineering our data at scale, as one unified asset, to unlock the value of our unique collection of data and predictions in real-timeAutomation of end-to-end data More ❯

seeking to recruit a DataOps Enginer on an initial 6 month contract based in London. It is hybrid and will require 2/3x days onsite per week. A Data Ops Engineer is a highly technical individual contributor, building modern, cloud-native, DevOps-first systems for standardizing and templatizing biomedical and scientific data engineering, with demonstrable experience across … the following areas: They are a full-stack shop consisting of product and portfolio leadership, data engineering, infrastructure and DevOps, data/metadata/knowledge platforms, and AI/ML and analysis platforms, all geared toward: - Building a next-generation, metadata- and automation-driven data experience for scientists, engineers, and decision-makers, increasing productivity and reducing time … spent on "data mechanics" - Providing best-in-class AI/ML and data analysis environments to accelerate our predictive capabilities and attract top-tier talent - Aggressively engineering our data at scale, as one unified asset, to unlock the value of our unique collection of data and predictions in real-time Automation of end-to-end data More ❯

Part Time Data Engineer - GCP, dbt, Semantic Layer London Day Rate: £400 - £450 Outside IR34 Fractional Opportunity - 5.5 Days per Month A fast-growing, VC-backed tech business in the education and wellbeing space is looking for a part time Data Engineer to support data ingestion, transformation and strategy as they scale. What you'll do … Build and manage API data ingestion into GCP Develop ETL pipelines (dbt, Airflow, DAGs) Model data in BigQuery and integrate with a semantic layer (Qube) Support reporting on behaviour, attendance and benchmarking Contribute to data strategy while delivering hands-on implementation What we're looking for... Experience with GCP is essential for this role. Mid to … Senior Data Engineer with SQL, dbt and Airflow experience. Knowledge of semantic layers (Qube ideal). Exposure to different data strategies. Collaborative, flexible, comfortable with remote delivery Please note: We are currently receiving a high volume of applications and are working through them carefully. To help us manage this process, we kindly ask that you refrain from calling More ❯

Graduate Data Engineer 📍 London (2 days in office, hybrid) | 💼 Full-time | 🎓 0–2 Years’ Experience Are you an aspiring Data Engineer eager to turn raw financial data into clean, structured, and scalable systems? We’re looking for a curious and motivated Graduate Data Engineer to join a hybrid-working team building cutting-edge data infrastructure … for the financial sector. You’ll spend 2 days a week in the office working directly with the Senior Data Engineer , and the rest working remotely — giving you the best of both worlds. You’ll be working on a platform that transforms diverse datasets into actionable insights — learning how to design ETL pipelines, manage cloud-based data systems … and deploy solutions that scale. In this role, you’ll help build, test, and maintain the pipelines and systems that power real-world applications. From data ingestion and transformation to monitoring and optimisation, you’ll gain hands-on experience across the full data engineering lifecycle — while contributing to a product used by clients in the financial industry. More ❯

Graduate Data Engineer 📍 London (2 days in office, hybrid) | 💼 Full-time | 🎓 0–2 Years’ Experience Are you an aspiring Data Engineer eager to turn raw financial data into clean, structured, and scalable systems? We’re looking for a curious and motivated Graduate Data Engineer to join a hybrid-working team building cutting-edge data infrastructure … for the financial sector. You’ll spend 2 days a week in the office working directly with the Senior Data Engineer , and the rest working remotely — giving you the best of both worlds. You’ll be working on a platform that transforms diverse datasets into actionable insights — learning how to design ETL pipelines, manage cloud-based data systems … and deploy solutions that scale. In this role, you’ll help build, test, and maintain the pipelines and systems that power real-world applications. From data ingestion and transformation to monitoring and optimisation, you’ll gain hands-on experience across the full data engineering lifecycle — while contributing to a product used by clients in the financial industry. More ❯

Graduate Data Engineer 📍 London (2 days in office, hybrid) | 💼 Full-time | 🎓 0–2 Years’ Experience Are you an aspiring Data Engineer eager to turn raw financial data into clean, structured, and scalable systems? We’re looking for a curious and motivated Graduate Data Engineer to join a hybrid-working team building cutting-edge data infrastructure … for the financial sector. You’ll spend 2 days a week in the office working directly with the Senior Data Engineer , and the rest working remotely — giving you the best of both worlds. You’ll be working on a platform that transforms diverse datasets into actionable insights — learning how to design ETL pipelines, manage cloud-based data systems … and deploy solutions that scale. In this role, you’ll help build, test, and maintain the pipelines and systems that power real-world applications. From data ingestion and transformation to monitoring and optimisation, you’ll gain hands-on experience across the full data engineering lifecycle — while contributing to a product used by clients in the financial industry. More ❯

Graduate Data Engineer 📍 London (2 days in office, hybrid) | 💼 Full-time | 🎓 0–2 Years’ Experience Are you an aspiring Data Engineer eager to turn raw financial data into clean, structured, and scalable systems? We’re looking for a curious and motivated Graduate Data Engineer to join a hybrid-working team building cutting-edge data infrastructure … for the financial sector. You’ll spend 2 days a week in the office working directly with the Senior Data Engineer , and the rest working remotely — giving you the best of both worlds. You’ll be working on a platform that transforms diverse datasets into actionable insights — learning how to design ETL pipelines, manage cloud-based data systems … and deploy solutions that scale. In this role, you’ll help build, test, and maintain the pipelines and systems that power real-world applications. From data ingestion and transformation to monitoring and optimisation, you’ll gain hands-on experience across the full data engineering lifecycle — while contributing to a product used by clients in the financial industry. More ❯