351 to 375 of 717 Data Pipeline Jobs in the UK

purpose-driven consultancy that sits at the intersection of advanced analytics, software engineering, and AI. They're on a mission to deliver real-world impact, through high-value, data-driven platforms and tools, across financial services, government, retail, SaaS, and beyond. To support this mission, we’re looking for two exceptional Engineering Managers to lead their Software Engineering … and Analytics Engineering functions respectively. These roles offer the chance to architect scalable products, lead talented technical teams, and shape the future of data and digital platforms in both the public and private sectors. Key Responsibilities: Depending on your background (Software or Data/Analytics), your focus will include: Software Engineering Leaders: Lead the design and … functional teams through agile delivery, technical planning, and product development. Translate business requirements into scalable technical solutions that support critical client outcomes. Analytics Engineering Leaders: Architect and optimise data pipelines, ELT workflows, and cloud warehouse platforms (Snowflake, BigQuery, Redshift). Lead teams working with dbt, SQL, Python, and Airflow to drive data transformation at scale. Ensure More ❯

into the clear and concise technical development specifications Managing and updating the project management board for responsible projects aligned with the I&D Analytics project roadmap Working with data engineers and business owners to create data pipelines for reporting and analytics with consideration of the planned system integration, transformation and data migration Delivering Power … BI data models and reports with the Power BI developers' best practise and compliance with the SNG data policies and procedures Supporting the maintenance, management and enhancement of I&D systems and ensuring they are fit for purpose and continuously improved, and supporting the routine review of the quality and completeness of data What … a similar role as an Insight Analyst, ideally in the Real Estate, or broader Investment and Development sectors. You will need strong Power BI reporting experience with advanced data analytics and visualisation skills. You'll also have: Strong financial acumen and commercial knowledge to interpret complex financial and investment analysis Excellent stakeholder management skills with clear communication and More ❯

into the clear and concise technical development specifications Managing and updating the project management board for responsible projects aligned with the I&D Analytics project roadmap Working with data engineers and business owners to create data pipelines for reporting and analytics with consideration of the planned system integration, transformation and data migration Delivering Power … BI data models and reports with the Power BI developers' best practise and compliance with the SNG data policies and procedures Supporting the maintenance, management and enhancement of I&D systems and ensuring they are fit for purpose and continuously improved, and supporting the routine review of the quality and completeness of data What … a similar role as an Insight Analyst, ideally in the Real Estate, or broader Investment and Development sectors. You will need strong Power BI reporting experience with advanced data analytics and visualisation skills. You'll also have: Strong financial acumen and commercial knowledge to interpret complex financial and investment analysis Excellent stakeholder management skills with clear communication and More ❯

Skills/requirements Deploy comprehensive cloud infrastructure for various products, including Astronomer Airflow and AccelData environments. Facilitate cross-functional integration between vendor products and other systems, such as data lakes, storage, and compute services. Establish best practices for cloud security, scalability, and performance. Manage and configure vendor product deployments, ensuring the setup and maintenance of environments. Ensure high … planning, resource allocation, and optimization of Airflow workers. Maintain and configure integrations with source control systems (e.g., GitHub, GitLab) for version control. Collaborate with cloud providers (e.g., AWS) for pipeline integration and scaling requirements. Identify, design, and implement internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes. Develop infrastructure … using AWS and SQL technologies. Work with stakeholders, including design, product, and executive teams, to address platform-related technical issues. Build analytical tools to leverage the data pipeline, providing actionable insights into key business performance metrics, such as operational efficiency and customer acquisition.? All profiles will be reviewed against the required skills and experience. Due to the More ❯

Skills/requirements Deploy comprehensive cloud infrastructure for various products, including Astronomer Airflow and AccelData environments. Facilitate cross-functional integration between vendor products and other systems, such as data lakes, storage, and compute services. Establish best practices for cloud security, scalability, and performance. Manage and configure vendor product deployments, ensuring the setup and maintenance of environments. Ensure high … planning, resource allocation, and optimization of Airflow workers. Maintain and configure integrations with source control systems (e.g., GitHub, GitLab) for version control. Collaborate with cloud providers (e.g., AWS) for pipeline integration and scaling requirements. Identify, design, and implement internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes. Develop infrastructure … using AWS and SQL technologies. Work with stakeholders, including design, product, and executive teams, to address platform-related technical issues. Build analytical tools to leverage the data pipeline, providing actionable insights into key business performance metrics, such as operational efficiency and customer acquisition.? All profiles will be reviewed against the required skills and experience. Due to the More ❯

Data Engineer Must be London-based, able to work from site in London with occasional travel to Croydon SC Clearance required/or eligible for clearance Contract duration: 2 years + Up To 41,000 per annum, depending on experience Job Purpose/The Role: We are seeking 3 experienced Data Engineers to support a high … Public Sector client. These roles will be part of a large project, contributing to the ongoing development and live support of critical systems within a secure, agile, and data-driven environment. Your Key Responsibilities: Work collaboratively in a multi-disciplinary Agile & DevOps team to deliver data solutions and maintain live systems. Apply best practices for data integration, transformation, and delivery in a secure and scalable environment. Support data pipelines and ETL processes across various technologies and cloud platforms Maintain, monitor, and troubleshoot systems using a wide array of tools and platforms. Contribute to automation and continuous integration/deployment (CI/CD) efforts. Engage directly with clients and stakeholders, demonstrating excellent problem More ❯

Data Engineer Must be London-based, able to work from site in London with occasional travel to Croydon SC Clearance required/or eligible for clearance Contract duration: 2 years + Up To £41,000 per annum, depending on experience Job Purpose/The Role: We are seeking 3 experienced Data Engineers to support a high … Public Sector client. These roles will be part of a large project, contributing to the ongoing development and live support of critical systems within a secure, agile, and data-driven environment. Your Key Responsibilities: Work collaboratively in a multi-disciplinary Agile & DevOps team to deliver data solutions and maintain live systems. Apply best practices for data integration, transformation, and delivery in a secure and scalable environment. Support data pipelines and ETL processes across various technologies and cloud platforms Maintain, monitor, and troubleshoot systems using a wide array of tools and platforms. Contribute to automation and continuous integration/deployment (CI/CD) efforts. Engage directly with clients and stakeholders, demonstrating excellent problem More ❯

Data Scientist (PropTech) London - Hybrid (3 days in office) Up to £70,000 About the Role An established data-led business is looking for a commercially-minded Data Scientist to join their growing London team. You’ll collaborate with cross-functional teams to design, build, and implement data science solutions that directly … into core product offerings. This is an exciting opportunity to apply ML and statistical techniques to real-world problems, while also developing in areas such as software engineering, data pipelines, and ethical product delivery. Key Responsibilities Collaborate with product teams to identify opportunities for ML and data science integration Build and maintain data pipelines … learning models in production environments Present insights and results to technical and non-technical stakeholders Apply statistical methods and model validation techniques to reach accurate conclusions Work with data engineers to ensure clean, usable datasets Contribute to ethical, privacy-aware, and user-centric data products Stay curious, research new methods, and continually build your technical capability More ❯

Data Scientist (PropTech) London - Hybrid (3 days in office) Up to £70,000 About the Role An established data-led business is looking for a commercially-minded Data Scientist to join their growing London team. You’ll collaborate with cross-functional teams to design, build, and implement data science solutions that directly … into core product offerings. This is an exciting opportunity to apply ML and statistical techniques to real-world problems, while also developing in areas such as software engineering, data pipelines, and ethical product delivery. Key Responsibilities Collaborate with product teams to identify opportunities for ML and data science integration Build and maintain data pipelines … learning models in production environments Present insights and results to technical and non-technical stakeholders Apply statistical methods and model validation techniques to reach accurate conclusions Work with data engineers to ensure clean, usable datasets Contribute to ethical, privacy-aware, and user-centric data products Stay curious, research new methods, and continually build your technical capability More ❯

Data Scientist (PropTech) London - Hybrid (3 days in office) Up to £70,000 About the Role An established data-led business is looking for a commercially-minded Data Scientist to join their growing London team. You’ll collaborate with cross-functional teams to design, build, and implement data science solutions that directly … into core product offerings. This is an exciting opportunity to apply ML and statistical techniques to real-world problems, while also developing in areas such as software engineering, data pipelines, and ethical product delivery. Key Responsibilities Collaborate with product teams to identify opportunities for ML and data science integration Build and maintain data pipelines … learning models in production environments Present insights and results to technical and non-technical stakeholders Apply statistical methods and model validation techniques to reach accurate conclusions Work with data engineers to ensure clean, usable datasets Contribute to ethical, privacy-aware, and user-centric data products Stay curious, research new methods, and continually build your technical capability More ❯

Data Scientist (PropTech) London - Hybrid (3 days in office) Up to £70,000 About the Role An established data-led business is looking for a commercially-minded Data Scientist to join their growing London team. You’ll collaborate with cross-functional teams to design, build, and implement data science solutions that directly … into core product offerings. This is an exciting opportunity to apply ML and statistical techniques to real-world problems, while also developing in areas such as software engineering, data pipelines, and ethical product delivery. Key Responsibilities Collaborate with product teams to identify opportunities for ML and data science integration Build and maintain data pipelines … learning models in production environments Present insights and results to technical and non-technical stakeholders Apply statistical methods and model validation techniques to reach accurate conclusions Work with data engineers to ensure clean, usable datasets Contribute to ethical, privacy-aware, and user-centric data products Stay curious, research new methods, and continually build your technical capability More ❯

Data Scientist (PropTech) London - Hybrid (3 days in office) Up to £70,000 About the Role An established data-led business is looking for a commercially-minded Data Scientist to join their growing London team. You’ll collaborate with cross-functional teams to design, build, and implement data science solutions that directly … into core product offerings. This is an exciting opportunity to apply ML and statistical techniques to real-world problems, while also developing in areas such as software engineering, data pipelines, and ethical product delivery. Key Responsibilities Collaborate with product teams to identify opportunities for ML and data science integration Build and maintain data pipelines … learning models in production environments Present insights and results to technical and non-technical stakeholders Apply statistical methods and model validation techniques to reach accurate conclusions Work with data engineers to ensure clean, usable datasets Contribute to ethical, privacy-aware, and user-centric data products Stay curious, research new methods, and continually build your technical capability More ❯

Data Engineer Watford | £85,000 + Excellent Benefits | Office-Based The Opportunity An excellent opportunity for a talented Data Engineer to join a well-established and growing financial services organisation based in Watford . Offering a salary of £85,000 plus a superb benefits package , this role is ideal for someone seeking a technically hands-on … position within a forward-thinking and data-driven environment. The company has been operating successfully for over 20 years, employs more than 300 staff, and is recognised as a market leader in specialist financial services. You'll be joining a collaborative data team where innovation, accuracy, and analytical thinking are highly valued. Duties & Responsibilities Design, build … and maintain robust data pipelines and reporting solutions using modern cloud technologies. Work closely with senior leadership to shape data and reporting strategies. Develop and enhance MI & BI reports, ensuring accuracy and integrity across all data outputs. Collaborate with Product Owners, Developers, and other technical teams to support business goals. Mentor junior members of More ❯

Senior Data Engineer (SQL BigQuery GCP) London/WFH to £110,000 Are you a data technologist with Media Streaming experience? You could be progressing your career in a senor, hands-on role at one of Europe's most successful streaming and social media platforms. As a Senior Data Engineer you'll design, build … and maintain high-performance services for content modelling, serving and integration, develop data pipelines (both batch and streaming) using cloud native tools and collaborate on rearchitecting the content model to support rich metadata. You'll also build and implement APIs and data services to expose the data to other teams across the business and … are seriously cool and offer a sociable environment with a range of perks and experiences. Please note you must be commutable to London. About you: You have strong Data Engineering experience including design and data modelling skills You can code with Golang, Python or Scala You have advanced SQL and NoSQL database knowledge You're familiar More ❯

Senior Data Engineer (SQL BigQuery GCP) London/WFH to £110,000 Are you a data technologist with Media Streaming experience? You could be progressing your career in a senor, hands-on role at one of Europe's most successful streaming and social media platforms. As a Senior Data Engineer you'll design, build … and maintain high-performance services for content modelling, serving and integration, develop data pipelines (both batch and streaming) using cloud native tools and collaborate on rearchitecting the content model to support rich metadata. You'll also build and implement APIs and data services to expose the data to other teams across the business and … are seriously cool and offer a sociable environment with a range of perks and experiences. Please note you must be commutable to London. About you: You have strong Data Engineering experience including design and data modelling skills You can code with Golang, Python or Scala You have advanced SQL and NoSQL database knowledge You're familiar More ❯

/ML model development and deployment Strong skills with Python, TensorFlow/PyTorch, and cloud platforms (AWS/Azure/GCP) MLOps assurance and risk experinece Knowledge of data pipelines, model optimisation, and real-world deployments Details: Contract: 6 months (with view to extend) Location: London (Hybrid) Start: ASAP If you’re interested, please apply or reach out More ❯

/ML model development and deployment Strong skills with Python, TensorFlow/PyTorch, and cloud platforms (AWS/Azure/GCP) MLOps assurance and risk experinece Knowledge of data pipelines, model optimisation, and real-world deployments Details: Contract: 6 months (with view to extend) Location: London (Hybrid) Start: ASAP If you’re interested, please apply or reach out More ❯

Data Engineer We’re looking for a Data Engineer with strong Python skills and experience in event-driven systems to join our growing data team. This isn’t your typical “pipelines-for-analysts” role -you’ll be building real-time systems that power applications, tooling, and commercial products across the business. What you’ll … be doing: -Build and maintain event-driven data pipelines that power Citywire’s Catalyst platform. -Design resilient, fault-tolerant workflows using AWS services such as Lambda, Kinesis, SQS, DynamoDB Streams, and EventBridge. -Implement processors that ensure data consistency across DynamoDB, PostgreSQL (Aurora), OpenSearch, and BigQuery. -Modernise legacy batch processes into stream-first architectures. -Build and integrate … and existing projects, balancing speed with resilience. -Take ownership of key pipelines and services, ensuring reliability, performance, and scalability. -Share best practices and mentor others in event-driven data engineering. What we’re looking for: -Technical Skills: Proven experience in data engineering or backend development, with solid Python skills and hands-on use of AWS event More ❯

Data Engineer We’re looking for a Data Engineer with strong Python skills and experience in event-driven systems to join our growing data team. This isn’t your typical “pipelines-for-analysts” role -you’ll be building real-time systems that power applications, tooling, and commercial products across the business. What you’ll … be doing: -Build and maintain event-driven data pipelines that power Citywire’s Catalyst platform. -Design resilient, fault-tolerant workflows using AWS services such as Lambda, Kinesis, SQS, DynamoDB Streams, and EventBridge. -Implement processors that ensure data consistency across DynamoDB, PostgreSQL (Aurora), OpenSearch, and BigQuery. -Modernise legacy batch processes into stream-first architectures. -Build and integrate … and existing projects, balancing speed with resilience. -Take ownership of key pipelines and services, ensuring reliability, performance, and scalability. -Share best practices and mentor others in event-driven data engineering. What we’re looking for: -Technical Skills: Proven experience in data engineering or backend development, with solid Python skills and hands-on use of AWS event More ❯

Data Engineer We’re looking for a Data Engineer with strong Python skills and experience in event-driven systems to join our growing data team. This isn’t your typical “pipelines-for-analysts” role -you’ll be building real-time systems that power applications, tooling, and commercial products across the business. What you’ll … be doing: -Build and maintain event-driven data pipelines that power Citywire’s Catalyst platform. -Design resilient, fault-tolerant workflows using AWS services such as Lambda, Kinesis, SQS, DynamoDB Streams, and EventBridge. -Implement processors that ensure data consistency across DynamoDB, PostgreSQL (Aurora), OpenSearch, and BigQuery. -Modernise legacy batch processes into stream-first architectures. -Build and integrate … and existing projects, balancing speed with resilience. -Take ownership of key pipelines and services, ensuring reliability, performance, and scalability. -Share best practices and mentor others in event-driven data engineering. What we’re looking for: -Technical Skills: Proven experience in data engineering or backend development, with solid Python skills and hands-on use of AWS event More ❯

Data Engineer We’re looking for a Data Engineer with strong Python skills and experience in event-driven systems to join our growing data team. This isn’t your typical “pipelines-for-analysts” role -you’ll be building real-time systems that power applications, tooling, and commercial products across the business. What you’ll … be doing: -Build and maintain event-driven data pipelines that power Citywire’s Catalyst platform. -Design resilient, fault-tolerant workflows using AWS services such as Lambda, Kinesis, SQS, DynamoDB Streams, and EventBridge. -Implement processors that ensure data consistency across DynamoDB, PostgreSQL (Aurora), OpenSearch, and BigQuery. -Modernise legacy batch processes into stream-first architectures. -Build and integrate … and existing projects, balancing speed with resilience. -Take ownership of key pipelines and services, ensuring reliability, performance, and scalability. -Share best practices and mentor others in event-driven data engineering. What we’re looking for: -Technical Skills: Proven experience in data engineering or backend development, with solid Python skills and hands-on use of AWS event More ❯

Data Engineer We’re looking for a Data Engineer with strong Python skills and experience in event-driven systems to join our growing data team. This isn’t your typical “pipelines-for-analysts” role -you’ll be building real-time systems that power applications, tooling, and commercial products across the business. What you’ll … be doing: -Build and maintain event-driven data pipelines that power Citywire’s Catalyst platform. -Design resilient, fault-tolerant workflows using AWS services such as Lambda, Kinesis, SQS, DynamoDB Streams, and EventBridge. -Implement processors that ensure data consistency across DynamoDB, PostgreSQL (Aurora), OpenSearch, and BigQuery. -Modernise legacy batch processes into stream-first architectures. -Build and integrate … and existing projects, balancing speed with resilience. -Take ownership of key pipelines and services, ensuring reliability, performance, and scalability. -Share best practices and mentor others in event-driven data engineering. What we’re looking for: -Technical Skills: Proven experience in data engineering or backend development, with solid Python skills and hands-on use of AWS event More ❯

solving, and hands-on contribution. The role involves Re-architecting an AWS environment that has grown inconsistently, addressing security, governance, and scalability. Designing and implementing distributed systems and data pipelines, with a focus on reliability and resilience. Working directly on code and infrastructure as code (around 50% of your time). Running proofs of concept to evaluate approaches … as Node.js, C#, or Java (currently use serverless architecture using Node.js and React). Understand SQL at a solid level, with broader experience across NoSQL and data pipeline design. Have experience with distributed systems or progressive web applications. Think carefully about problems, explore alternatives, and can explain the reasoning behind their choices. This role may suit someone … currently working as Lead Developer/Senior Software Engineer Lead DevOps Engineer Solutions Architect with hands-on coding experience Senior Data Engineer with strong AWS expertise Technical Lead in a small-to-medium software team It would particularly fit someone who enjoys combining practical engineering with higher-level architectural thinking, and who wants the space to solve technical More ❯

Data Engineer (Databricks) Location: London (Hybrid - 2 days onsite and 3 days from home ) Type: Permanent Note: No sponsorship available Represented by: Robert Half Limited Our client is a fast-growing tech business building a modern data platform. They're looking for a Data Engineer with strong Databricks skills to design, build, and optimise … scalable data pipelines and models that power analytics across the company. Key Skills: 3-5+ years as a Data Engineer Hands-on with Databricks, Spark, Python, SQL Cloud experience (Azure, AWS, or GCP) Strong understanding of data quality, governance, and security Nice to Have: Delta Lake, DBT, Snowflake, Terraform, CI/CD, or … You'll: Build and optimise ETL pipelines Enable analytics and reporting teams Drive automation and best practices Why Apply? Modern tech stack, fast-paced environment Direct impact on data strategy Quick 2-stage process + excellent benefits Robert Half Ltd acts as an employment business for temporary positions and an employment agency for permanent positions. Robert Half is More ❯

team on an initial 6-month contract. Key skills/knowledge required: Strong proficiency in Python-based testing frameworks (e.g., Behave, PyTest, unittest) Experience in automated testing of data pipelines and ETL processes Experience with data validation frameworks and testing data transformations across Snowflake and AWS environments Knowledge of API testing tools and frameworks … Postman, REST Assured, or similar) Hands-on experience testing OMS platform integrations (Aladdin, Murex, Charles River) and validating trade lifecycle data flows Experience integrating automated tests into CI/CD pipelines for continuous delivery Hands-on experience validating data accuracy and integrity across multiple systems Experience working embedded within cross-functional agile teams rather than separate … QA departments Strong problem-solving skills with ability to design effective test strategies for complex data workflows Ability to work collaboratively with engineers, delivery roles, and business stakeholders Knowledge of testing methodologies for data pipelines, APIs, and integrations Knowledge of data quality testing patterns and anomaly detection in capital markets data Understanding More ❯