1 to 25 of 187 Data Pipeline Jobs in the South East

Engineer to join our Service Operations team. This role offers an exciting opportunity to learn, contribute to, and grow within the critical areas of data pipeline development, maintenance, and optimisation for our client-facing products and services. Working closely with experienced developers, data analysts, data scientists, and other stakeholders, you will … be instrumental in ensuring the efficient and reliable flow of data, building foundational skills in a supportive environment. Key Responsibilities Data Pipeline Development • Contribute to the design, building, and maintenance of scalable data pipelines • Implement ETL (Extract, Transform, Load) processes to ingest data from various sources • Ensure data quality … for data access and manipulation Documentation and Reporting: • Document data processes, pipelines, and architectures • Generate reports and visualisations to communicate data insights and pipeline performance Technical Support: • Provide technical support for data-related issues and troubleshoot problems as they arise • Assist in the implementation of data governance and compliance More ❯

global open economy that works for everyone. We exist behind the scenes, securely connecting companies - from growth to enterprise - to thousands of banks worldwide, enabling them to access data and initiate payments through the power of open banking. What we're looking for As a Java Software Engineer specializing in Data Products at Yapily, you will … Database Management: Background in managing both SQL and NoSQL databases. Version Control & CI/CD: Knowledge of version control (Git) and CI/CD practices for data pipeline deployment and Exposure to tools such as Terraform. Data Modelling & Schema Design: Familiarity with data modelling and schema design for operational or analytical systems. API … compliance principles (GDPR, ISO27001). SaaS/API-driven Environments: Background in SaaS/API-driven environments, ideally with experience in billing or usage-based data. Data Pipeline Monitoring & Troubleshooting: Basic skills in monitoring and troubleshooting data pipelines. Data Security: Understanding of data security best practices and encryption for sensitive data. More ❯

Position: Analytics Engineer Location: London (hybrid 3 times a week)Department: ITType: 3 months rolling contractOutside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. … Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and More ❯

DATA ARCHITECT - DV CLEARED NEW PERMANENT JOB OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A DATA ARCHITECT WITH ENHANCED DV CLEARANCE Permanent job opportunity for a Data Architect Leading National Security SME Salary up to £100,000 + Bonus London based organisation in an easily accessible location To apply please call or … email WHO WE ARE? We are recruiting multiple a Data Architect to support urgent National Security & Defence projects in London. Due to the nature of these projects you must hold enhanced DV Security Clearance. WHAT WILL THE DATA ARCHITECT BE DOING? You will be joining a leading SME who is working hard to support National Security … projects within UK Govt. Departments in London. As part of a team, you will be responsible for designing and implementing Data Solutions in Mission-Critical areas. WE NEED THE DATA ARCHITECT TO HAVE.... Current DV clearance - Enhanced Good at understanding complexity and abstracting that into a form that is consumable for a non-technical audience. Experience More ❯

If you are a natural collaborator and a creative problem solver - this position might be for you! We are looking for a Senior Data Engineer to join our Data team and be a part of Nest's transformation to use its data and insight to drive decision making.You’ll work closely with stakeholders to … develop, test, and maintain data products and solutions that align with their requirements. You’ll be helping to develop and maintain a cloud-based data and analytics platform on Azure and help end-users of every level integrate their workflows between this and other platforms, including AWS-based solutions.You’ll combine data from different … sources, and build data pipelines to transform and organise raw data into formats and structures that can be easily used for business intelligence, visualisations and dashboards, deep dive analysis, and long term forecasting/modelling. To be successful in this positions you will have the following skills, competencies and experience: You’re a natural collaborator and More ❯

what we do that matters–it's how we do it. DRW is a place of high expectations, integrity, innovation and a willingness to challenge consensus. As a Data Engineer on our Data Experience team, you will play an integral role in bringing vendor datasets into our data platform, governing our centralized data pipelines, supporting rapid data product development, and working alongside individual Traders, Quantitative Researchers, and Back-Office personnel to best utilize the firm's data and platform tools. Technical Requirements Summary: Have experience designing and building data pipelines Have experience working within modern batch or streaming data ecosystems An expert in … their associated trade-offs. Experience with multiple data formats and serialization systems (e.g. Arrow, Parquet, Protobuf/gRPC, Avro, Thrift, JSON, etc.) Experience managing data pipeline orchestration systems (e.g. Kubernetes, Argo Workflows, Airflow, Prefect, Dagster, etc.) Proven experience in managing the operational aspects of large data pipelines such as backfilling datasets, rerunning batch More ❯

Engineer, you will operate within an Agile delivery environment, working closely with the Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from our wide-ranging dataset to support executive and operational decision making that will underpin sustained growth of BFS business units … solutions. You will also have management responsibilities for a small team of Data Engineers, who you will coach, support and organise to ensure we sustain a predictable pipeline of delivery, whilst ensuring all appropriate governance and best practice is adhered to. Your responsibilities as our Senior Databricks Engineer will include: Understand the business/product strategy and … Azure DevOps, collaborating on Sprint cycles, code deployment, version control, and development practices Develop and maintain data models, schemas, and documentation Monitor and troubleshoot data pipeline issues, ensuring timely resolution Stay up-to-date with the latest industry trends and technologies in data engineering and recommend improvements to existing systems or processes as More ❯

COSTAR GROUP - SENIOR DATA ENGINEER ONTHEMARKET TECHNOLOGY - LONDON OVERVIEW CoStar Group (NASDAQ: CSGP) is a leading global provider of commercial and residential real estate information, analytics, and online marketplaces. Included in the S&P 500 Index and the NASDAQ 100, CoStar Group is on a mission to digitize the worlds real estate, empowering all people to discover properties … content, imagery, design and functionality. The OnTheMarket product will support home buyers, sellers, and agents better than ever! Learn more about OnTheMarket . ROLE DESCRIPTION The role ofSenior Data Engineerwill be a key role within the OTM Data team, reporting to the Head of Data and working closely with the CTO. The Data team is responsible for data acquisition, data loading, data quality, data processing, data delivery, and data support for OnTheMarket. This role will provide focus on delivering value to our customers, by leveraging data to improve the OTM users and agents experience. This role would More ❯

An opportunity has arisen for a Senior Data Engineer to join a well-established biotech company using large-scale genetic data and AI to predict disease risk and advance precision healthcare. As a Senior Data Engineer , you will be responsible for developing, automating, and optimising scalable data pipelines using modern cloud technologies. … with hybrid/remote working options offering a salary of £500 - £650 per day (Inside IR35) and benefits. You Will Be Responsible For: Designing and implementing cloud-based data architectures using Azure services. Building robust and scalable data pipelines to support complex, high-volume processing. Deploying and managing containerised workloads through Kubernetes, Helm, and Docker. Automating … Engineer, Data Engineer, Data Platform Engineer, Data Architect, Data Infrastructure Engineer, Cloud Data Engineer, DataOps Engineer, Data Pipeline Engineer, Devops Engineer or in a similar role. Proven experience with Azure cloud platforms and related architecture. Highly skilled in Python for data engineering, scripting, and automation. More ❯

We’re looking for an innovative Data Architect to lead the creation and rollout of a comprehensive Data Governance Strategy, alongside the implementation of a data platform. This role will be instrumental in ensuring organizational data is accurate, secure, compliant, and strategically utilized to unlock business value. This is a new role … created due to growth, offering an exciting opportunity to shape and lead our data governance and platform strategy from the ground up. Key responsibilities Design and implement a comprehensive data governance framework. Define and maintain data governance policies, standards, and procedures. Establish and lead a cross-functional Data Governance Council. Collaborate with … data owners, stewards, and custodians across departments to ensure alignment and accountability. Develop and enforce data quality standards, metrics, and monitoring processes. Ensure compliance with data protection regulations such as GDPR, CCPA, and other relevant standards. Evaluate, select, and implement data governance tools (e.g. data catalogues, data quality More ❯

Hatfield based (with min of 2 days in the office) Competitive basic salary + Core & flexible benefits Join Data Management within our Data & Analytics function. We operate across four pillars: Data Management, Data Strategy, Data Governance and Data Delivery. We work in agile, delivering in sprints with modern … software engineering and DevOps practices to ship reliable data products at pace. The team owns the Group Data Warehouse, a global data platform that supports Computacenters combined reporting requirements across the world. In Data Management, you will turn Data Demand into prioritised backlog requirements and partner with business units to … deliver maximum customer value from data. If you enjoy practical delivery, measurable outcomes and helping teams make better decisions with trusted data, you will fit right in. What youll do As a Senior Data Consultant, you will play a key role in shaping and delivering data solutions that support business strategy. You will lead More ❯

Fabric tools to support innovative solutions. Collaborating with product owners and analysts, this role will ensure data flows and outputs drive CRB decision making. The role combines pipeline development and quality with dashboard delivery and results. Responsibilities The Role: Pipeline Development & Quality (≈ 60%) – Building Robust Data Infrastructure: • Build and maintain CRB specific data … dashboard performance, incorporating user feedback for improvements Qualifications The Requirements: • Bachelor's degree in Computer Science, Data Science, or related field • Experience in data engineering, pipeline development, or ETL processes • Proficiency in scripting (Python, SQL) and data integration tools • Expertise with Microsoft Azure/Fabric for data storage and processing • Knowledge … with APIs and data architecture in cloud environments • Ability to work in agile settings, collaborating with cross-functional teams • Strong problem-solving skills for data pipeline and quality issues •Proven delivery on data-driven projects •Ability to translate technical outputs into business-relevant results Enjoy a benefits package designed to help you thrive More ❯

Data Architect (Azure) 6 Month Contract UK Based Via Umbrella Our UK leading life and pensions client are looking for a Data Architect to joun their team on an initial 6 month contract. Key Responsibilities Data Architecture & Design Define and maintain enterprise data architecture standards, models, and frameworks. Design data solutions leveraging Azure services such as Azure Data Lake, Azure SQL Database, Azure Synapse Analytics, Azure Data Factory, and Azure Databricks. Data Integration & ETL Develop and optimize data pipelines for ingestion, transformation, and storage using Azure Data Factory and Databricks. Governance & Security Implement data governance, security … and compliance practices aligned with financial services regulations (e.g., GDPR, PCI DSS). Performance & Scalability Ensure data solutions are optimized for performance and scalability across large datasets. Collaboration Work closely with data engineers, analysts, and business stakeholders to deliver robust data platforms. Innovation Stay current with emerging Azure technologies and best practices in data More ❯

Lead Data Engineer Lead Data Engineer required to join my Client’s Technology Team and contribute to their Group Architecture Board in building the next generation of their eCommerce data platform. You will play a pivotal role in designing and maintaining a modern, cloud-based data architecture. Working closely with product, technology … and wider business stakeholders, you will be crucial in developing scalable data pipelines, improving reliability, and enabling smarter, data-driven insights and decisions. You will have a direct voice in how we design, build, and evolve our platform and the technologies we adopt! We are looking for a pragmatic, innovative, and self-motivated problem solver who … Architecture Specialist: Design and develop robust data models, ETL/ELT processes, and data marts that support complex analytics and operational use cases. Pipeline Optimization: Maintain and optimize data pipelines and warehouse environments to ensure top-tier performance, reliability, and scalability. Cross-Functional Collaboration: Effectively collaborate with cross-functional teams to More ❯

Job Description: Data Engineer (with Tableau Specialism) Responsibilities: * Design, develop, and manage data models, schemas, and metadata to support advanced analytics and reporting requirements. * Work closely with Data Analysts, Data Scientists, and Business Analysts to ensure data availability, accessibility, and usability across the organisation. * Implement data quality checks … integrity and reliability. * Optimise data workflows for performance, cost-efficiency, and maintainability using tools such as Azure Data Factory, AWS Data Pipeline, Databricks, or Apache Spark . * Integrate and prepare data for Tableau dashboards and reports , ensuring optimal performance and alignment with business needs. * Collaborate with visualisation teams to … develop, maintain, and enhance Tableau data sources (including Extracts, Hyper files, and live connections) and to support self-service analytics. * Ensure compliance with data governance, security, and privacy policies , maintaining adherence to organisational standards. * Document data architecture, pipelines, and Tableau data models to promote transparency, maintainability, and knowledge sharing. * Contribute to the More ❯

Adword Job Title: Data Architect (Insurance Domain) Location: London UK Hybrid role Responsibilities: Experience Level 15 years with at least 5 years in Azure ecosystem Role: We are seeking a seasoned Data Architect to lead the design and implementation of scalable data solutions for a strategic insurance project The ideal candidate will have deep … expertise in Azure cloud services Azure Data Factory and Databricks with a strong understanding of data modeling data integration and analytics in the insurance domain Key Responsibilities Architect and design end to end data solutions on Azure for insurance related data workflows Lead data ingestion transformation and orchestration … using ADF and Databricks Collaborate with business stakeholders to understand data requirements and translate them into technical solutions Ensure data quality governance and security compliance across the data lifecycle Optimize performance and cost efficiency of data pipelines and storage Provide technical leadership and mentoring to data engineers and developers Mandatory More ❯

- someone who gets excited about finding edge cases, building robust pipelines, and seeing their clean, validated datasets power accurate ML predictions. What You'll Do Data Pipeline Engineering Build production ETL pipelines that process mobile analytics events from AppsFlyer, Adjust, and other MMPs Engineer features for ML models: temporal patterns, user behaviour sequences, campaign attribution Ensure … real-world data: duplicates, out-of-order events, schema drift, missing fields, null handling Infrastructure & Orchestration Build Prefect workflows to orchestrate our multi-stage data pipeline Optimize and monitor BigQuery operations for cost and performance Establish CI/CD practices as we scale from a dozen clients to 100+ within 12 months Platform Development Build … pre-commit) Experience with messy real-world data: you've dealt with duplicates, nulls, schema drift, out-of-order events, and know how to build defensive pipelines Data quality mindset: you instinctively add validation checks, sanity-check implausible results, and think about edge cases before they become production bugs Right to Work in the UK (we More ❯

Data Architect - Fully Remote role Data Architect sought by a well-established Data & Analytics consultancy. Our client is seeking a cloud professional to spearhead strategic leadership and technical expertise within their data solutions landscape, focussed on cloud data platforms, but including IoT analytics, data integration, and migration projects. … This opportunity is offered on a remote-first basis, but with an open invitation to their headquarters any time. Role Responsibilities: The Data Architect will be pivotal in shaping the approach to designing and developing data solutions using cloud technologies such as AWS and Azure. This position involves: Leading the design and implementation of innovative cloud … using Azure, AWS, or exceptional on-premise design expertise. At least 5 years in data engineering or business intelligence roles. Proficiency in ETL and data pipeline design, with a technology-agnostic approach. A solid understanding of data warehouse and data lake principles. Expert SQL skills and demonstrable data modelling More ❯

Robert Half have partnered with a market leading property development organisation for an experienced Azure Data Engineer to join their data and analytics team, on an initial 3-6 month contract based out of their London office The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines in … the Azure cloud environment, with hands-on experience in modern data platforms and ETL frameworks. Key Responsibilities: Design, develop, and maintain data pipelines and data integration solutions using Azure Data Factory, Synapse Analytics, and related services. Build and optimise data models to support reporting, analytics, and machine learning use cases. … Implement robust data quality, governance, and security practices across the data ecosystem. Collaborate with analysts, data scientists, and business stakeholders to understand data needs and translate them into technical solutions. Work with structured and unstructured data sources, ensuring scalability, reliability, and performance. Monitor and troubleshoot data workflows, ensuring More ❯

Engineer - £400PD - Remote Seeking an experienced Data Engineer to design, build, and optimise data solutions within the Microsoft Azure ecosystem. The role focuses on pipeline development, data modelling, governance, and supporting analytics teams with high-quality, reliable data. Key Responsibilities: Develop and maintain scalable data pipelines using Azure Data Factory, Synapse, Databricks, and Microsoft Fabric. Build efficient ETL/ELT processes and data models to support analytics, reporting, and dashboards. Optimise existing pipelines for performance, reliability, and cost efficiency. Implement best practices for data quality, error … handling, automation, and monitoring. Manage data security and governance, including Row-Level Security (RLS) and compliance standards. Maintain documentation and metadata for data assets and pipeline architecture. Collaborate with analysts, data scientists, and stakeholders to deliver trusted data products. Provide technical support and troubleshoot production issues. Contribute to improving data More ❯

The role: We are looking for a Data Engineer to establish our internal data engineering capability and play a key role in designing and developing the data infrastructure and systems, ensuring that our data pipelines are reliable, scalable, and efficient. Reporting into the Software Team Lead, you will be responsible for designing … data infrastructure to support modelling and data analytics, with strong competence in Python, and work with stakeholders to understand their data needs and help design data solutions. The Technology team within Prevail underpins all of our mission focused teams. No two days are the same and the real-world problems we seek … solve are complicated. You will get the opportunity to interact with a wide range of datasets and support both the wider tech team and mission teams turning this data into insights to drive our customers decision making. The team structure is flat with a network centric approach to leadership. A small team, we rely on strong communication and More ❯

primarily images) and keep our training flows fast, reliable, and repeatable. You'll design and operate high-throughput ingestion and preprocessing on Kubernetes, evolve our internal data-pipeline framework, and work hand-in-hand with ML engineers to ship datasets that move model quality forward. Key Responsibilities Develop and maintain data-ingestion pipelines to source … to-end flow: raw data → quality/beauty/relevance filtering → dedup/validation → ready-to-train artifacts. Operate and improve our Kubernetes-based data-pipeline framework (distributed jobs, retries, monitoring, automation). Work with S3-style object storage: efficient layouts, lifecycle, throughput, and cost awareness. Add tooling around pipelines (progress/health visualization, metrics … fast. Nice-to-have Familiarity with ML workflows (PyTorch) and downstream training considerations. Experience with image quality scoring, captioning, or image-to-text pipelines. DAG/workflow visualizations or pipeline UX tooling. DevOps fluency: Docker, CI/CD, infra automation. What We Offer Competitive salary and equity. We're able to offer Skilled Worker visa sponsorship in the UK More ❯

AWS Data Engineer | £ 75 ,000 - £90,000 | Central London (Hybrid - 3 days on-site) We're proud to partner with a high-growth fintech on the lookout for an AWS Data Engineer to join their fast-paced, data-driven organisation. This role is a great opportunity for someone who's eager to make an … impact and get hands-on with modern tools. You'll design, build, and optimise scalable data pipelines and warehouse solutions while collaborating across teams to ensure reliable, secure, and efficient data infrastructure for the organisation. What you'll be doing: Design, develop, and maintain scalable data architectures and ETL pipelines Build and manage data models and data warehouse solutions (Airflow, dbt, and Redshift) Write clean, efficient Python and SQL code for data processing and transformation Integrate data from internal and third-party APIs and services Optimise data pipelines for performance, scalability, and reliability Collaborate with data scientists, analysts, and engineering teams to More ❯

Head of Data Engineering Hybrid - Kent (3 Days per Week) Up to £130,000 + 30% Bonus + Benefits Are you an experienced data leader ready to take ownership of a large-scale data transformation? We're working with a leading UK financial services group that's on a major journey to modernise its … data estate - and they're looking for a Head of Data Engineering to lead the migration from on-premise to a modern, Azure-based cloud platform. This is a high-impact leadership role, ideal for someone who combines strategic vision with hands-on technical credibility and thrives on building, scaling, and delivering enterprise-grade data … solutions. Why this role? Lead the migration of legacy on-premise systems to Azure , driving one of the organisation's most strategic technology programmes. Shape and execute the data engineering roadmap , delivering scalable, secure, and compliant data solutions. Build and mentor a high-performing engineering team , embedding modern cloud-first practices. Hybrid flexibility: 3 days per More ❯

Robert Half Technology are assisting a market leading real estate organisation to recruit a Data Engineer on a contract basis - Hybrid Working - London based - Outside IR35 In this role, you will manage and optimise the data infrastructure, driving insights and supporting our decision-making processes. Your expertise will be crucial in contributing to our organisation's … data-driven success. As a key member of our team, you will have the opportunity to work on exciting projects and make a significant impact on our business Role The Data Engineer will design, develop, and maintain data infrastructure to ensure reliability, scalability, and performance. Collaborate with data scientists and analysts to … integrity. Implement efficient data models and schemas for storage and retrieval. Optimise data processes for improved performance and efficiency. Troubleshoot and resolve data pipeline issues swiftly and effectively. Keep abreast of industry trends and emerging technologies in data engineering, and continuously improve your skills and knowledge. Profile The Data More ❯