51 to 75 of 492 Data Pipeline Jobs in the UK

Active SC Candidate only Experience 10 Years with relevant 6-8+ Years Essential Skills & Experience: 10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks . Good proficiency in Python and Spark (PySpark) or Scala. Deep understanding of data warehousing principles, data modelling techniques … and data integration patterns. Extensive experience with Azure data services, including Azure Data Factory, Azure … Blob Storage, and Azure SQL Database . Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks ,including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

The Chief Data Office (CDO) sits within our Chief Technology Office (CTO) and ensures data is managed as one of the organisation’s most valuable assets. As an apprentice in this area, you’ll be exposed to data governance and quality, data management, analytics and insight, regulation and compliance, data culture. Role The Data Engineering and Analytics Apprenticeship Programme within our CTO Function is a 12-month programme and offers an excellent grounding in the fundamental principles of data in one of the UK’s largest wealth management organisations. Key responsibilities: Learn and apply data engineering fundamentals - gain hands-on experience with … databases, cloud platforms, and data integration tools Support data pipelines - help design, build, and maintain processes that move and transform data between systems Assist with data quality checks - monitor data for accuracy, completeness, and consistency, escalating issues where needed Work with senior engineers - shadow and support in developing efficient, reliable More ❯

Data Engineer Permanent Hybrid (3 days in the office, 2 days WFH) London McCabe & Barton are partnering with a leading financial services client to recruit an experienced Data Platform Engineer . This is an excellent opportunity to join a forward-thinking team driving innovation with modern cloud-based data technologies. Role Overview As a … pipelines using Azure Data Factory , Databricks , and related Azure services. Build ETL/ELT processes to transform raw data into structured, analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage . … Data Factory, ADLS, Azure SQL Database, Synapse Analytics). Strong hands-on expertise in Databricks , Delta Lake , and cluster management . Proficiency in SQL and Python for pipeline development. Familiarity with Git/GitHub and CI/CD practices. Understanding of data modelling , data governance , and security principles. Desirable Skills Experience with Terraform More ❯

Following the acquisition of Brookson by People2.0, the responsibilities of the Data and Analytics team have significantly expanded. We are now seeking a skilled Data Engineer to join our dynamic team. As a Data Engineer within the Data and Analytics team, you will play a pivotal role in designing, maintaining, and optimising … the data architecture for People2.0 Group. This architecture ensures the seamless flow of data from source systems to the Data Warehouse, providing a robust foundation of aggregated analytical base tables and operational sources. Our data architecture is critical in enabling People2.0 Group to become a data-driven organisation. Your responsibilities … will include maintaining existing data pipelines, developing new branches within the architecture, and collaborating closely with stakeholders across the business. You will engage with stakeholders from various regions, including EMEA, US, and APAC, ensuring that the architecture aligns with regional and global requirements. Reporting directly to the Senior Data Engineer within the Data and More ❯

Azure Data Engineer - Security Cleared Role Summary: We're looking for an experienced Azure Data Engineer to join our team and lead the development of scalable, secure, and high-performing data solutions within the Microsoft Azure ecosystem. This role will focus on building and refining data pipelines, enhancing data architecture … and ensuring robust governance and compliance across enterprise data platforms. You'll collaborate closely with cross-functional teams to deliver reliable data products that support business intelligence, analytics, and operational reporting. Core Responsibilities: 1. Data Engineering & Architecture Design and implement end-to-end data pipelines using Azure Data Factory, Synapse … Analytics, and Databricks (or Microsoft Fabric where applicable). Develop ETL/ELT workflows to integrate and transform data from diverse sources. Create and maintain data models to support reporting, dashboards, and analytical use cases. 2. Performance & Reliability Audit and enhance existing data infrastructure for improved efficiency and scalability. Apply best practices for data More ❯

Engineer - £400PD - Remote Seeking an experienced Data Engineer to design, build, and optimise data solutions within the Microsoft Azure ecosystem. The role focuses on pipeline development, data modelling, governance, and supporting analytics teams with high-quality, reliable data. Key Responsibilities: Develop and maintain scalable data pipelines using Azure Data Factory, Synapse, Databricks, and Microsoft Fabric. Build efficient ETL/ELT processes and data models to support analytics, reporting, and dashboards. Optimise existing pipelines for performance, reliability, and cost efficiency. Implement best practices for data quality, error … handling, automation, and monitoring. Manage data security and governance, including Row-Level Security (RLS) and compliance standards. Maintain documentation and metadata for data assets and pipeline architecture. Collaborate with analysts, data scientists, and stakeholders to deliver trusted data products. Provide technical support and troubleshoot production issues. Contribute to improving data More ❯

Engineer - £400PD - Remote Seeking an experienced Data Engineer to design, build, and optimise data solutions within the Microsoft Azure ecosystem. The role focuses on pipeline development, data modelling, governance, and supporting analytics teams with high-quality, reliable data. Key Responsibilities: Develop and maintain scalable data pipelines using Azure Data Factory, Synapse, Databricks, and Microsoft Fabric. Build efficient ETL/ELT processes and data models to support analytics, reporting, and dashboards. Optimise existing pipelines for performance, reliability, and cost efficiency. Implement best practices for data quality, error … handling, automation, and monitoring. Manage data security and governance, including Row-Level Security (RLS) and compliance standards. Maintain documentation and metadata for data assets and pipeline architecture. Collaborate with analysts, data scientists, and stakeholders to deliver trusted data products. Provide technical support and troubleshoot production issues. Contribute to improving data More ❯

My client is seeking an experienced Analytics Engineer to join on a contract basis. This role will focus on designing, developing, and optimising data infrastructure and analytical tools. You will collaborate closely with data … engineers, analysts, and business stakeholders to ensure data is reliable, accessible, and actionable. This role is 'remote first' and falls OUTSIDE IR35. Key Responsibilities Data Pipeline Development: Design, build, and maintain robust, scalable, and efficient data pipelines using DBT, SQL, Python, and ETL frameworks. Data Modelling: Develop and maintain logical and … physical data models to support reporting and analysis needs, implementing best practices for data warehouse design. Data Transformation: Convert raw data from multiple sources into clean, reliable datasets for analysis and reporting, ensuring quality and consistency. Collaboration with Stakeholders: Partner with analysts, scientists, and business teams to translate requirements into technical solutions. More ❯

Data Engineer Remote (UK-Based) Salary: c£60,000 - £65,000 per annum + benefits Employment Type: Full-time, Permanent About the Role Were looking for an experienced Data Engineer to join a highly acquisitive, fast-growing organisation thats driving a major digital transformation. The Data Engineer will be responsible for designing, building, and … maintaining robust data infrastructure to support the organisations expanding data and analytics needs. This includes leading the migration of legacy systems to modern cloud-based platforms , managing APIs for seamless data exchange , and ensuring data quality, security, and reliability across all systems. Youll play a key role in developing and optimising ETL … pipelines , automating data workflows , and supporting integration between core business systems such as Microsoft Dynamics . Strong hands-on experience with Azure, SQL, and Python-based data engineering is essential. This is an excellent opportunity for someone who thrives in a scalable, acquisition-driven environment , enjoys solving complex data challenges, and wants to help More ❯

Pentaho Developer - Pentaho, Data Engineering, Cloud, Big Data, ETL, Informatica A large global organisation is looking for an experienced Pentaho Developer to design, develop, and maintain ETL workflows and data integration solutions within a modern data engineering ecosystem. The role involves building scalable data pipelines, ensuring data quality … time, permanent position. The location is London, UK. You will be required to work 3 days on-site, and 2 days remote. KEY SKILLS & EXPERIENCE: Expertise in Pentaho Data Integration for ETL processes. Strong understanding of data warehousing concepts, data modelling, and schema design. Experience with other ETL tools (Informatica, Talend, SSIS). Advanced … SQL for querying and data manipulation. Familiarity with Java, Python, or VBA for Scripting and automation. Knowledge of RESTful APIs and various database systems. Skills in data profiling, data analysis, and ensuring data quality. Exposure to Banking/Financial Services domain is a plus. RESPONSIBILITIES: Design, develop, and manage ETL workflows using More ❯

Senior Data Analyst Location: Hybrid (2/3 day per week in our Heald Green office) Salary: Up to £65,000 depending on experience About My Claim Group My Claim Group is a fast-growing UK-based company helping customers navigate complex financial and insurance claims. Our success is built on trust, innovation, and data-driven … decision-making - and now we're looking to expand our analytics team to support smarter, faster business outcomes. We're on the lookout for a talented Senior Data Analyst with strong SQL skills and exposure to Snowflake. This is a fantastic opportunity to join a growing team, create meaningful insights, and influence decisions across the business. The Role … experience in data visualisation (Power BI ideal). Solid SQL skills and comfort working with large, complex datasets. Strong grasp of ETL concepts and data pipeline processes. Proficiency in data cleaning, transformation, and preparation. Excellent communication and data storytelling ability. Ability to collaborate with stakeholders and translate requirements into analytical solutions. More ❯

Engineer London (five days per week on site) £65-75k (dependent on experience) The Highlights This global logistics business, based in central London, is undergoing an exciting data transformation programme as it invests in a new team charged with building and implementing a new Azure Databricks platform. Working five days a week in the central London office … you'll join this growing team and play a key role in building and deploying robust data infrastructure and analytics solutions using modern data stack The business is evolving and maturing its approach to data, creating an opportunity to build data solutions that have a significant impact on the performance of the … business. Working closely with a multi skilled team of the Data Science, Engineers, Analytics and Analyst professionals, you will have the opportunity build your skills in analytics engineering, responding to business and project needs rather than operating as a narrow silo. You'll work hands-on with Azure Databricks, Azure Data Factory, Delta Lake, and Power More ❯

pace and role dependant. You will be open and honest and show that you are reliable. As our Business Intelligence Analyst , you will be responsible for transforming organisational data into meaningful insight. This role will focus on understanding data structures within Databricks, preparing data for analysis, and building impactful Power BI dashboards and automated … reports. It is ideal for someone who is technically capable, data-driven and passionate about enabling the business to make informed decisions. This role sits within our Commercial Finance team and reports into the FP&A Manager. Job Responsibilities Your key responsibilities as Business Intelligence Analyst... Build, enhance and maintain Power BI dashboards and reports that effectively visualise … and reduce manual workload. Work closely with business stakeholders to gather requirements and translate them into BI reporting solutions. Continuously review and enhance BI processes, reporting capabilities and data preparation workflows. Understand data structures, data flows and how data is built within Databricks. Explore and test existing data pipelines to More ❯

The role: We are looking for a Data Engineer to establish our internal data engineering capability and play a key role in designing and developing the data infrastructure and systems, ensuring that our data pipelines are reliable, scalable, and efficient. Reporting into the Software Team Lead, you will be responsible for designing … data infrastructure to support modelling and data analytics, with strong competence in Python, and work with stakeholders to understand their data needs and help design data solutions. The Technology team within Prevail underpins all of our mission focused teams. No two days are the same and the real-world problems we seek … solve are complicated. You will get the opportunity to interact with a wide range of datasets and support both the wider tech team and mission teams turning this data into insights to drive our customers decision making. The team structure is flat with a network centric approach to leadership. A small team, we rely on strong communication and More ❯

primarily images) and keep our training flows fast, reliable, and repeatable. You'll design and operate high-throughput ingestion and preprocessing on Kubernetes, evolve our internal data-pipeline framework, and work hand-in-hand with ML engineers to ship datasets that move model quality forward. Key Responsibilities Develop and maintain data-ingestion pipelines to source … to-end flow: raw data → quality/beauty/relevance filtering → dedup/validation → ready-to-train artifacts. Operate and improve our Kubernetes-based data-pipeline framework (distributed jobs, retries, monitoring, automation). Work with S3-style object storage: efficient layouts, lifecycle, throughput, and cost awareness. Add tooling around pipelines (progress/health visualization, metrics … fast. Nice-to-have Familiarity with ML workflows (PyTorch) and downstream training considerations. Experience with image quality scoring, captioning, or image-to-text pipelines. DAG/workflow visualizations or pipeline UX tooling. DevOps fluency: Docker, CI/CD, infra automation. What We Offer Competitive salary and equity. We're able to offer Skilled Worker visa sponsorship in the UK More ❯

Senior Engineer (Data) Department Digital, Data & Technology Group Salary Starting from £38,784, rising to £46,049 Closing date Sunday 30 November 2025We are seeking a talented Senior Data Engineer to join our team and contribute to the development and implementation of our University Data Hub. What is our University Data Hub? The University is undertaking a strategic project to enhance its Data & Insights capability - enabling the right people to access information and insights at the right time, with the right quality. We are building a brand-new University Data Hub in the AWS cloud, replacing an old on-premises solution. The UDH will transform … and integrate data from various sources, including live operating data, key data snapshots, and external datasets. It will manage structured and unstructured datasets, analyse incoming data quality to inform data governance, act as a single source of truth, and support all analytics activities. It will underpin reports and dashboards via More ❯

Senior Engineer (Data) Department Digital, Data & Technology Group Salary Starting from £38,784, rising to £46,049 Closing date Sunday 30 November 2025 We are seeking a talented Senior Data Engineer to join our team and contribute to the development and implementation of our University Data Hub. What is our University Data Hub? The University is undertaking a strategic project to enhance its Data & Insights capability enabling the right people to access information and insights at the right time, with the right quality. We are building a brand-new University Data Hub in the AWS cloud, replacing an old on-premises solution. The UDH will transform … and integrate data from various sources, including live operating data, key data snapshots, and external datasets. It will manage structured and unstructured datasets, analyse incoming data quality to inform data governance, act as a single source of truth, and support all analytics activities. It will underpin reports and dashboards via More ❯

Senior Data Engineer - Burton on Trent £64-67k x1 day per week on-site (Sponsorship is not provided for this opportunity) As a Senior Data Engineer, you will lead the development and optimisation of our customers Azure-based data platform, ensuring efficient, secure, and high-performing data services. You'll design … scalable pipelines and data models, champion data governance and best practices, and drive continuous improvement to enhance data quality and accessibility across the business. Key Responsibilities Data Engineering: Design, build, and maintain Azure data pipelines using Data Factory, Databricks, and related services. Data Architecture: Develop and … optimise scalable data models, warehouses, and lakes (Azure Synapse, Data Lake Storage). Governance & Security: Enforce compliance and data protection standards (GDPR, DPA) through robust security and governance practices. Automation: Implement CI/CD pipelines and Infrastructure as Code (Terraform, Bicep, ARM) via Azure DevOps. Performance & Monitoring: Optimise data systems for cost More ❯

AWS Data Engineer | £ 75 ,000 - £90,000 | Central London (Hybrid - 3 days on-site) We're proud to partner with a high-growth fintech on the lookout for an AWS Data Engineer to join their fast-paced, data-driven organisation. This role is a great opportunity for someone who's eager to make an … impact and get hands-on with modern tools. You'll design, build, and optimise scalable data pipelines and warehouse solutions while collaborating across teams to ensure reliable, secure, and efficient data infrastructure for the organisation. What you'll be doing: Design, develop, and maintain scalable data architectures and ETL pipelines Build and manage data models and data warehouse solutions (Airflow, dbt, and Redshift) Write clean, efficient Python and SQL code for data processing and transformation Integrate data from internal and third-party APIs and services Optimise data pipelines for performance, scalability, and reliability Collaborate with data scientists, analysts, and engineering teams to More ❯

Head of Data Engineering Hybrid - Kent (3 Days per Week) Up to £130,000 + 30% Bonus + Benefits Are you an experienced data leader ready to take ownership of a large-scale data transformation? We're working with a leading UK financial services group that's on a major journey to modernise its … data estate - and they're looking for a Head of Data Engineering to lead the migration from on-premise to a modern, Azure-based cloud platform. This is a high-impact leadership role, ideal for someone who combines strategic vision with hands-on technical credibility and thrives on building, scaling, and delivering enterprise-grade data … solutions. Why this role? Lead the migration of legacy on-premise systems to Azure , driving one of the organisation's most strategic technology programmes. Shape and execute the data engineering roadmap , delivering scalable, secure, and compliant data solutions. Build and mentor a high-performing engineering team , embedding modern cloud-first practices. Hybrid flexibility: 3 days per More ❯

Job Description Data Technical Specialist - Submarines Full Time Derby, Raynesway We are seeking a talented and highly motivated Senior Data Engineering specialist to join our Central Digital team. As a Data Technical Specialist, you will be recognised as an authoritative source of technical expertise and advice within the business for Data and … viewed as a key member of the broader team of technical expertise by senior leadership. You will be responsible for defining data management policies and procedures that ensure that data is properly collected, stored, maintained, and utilised, in line with the business strategy. This incorporates defining and managing data governance, quality, and retention policies. … You will be responsible for designing, building and maintaining data pipelines and architecture that empowers data-driven decisions across the organisation. You will work closely with the cross-functional teams, including data engineers, data scientist, product teams to ensure data is seamlessly integrated, transformed and accessible for analytical and operational More ❯

Robert Half Technology are assisting a market leading real estate organisation to recruit a Data Engineer on a contract basis - Hybrid Working - London based - Outside IR35 In this role, you will manage and optimise the data infrastructure, driving insights and supporting our decision-making processes. Your expertise will be crucial in contributing to our organisation's … data-driven success. As a key member of our team, you will have the opportunity to work on exciting projects and make a significant impact on our business Role The Data Engineer will design, develop, and maintain data infrastructure to ensure reliability, scalability, and performance. Collaborate with data scientists and analysts to … integrity. Implement efficient data models and schemas for storage and retrieval. Optimise data processes for improved performance and efficiency. Troubleshoot and resolve data pipeline issues swiftly and effectively. Keep abreast of industry trends and emerging technologies in data engineering, and continuously improve your skills and knowledge. Profile The Data More ❯

Senior Data Engineer (Python/AWS) Full-time | London, UK Hybrid working: 2-3 days per week on-site Must be SC cleared, or eligible to obtain SC-level security clearance Job description We are looking for a developer with expertise in Python, AWS Glue, step function, EMR cluster and redshift, to join our AWS Development Team. You … be required as part of the interview process. Key responsibilities Hands-on experience in Python programming is essential Responsible for Build, Test and Maintain optimal AWS data pipeline architecture. Building AWS glue, step function, AWS Lambda functions and unit test Work with fellow developers, data architects, data analysts and data scientists … application infrastructure, suggesting changes to it Define and document best practices and strategies regarding application deployment and infrastructure maintenance Define service capacity planning strategies Implement the application's CICD pipeline using the AWS CICD stack Project tasks deliverables and management Deliver on project progress and ensure adherence to client expectation Communications, including deck writing Delivery of output Gathering of More ❯

Senior Data Engineer Salary: Up to £70,000 I am working with a forward-thinking organisation that is modernising its data platform to support scalable analytics and business intelligence across the Group. With a strong focus on Microsoft technologies and cloud-first architecture, they are looking to bring on a Data Engineer to help … design and deliver impactful data solutions using Azure. This is a hands-on role where you will work across the full data stack, collaborating with architects, analysts, and stakeholders to build a future-ready platform that drives insight and decision-making. In this role, you will be responsible for: Building and managing data pipelines … using Azure Data Factory and related services. Building and maintaining data lakes, data warehouses, and ETL/ELT processes. Designing scalable data solutions and models for reporting in Power BI. Supporting data migration from legacy systems into the new platform. Ensuring data models are optimised for performance and More ❯

Senior Data Engineer (Databricks) Full Time/Permanent £70,000 - £80,000 + car allowance, generous bonus, private medical and other extensive benefits Hybrid - 1 day a week in the Oxfordshire head office required The Company: My client is an industry leading and award-winning financial services organisation who operate on a global scale. They are headquartered in … Oxfordshire, UK and also have offices throughout the UK, Europe and the US. The Role: We are looking for a driven and experienced Senior Data Engineer with extensive Databricks experience to join a rapidly growing in house Data team during an exciting period of change. This is a role where you can really make your mark … an Agile delivery environment, working closely with the Head of Data, Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from a wide-ranging dataset to support executive and operational decision making that will underpin sustained growth of business units domestically More ❯