76 to 100 of 492 Data Pipeline Jobs in the UK

Databricks Engineer - £400PD - Remote About the Role We are seeking a highly skilled Azure Data Engineer with deep, hands-on Databricks experience to join our growing data engineering team. In this role, you will design, build, and optimise scalable data pipelines and lakehouse architectures on Azure, enabling advanced analytics and data-driven … decision making across the business. Key Responsibilities Design, develop, and maintain ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake. Build and optimise data lakehouse architectures on Azure Data Lake Storage ( ADLS ) . Develop high-performance data solutions using Azure Synapse, Azure Data Factory, and Databricksworkflows Implement best practices for … security, and quality across all pipelines. Collaborate with data scientists, analysts, and cross-functional teams to deliver reliable, production-grade data models. Monitor and tune pipeline performance to ensure efficiency, reliability, and cost optimisation. Participate in CI/CD processes and infrastructure-as-code solutions using tools like Terraform, GitHub Actions, or Azure DevOps Required More ❯

Data Engineer - Azure & Databricks - £70,000 Location: Hybrid, London Employment Type: Full-time About the Role: A global leader within their relevant field is looking for an experienced Data Engineer to join a global organization and help deliver innovative data solutions. You'll work on the full data engineering and business intelligence … lifecycle, supporting analytics and AI needs across multiple business functions. Key Responsibilities: Design and implement scalable data solutions using Azure Data Services and Databricks Develop and maintain data pipelines for automated data collection and integration Create and optimize data models and transformations Build interactive dashboards and reports using Power BI … Ensure data quality, integrity, and security across all stages of processing Collaborate with cross-functional teams to define requirements and deliver solutions Requirements: Hands-on experience with Azure Data Factory, Databricks, SQL, Data Lake, Power BI Strong skills in T-SQL and DAX Experience in ETL development, data modelling, and data More ❯

Data Engineer - Azure & Databricks - £70,000 Location: Hybrid, London Employment Type: Full-time About the Role: A global leader within their relevant field is looking for an experienced Data Engineer to join a global organization and help deliver innovative data solutions. You'll work on the full data engineering and business intelligence … lifecycle, supporting analytics and AI needs across multiple business functions. Key Responsibilities: Design and implement scalable data solutions using Azure Data Services and Databricks Develop and maintain data pipelines for automated data collection and integration Create and optimize data models and transformations Build interactive dashboards and reports using Power BI … Ensure data quality, integrity, and security across all stages of processing Collaborate with cross-functional teams to define requirements and deliver solutions Requirements: Hands-on experience with Azure Data Factory, Databricks, SQL, Data Lake, Power BI Strong skills in T-SQL and DAX Experience in ETL development, data modelling, and data More ❯

Databricks Engineer - £400PD - Remote About the Role We are seeking a highly skilled Azure Data Engineer with deep, hands-on Databricks experience to join our growing data engineering team. In this role, you will design, build, and optimise scalable data pipelines and lakehouse architectures on Azure, enabling advanced analytics and data-driven … decision making across the business. Key Responsibilities Design, develop, and maintain ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake. Build and optimise data lakehouse architectures on Azure Data Lake Storage ( ADLS ) . Develop high-performance data solutions using Azure Synapse, Azure Data Factory, and Databricks workflows Implement best practices … security, and quality across all pipelines. Collaborate with data scientists, analysts, and cross-functional teams to deliver reliable, production-grade data models. Monitor and tune pipeline performance to ensure efficiency, reliability, and cost optimisation. Participate in CI/CD processes and infrastructure-as-code solutions using tools like Terraform, GitHub Actions, or Azure DevOps Required More ❯

Job Description: Role Summary/Purpose: The SVP, Model Operations & Analytics Leader leads a multidisciplinary team supporting the full model development lifecycle, from data readiness to post-implementation monitoring. This role drives the creation of standardized frameworks, tools, and automation — including developing data pipelines, managing enterprise end-to-end data warehouse, performing preventative and … detective data analysis and monitoring of systems — to ensure model quality, consistency, and regulatory compliance. Partnering closely with model developers, risk management, and audit functions, the leader ensures operational alignment with governance standards and regulatory expectations. They oversee strategic initiatives to enhance efficiency, scalability, and performance monitoring, while also managing business risk. As a key thought partner to … Proficiency in developing business requirements for enterprise data warehouses and model enablement tools; experience with model documentation standards and governance protocols. Technical expertise in data pipeline development, data testing, and statistical performance monitoring using tools such as Python, SQL, R, and/or commercial analytics platforms. Demonstrated ability to lead root cause diagnostics More ❯

markets or alternative asset classes. About the Department Join Aberdeen's Investments department, where we help clients plan and invest with confidence for a brighter future. Our dynamic Data Tribe, led by the Chief Data Officer, is revolutionizing data accessibility and innovation through a cutting-edge Data Mesh architecture. Collaborate with passionate … teams to unlock powerful data insights that drive sustainable investment and fuel scalable growth. Be part of a forward-thinking environment making a real impact. About the Role As a Data Platform Engineer at Aberdeen, you will be at the heart of building and evolving a modern, cloud-based data platform using Azure, Snowflake … data models. Proficiency in infrastructure-as-code deployments, especially using Terraform for Azure, Snowflake, and related platforms. Experience with Azure DevOps CI/CD pipelines, including YAML pipeline creation, artifact management, and automated secrets handling. Good Scripting skills in SQL, PowerShell, and Python, combined with a strong understanding of security, compliance, and agile delivery methodologies.About the department More ❯

Job Description Data Technical Specialist - Submarines Full Time Derby, Raynesway We are seeking a talented and highly motivated Senior Data Engineering specialist to join our Central Digital team. As a Data Technical Specialist, you will be recognised as an authoritative source of technical expertise and … advice within the business for Data and viewed as a key member of the broader team of technical expertise by senior leadership. You will be responsible for defining data management policies and procedures that ensure that data is properly collected, stored, maintained, and utilised, in line with the business strategy. This incorporates defining and … managing data governance, quality, and retention policies. You will be responsible for designing, building and maintaining data pipelines and architecture that empowers data-driven decisions across the organisation. You will work closely with the cross-functional teams, including data engineers, data scientist, product teams to ensure data is More ❯

engaging with clients, understanding their needs, and translating them into effective outcomes. You understand and align with Solirius Reply Values. The Role: We are looking for an experienced Data Engineer to join our team here at Solirius. You will be working as part of a team, developing and delivering exciting projects with a fantastic team of technology experts. … You will be responsible for designing, developing, and maintaining data pipelines and systems that enable data analysis and machine learning. You will also collaborate with data scientists, analysts, and other stakeholders to ensure data quality and reliability. Requirements Key Responsibilities: Develop Data Engineering solutions for our clients/projects Design … and build data models, schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client and More ❯

AWS Data Engineer Rate: Up to £400 per day (Outside IR35) Contract Length: 6 Months (with strong potential for extension) Location: Hybrid - 1 day per week onsite in Central London A leading organisation is seeking an experienced AWS Data Engineer to join their data and analytics team, contributing to the design, development and optimisation … of large-scale data solutions within a modern cloud environment. This contract offers the opportunity to work on high-impact projects, delivering data platforms and pipelines that drive real-time insights and strategic business decisions. Responsibilities for the AWS Data Engineer: Design, build and maintain scalable data pipelines and architectures within the … AWS ecosystem Leverage services such as AWS Glue, Lambda, Redshift, EMR and S3 to support data ingestion, transformation and storage Work closely with data analysts, architects and business stakeholders to translate requirements into robust technical solutions Implement and optimise ETL/ELT processes , ensuring data integrity, consistency and quality across multiple sources Apply best More ❯

deliver high quality business and complex clinical reporting structures from disparate systems within a consistent and stable framework that is monitored and assured. We are seeking an experienced Data Engineer with a strong background in SQL Server and a commitment to formal change management procedures (RFC) and version control (Git). We are at the start of a … five-year Data Engineering modernisation programme; this includes the adoption of Python for developing data pipelines, automating data integration, and building robust data solutions. We are proud to be a diverse and inclusive organisation and there is a choice of staff networks that help you meet like-minded people. The Christie supports … for the NHS or in a healthcare environment is not essential for this role. 12-month fixed term (renewable) Main duties of the job To deliver and maintain data transfer pipelines between clinical/operational source systems and the Christie Central Data Repository (CCDR). Ensure that all extract, transform and load (ETL) processes are robust More ❯

the financial year (March 26) Start ASAP Active SC required Key Responsibilities Dashboard Development: Design and build interactive Tableau dashboards and reports to provide actionable insights for stakeholders. Data Engineering: Develop and maintain robust data pipelines, ensuring data integrity, scalability, and performance across multiple sources. Data Integration: Work with structured and unstructured … data from various internal and external systems, applying ETL best practices. Collaboration: Partner with analysts, data scientists, and business teams to understand requirements and deliver high-quality solutions. Performance Optimisation: Monitor and tune Tableau Server performance, ensuring efficient query execution and user experience. Governance & Security: Implement data governance standards and ensure compliance with Home … Experience Proven experience in Tableau development (Desktop and Server) with strong visualisation and storytelling skills. Solid background in data engineering, including ETL processes and data pipeline development. Proficiency in SQL and experience with relational databases (e.g., Oracle, PostgreSQL, SQL Server). Familiarity with cloud platforms (AWS, Azure, or GCP) and data warehousing solutions. More ❯

Robert Half Technology are assisting a large, market-leading media organisation to recruit a Data Engineer on a contract basis . Hybrid working - London based. Role * The Data Engineer will design, develop, and maintain modern cloud-based data infrastructure to ensure reliability, scalability, and performance.* Collaborate with data analysts, BI teams, and … product stakeholders to fulfil their data requirements and support business deliverables.* Collect, transform, and process datasets from various internal and external sources, ensuring data quality, governance, and integrity.* Implement efficient data models and schemas … within Snowflake, and use DBT for transformation, orchestration, and workflow management.* Optimise ELT/ETL processes for improved performance, cost efficiency, and scalability.* Troubleshoot and resolve data pipeline issues swiftly and effectively across the data platform.* Work with orchestration tools such as Airflow, ADF, or Prefect to schedule and automate workflows.* Keep abreast of industry More ❯

Data Engineer - Cardiff/Hybrid - £45,000 - £50,000 + benefits Yolk Recruitment are excited to be working with a global technology business that's continuing to expand its data capability and invest in modern cloud solutions. Known for their collaborative culture and commitment to innovation, they're offering an excellent opportunity for a Data Engineer to make a real impact. We're looking for a Data Engineer to help design, build, and maintain scalable data pipelines and systems that power analytics and business intelligence across the organisation. You'll play a key role in ensuring data is accurate, accessible, and high-quality - driving data-led decision making at every level. What you'll be doing: Design, build, and maintain scalable data pipelines and ETL processes to support analytics and operations. Develop and optimise data models and storage solutions for performance and reliability. Ensure data quality, integrity, and security throughout the data lifecycle. Collaborate with More ❯

have both strength and scale to capitalise on the key themes shaping the market, through either public markets or alternative asset classes. About the Department As an Engineer (Data Platforms), you will be a member of the Development Chapter within the Asset Management Tribe, reporting to the Development Chapter Lead and working as part of a cross-functional … you visibility and the chance to make a tangible impact on decision-making and operational efficiency. About the Role This role is focused on building and maintaining the data platforms and pipelines that underpin our investment data products. These products are central to many of the initiatives within the Tribe and are used across the organisation … to support portfolio management, analytics, and regulatory reporting. As such, we are looking for engineers who can hit the ground running - with the ability to quickly understand investment data, navigate complex data flows, and deliver robust, scalable solutions. You will work closely with data analysts, investment teams, and other engineers to ensure the quality More ❯

Data Engineering AnalystHybrid – Ipswich (3 days onsite, Tuesday & Wednesday mandatory) Salary: £35,000–£50,000 (depending on experience) REED Technology are working with a client who have created two positions at different levels to be a key part of an exciting transition: the organisation is moving its data warehouse from Netezza to Snowflake , and building an … in-house data engineering capability for the first time. If you want to help shape a modern data platform, support critical migrations, and work closely with BI and Analytics teams to deliver high-quality data products, this role offers significant technical growth and hands-on experience. Responsibilities Supporting the migration of data … engineering, ETL/ELT development, or data warehousing. Strong SQL skills and experience with Snowflake, Netezza, or similar DW platforms. Understanding of data modelling and pipeline design. Ability to articulate technical concepts clearly. A collaborative approach—this team values people who work with others, not in isolation. Desirable Experience with cloud platforms (AWS, Azure, GCP More ❯

Data Engineering Analyst Hybrid - Ipswich (3 days onsite, Tuesday & Wednesday mandatory) Salary: £35,000-£50,000 (depending on experience) REED Technology are working with a client who have created two positions at different levels to be a key part of an exciting transition: the organisation is moving its data warehouse from Netezza to Snowflake , and building … an in-house data engineering capability for the first time. If you want to help shape a modern data platform, support critical migrations, and work closely with BI and Analytics teams to deliver high-quality data products, this role offers significant technical growth and hands-on experience. Responsibilities Supporting the migration of data … engineering, ETL/ELT development, or data warehousing. Strong SQL skills and experience with Snowflake, Netezza, or similar DW platforms. Understanding of data modelling and pipeline design. Ability to articulate technical concepts clearly. A collaborative approach-this team values people who work with others, not in isolation. Desirable Experience with cloud platforms (AWS, Azure, GCP More ❯

Data Engineer Role Opportunity for a Data Engineer to join a scaling startup in the Education sector Salary up to £70k , depending on experience Fully remote role (Europe-based) Tech stack: Python, GCP, PostgreSQL, MongoDB, ML If you wish to keep your CV/data private, feel free to WhatsApp your details/CV … powered platform that helps university students prepare for exams and supports children in improving their reading and writing skills. As we continue to scale, we're expanding our data engineering capability to strengthen our AI-driven learning ecosystem. WHAT YOU WILL BE DOING: Design, build, and maintain ETL/ELT data pipelines and integrations to support … platform and analytics needs. Develop scalable, reliable data architectures using GCP, MongoDB, PostgreSQL, and Python. Collaborate with cross-functional teams to define and implement data models and storage strategies. DATA ENGINEER - ESSENTIAL SKILLS: Proven experience as a Data Engineer, Data Developer, or similar role. Strong experience with GCP (e.g. BigQuery More ❯

Role OverviewAs a Data Engineer, you’ll work closely with stakeholders to develop, test, and maintain data products and solutions that align with their requirements. You’ll be helping to develop a cloud-based data and analytics platform on Azure and help end-users integrate their workflows between this and other platforms, including AWS … based solutions. You’ll combine data from different sources, and you’ll build data pipelines to transform and organise raw data into formats that can be easily used for business intelligence as well as deep dive analysis and modelling. Interviews will be face to face in the London office. We are open to discussing … see the benefits we offer at Nest.For more information about our recruitment process click hereDirectorate/Department OverviewNest is at the beginning of a transformation to use its data and insight to drive decision making, and Data, Analytics and Customer Insight is leading how we do it: putting our customers at the heart of our work More ❯

Role Title: Data Integration & Risk Management Specialist Location: London Duration: 28/02/2026 Days on site: 2-3 Rate : 434 £/day on umbrella MUST BE PAYE THROUGH UMBRELLA Role Description: We are seeking a highly skilled Data Integration & Risk Management Specialist to support our BCBS239 compliance program and enhance our enterprise data ecosystem. The ideal candidate will have strong expertise in data engineering, risk data aggregation, and reporting frameworks, with hands-on experience in Teradata, Informatica, and risk modeling tools. Key Responsibilities Data Platform Management: o Utilize Teradata Vantage for data warehousing and advanced analytics. o Optimize queries and data structures for efficient risk data aggregation. Data Integration & ETL Development: o Design, develop, and optimize ETL workflows using Informatica PowerCenter and related tools. o Manage large-scale data integration projects across multiple platforms, ensuring high performance and scalability. o Perform data quality checks and implement controls for risk-related datasets. More ❯

Data Software Engineer Role Opportunity for a Data Software Engineer to join a scaling startup in the AI and Healthcare technology sector Salary up to £110k + Equity Paddington-based (4-5 days per week in-office) Tech environment: Python | Spark | Databricks | AWS | PostgreSQL | Scala | Data Pipelines | AI Infrastructure If you wish to keep … your CV/data private, feel free to WhatsApp your details/CV to me, Dan - WHO WE ARE: We're a fast-growing AI-driven technology company on a mission to transform how data and automation power smarter care and decision-making. Our intelligent platform leverages advanced AI, automation, and scalable data systems … to unlock powerful insights and efficiency across the healthcare ecosystem. Backed by strong Series A funding, we're expanding our technical team and looking for a Data Software Engineer to help shape and scale our AI infrastructure. WHAT YOU WILL BE DOING: Lead the development of an AI-driven data orchestration platform used to power large More ❯

Your New Role We're looking for an experienced Principal Data Engineer to lead the development and evolution of a modern enterprise data platform. This strategic, hands-on position involves architecting and optimising scalable data pipelines to support advanced analytics, AI/ML initiatives, and actionable insights across the organisation.You'll take full ownership … Acting as the technical authority, you'll define best practices, establish governance frameworks, and mentor engineers and analysts to maximise platform value.This is an opportunity to shape the data landscape and deliver solutions that empower decision-making and innovation. Key Responsibilities Platform Leadership: Design, implement, and manage Snowflake as the … enterprise data hub, ensuring scalability, security, and performance. Data Architecture & Strategy: Define frameworks for ingestion, replication, storage, and transformation across diverse data sources. Pipeline Development: Build efficient ELT pipelines using tools such as DBT and Python, integrating operational, financial, and network data. Performance Optimisation: Configure Snowflake warehouses and partitioning strategies for cost efficiency More ❯

support flexible working – click here for more information on flexible working options Flexible Working Options Hybrid Working, Job Share Job Description Summary . Job Description JOB TITLE: Senior Data Operations Specialist SALARY: £70,929 - £78,810 LOCATION: Bristol HOURS: Full-time WORKING PATTERN: Our work style is hybrid, which involves spending at least two days per week, or … this opportunity... Join the Personalised Experiences and Communications (PEC) Platform, a cornerstone of our Consumer Relationships Business Unit, where we're redefining how Lloyds Banking Group delivers personalised, data-driven customer experiences at scale. As a Senior Data Operations Specialist, you'll be embedded within a cross-functional product engineering team during a transformative phase of … and operations. Participate in design and implementation reviews, promoting best practices and continuous improvement. Read, understand, and write code in Python, Java, Spark, or Terraform to support automation and pipeline enhancements. Conduct holistic problem-solving across the stack—from data pipelines to CI/CD and DAG orchestration. Perform Level 1 triage for customer-impacting issues and More ❯

Claim Group My Claim Group is a fast-growing UK-based company helping customers navigate complex financial and insurance claims. Our success is built on trust, innovation, and data-driven decision-making - and now we're looking to expand our analytics team to support smarter, faster business outcomes. We are seeking a Senior Data Engineer to … join their high-performing data team and help shape the future of our data ecosystem. You will play a key role in designing, building, and scaling robust data platforms that support analytics, reporting, and advanced insight across the organisation. Key Responsibilities of the Role: Design, build, and maintain reliable ETL processes and end-to … end data pipelines. Develop and manage data warehousing solutions to support analytics and reporting needs. Work with both structured and unstructured data, ensuring high-quality data management practices. Design scalable, high-performance data models that enable efficient data use across the business. Build and transform datasets for downstream More ❯

Proud to be partnering with a leading national organisation looking for a fully remote Head of Data to join on a permanent basis to be a key part of the digital transformation. This is a fully remote role paying up to £90,000 base + benefits. Purpose This role will be responsible for leading, aligning, and delivering a … data strategy, building on existing work, through the design, implementation, and ongoing development of our data and business intelligence platforms, whilst ensuring compliance and security. This role will lead the Data department, interconnected with Technology, Security, Solutions Architecture, Change and … the Operational Excellence department to identify, prioritise and deliver continual improvement. This role will lead the development and maintenance of the data warehouse, master data pipeline, utilising Microsoft Fabric and associated toolset. They will lead the development through all phases of the delivery lifecycle, from requirements capture, high level design, low level design, development They More ❯

We are looking for a skilled data engineer to help build and maintain our data infrastructure. The ideal candidate will have expertise working with cloud data platforms and architecting robust and scalable data pipelines. This role is for contractors Tasks As a Senior Data Engineer, you will work cross-functionally … to intake, process, transform, and store large amounts of data from various sources. You will build and maintain the core data infrastructure that powers the organisation's advanced analytics, machine learning applications, and data-driven decision-making. This role requires strong technical skills, problem-solving abilities, and excellent communication with stakeholders at all levels. … Requirements 10+ years of experience with building and maintaining data warehouses, data lakes, and other large-scale data storage systems Knowledge of ETL processes and data transformation techniques Ability to design and develop data aggregation and integration processes across different systems and environments Understanding of data architecture patterns More ❯