551 to 575 of 1,153 Data Pipeline Jobs in the UK excluding London

Senior Data Engineer - Hybrid - Contract - Outside IR35 Our client is expanding their core data engineering function and hiring two Senior Data Engineers to strengthen a modern, scalable data platform. Both will be Databricks specialists, with one role focused on deep technical delivery and the other combining hands-on engineering with team leadership … Tables (DLT) pipelines across Bronze → Silver → Gold layers, ensuring quality, scalability, and reliability. Develop and optimise Spark (PySpark) jobs for large-scale distributed processing. Design and implement streaming data pipelines with Kafka/MSK, applying best practices for late event handling and throughput. Use Terraform and CI/CD pipelines (GitHub Actions or similar) to manage infrastructure and … deployments. (For leadership-focused role) Mentor and upskill engineers, define coding standards, and embed engineering excellence across the team. What's Expected Proven experience delivering end-to-end data pipelines in Databricks and Spark environments. Strong understanding of data modelling, schema evolution, and data contract management. Hands-on experience with Kafka, streaming architectures, and More ❯

Our client, a global asset management firm is seeking to hire an ESG Data Scientist to join their Sustainable Research team. This role will play a key part in developing the tools, infrastructure and analytics that drive the firm’s ESG and sustainability capabilities. Key Responsibilities: Build analytics tools and machine learning models to assess ESG and sustainability … characteristics of companies Design and implement scalable, high-performance sustainability data systems and infrastructure Develop, maintain and enhance dashboards and tools for investment teams Build data pipelines integrating multiple sources into a unified, high-quality and trusted data environment Collaborate with investment and technology teams to prototype new metrics and deploy AI-driven insights … that enhance ESG integration and sustainability performance measurements Monitor and improve data quality, investigating and resolving data issues Engage with internal stakeholders across the firm to understand business needs and translate them into technical solutions Contribute to vendor evaluations and development of data governance standards Candidate Requirements: Minimum 4 years of experience in a More ❯

Our client, a global asset management firm is seeking to hire an ESG Data Scientist to join their Sustainable Research team. This role will play a key part in developing the tools, infrastructure and analytics that drive the firm’s ESG and sustainability capabilities. Key Responsibilities: Build analytics tools and machine learning models to assess ESG and sustainability … characteristics of companies Design and implement scalable, high-performance sustainability data systems and infrastructure Develop, maintain and enhance dashboards and tools for investment teams Build data pipelines integrating multiple sources into a unified, high-quality and trusted data environment Collaborate with investment and technology teams to prototype new metrics and deploy AI-driven insights … that enhance ESG integration and sustainability performance measurements Monitor and improve data quality, investigating and resolving data issues Engage with internal stakeholders across the firm to understand business needs and translate them into technical solutions Contribute to vendor evaluations and development of data governance standards Candidate Requirements: Minimum 4 years of experience in a More ❯

Our client, a global asset management firm is seeking to hire an ESG Data Scientist to join their Sustainable Research team. This role will play a key part in developing the tools, infrastructure and analytics that drive the firm’s ESG and sustainability capabilities. Key Responsibilities: Build analytics tools and machine learning models to assess ESG and sustainability … characteristics of companies Design and implement scalable, high-performance sustainability data systems and infrastructure Develop, maintain and enhance dashboards and tools for investment teams Build data pipelines integrating multiple sources into a unified, high-quality and trusted data environment Collaborate with investment and technology teams to prototype new metrics and deploy AI-driven insights … that enhance ESG integration and sustainability performance measurements Monitor and improve data quality, investigating and resolving data issues Engage with internal stakeholders across the firm to understand business needs and translate them into technical solutions Contribute to vendor evaluations and development of data governance standards Candidate Requirements: Minimum 4 years of experience in a More ❯

A large global organisation are looking to hire a Senior Data Analyst to lead end-to-end data workflows from requirement to delivery including data product creation and secure data transfer from Google Cloud Platform to PostgreSQL. This will be a full time permanent position. The role will be on a Hybrid … automate those views, ensuring reliable execution on daily, weekly, or customized schedules. Execute Cross‐Platform ETL with AWS Glue Develop, deploy, and maintain AWS Glue jobs to extract data from GCP (such as BigQuery or GCS) and load it into PostgreSQL. Set up secure connectivity, schedule jobs via cron or trigger mechanisms, and ensure data … pipelines are reliable and idempotent. Monitor, Troubleshoot & Resolve Incidents Continuously oversee ETL workflows in Airflow and AWS Glue, proactively responding to alerts and errors. Conduct root cause analysis for pipeline failures—whether due to schema mismatches or performance bottlenecks—and apply robust fixes. Document resolutions to strengthen system resilience. Design, Build, & Govern Data Products Architect, construct, and More ❯

A large global organisation are looking to hire a Senior Data Analyst to lead end-to-end data workflows from requirement to delivery including data product creation and secure data transfer from Google Cloud Platform to PostgreSQL. This will be a full time permanent position. The role will be on a Hybrid … automate those views, ensuring reliable execution on daily, weekly, or customized schedules. Execute Cross‐Platform ETL with AWS Glue Develop, deploy, and maintain AWS Glue jobs to extract data from GCP (such as BigQuery or GCS) and load it into PostgreSQL. Set up secure connectivity, schedule jobs via cron or trigger mechanisms, and ensure data … pipelines are reliable and idempotent. Monitor, Troubleshoot & Resolve Incidents Continuously oversee ETL workflows in Airflow and AWS Glue, proactively responding to alerts and errors. Conduct root cause analysis for pipeline failures—whether due to schema mismatches or performance bottlenecks—and apply robust fixes. Document resolutions to strengthen system resilience. Design, Build, & Govern Data Products Architect, construct, and More ❯

A large global organisation are looking to hire a Senior Data Analyst to lead end-to-end data workflows from requirement to delivery including data product creation and secure data transfer from Google Cloud Platform to PostgreSQL. This will be a full time permanent position. The role will be on a Hybrid … automate those views, ensuring reliable execution on daily, weekly, or customized schedules. Execute Cross‐Platform ETL with AWS Glue Develop, deploy, and maintain AWS Glue jobs to extract data from GCP (such as BigQuery or GCS) and load it into PostgreSQL. Set up secure connectivity, schedule jobs via cron or trigger mechanisms, and ensure data … pipelines are reliable and idempotent. Monitor, Troubleshoot & Resolve Incidents Continuously oversee ETL workflows in Airflow and AWS Glue, proactively responding to alerts and errors. Conduct root cause analysis for pipeline failures—whether due to schema mismatches or performance bottlenecks—and apply robust fixes. Document resolutions to strengthen system resilience. Design, Build, & Govern Data Products Architect, construct, and More ❯

As a Business Intelligence Analyst, you will play a key role in driving data-informed decision-making by delivering insightful analyses and reports. You will contribute to gathering, analysing, and visualising data to uncover trends and actionable insights that enhance business operations, customer satisfaction, and overall performance. Working closely with cross-functional teams, you will leverage … and optimise Power BI dashboards and interactive reports tailored to various stakeholders and needs. Automate and streamline recurring reports to improve efficiency and reduce manual effort. Translate complex data into visually intuitive reports that cater to both technical and non-technical audiences. Write and optimise advanced SQL queries to extract, transform, and analyse data. Design and implement data models to support reporting and analysis needs. Ensure data pipelines are efficient, reliable, and scalable. Track and report on Key Performance Indicators (KPIs) across multiple business areas and divisions. Highlight trends and anomalies to seniors and managers, while also analysing and resolving them to ensure a clear understanding of root causes and implementing necessary solutions. Work More ❯

Senior Data Engineer Location: Trowbridge (Hybrid, 2 days in office) Salary: £70,000 Be part of a forward-thinking data and analytics team shaping the future of our client's operations across the UK, Ireland, and North America. This is a fantastic opportunity to lead data infrastructure projects, including a major cloud migration to … Azure, and support strategic analytics initiatives for both B2C and B2B businesses. Key Responsibilities: Manage and optimize cloud-based data infrastructure (Azure). Lead the migration of data environments to Azure. Maintain high-quality, secure data pipelines with a strong focus on governance. Mentor and support a Data Engineer apprentice. Collaborate across … teams to enhance reporting, analytics, and business intelligence solutions. What You Bring: 5+ years of experience in SQL production environments and data engineering. Strong expertise in SQL, T-SQL, Azure, DataBricks, and SSIS. Experience with data integration, modeling, and performance optimization. Problem-solving mindset and a collaborative approach. Desirable: Experience handling data from APIs More ❯

Senior Data Engineer Location: Trowbridge (Hybrid, 2 days in office) Salary: £70,000 Be part of a forward-thinking data and analytics team shaping the future of our client's operations across the UK, Ireland, and North America. This is a fantastic opportunity to lead data infrastructure projects, including a major cloud migration to … Azure, and support strategic analytics initiatives for both B2C and B2B businesses. Key Responsibilities: Manage and optimize cloud-based data infrastructure (Azure). Lead the migration of data environments to Azure. Maintain high-quality, secure data pipelines with a strong focus on governance. Mentor and support a Data Engineer apprentice. Collaborate across … teams to enhance reporting, analytics, and business intelligence solutions. What You Bring: 5+ years of experience in SQL production environments and data engineering. Strong expertise in SQL, T-SQL, Azure, DataBricks, and SSIS. Experience with data integration, modeling, and performance optimization. Problem-solving mindset and a collaborative approach. Desirable: Experience handling data from APIs More ❯

Senior Data Engineer Contract | Edinburgh (2 days onsite) £500/day (Likely Outside IR35) 3 months initially Bright Purple is delighted to be working with an exciting, product-focused consultancy delivering some of the UKs most high-profile and widely used consumer applications . Their client list features some of the biggest names in tech and beyond. Were … seeking an experienced Senior Data Engineer to join their growing Data Practice on a 3-month engagement, helping shape and deliver scalable, cloud-native data solutions for household-name clients. What youll be doing Designing, building and maintaining robust data pipelines Automating and orchestrating workflows (AWS Glue, Azure Data Factory … and optimising modern data architectures (e.g. Databricks, Snowflake) Collaborating with multidisciplinary teams to deliver real business value What were looking for Strong experience with Python, SQL , and pipeline tools such as dbt or Airflow Proven background in data modelling, warehousing, and performance optimisation Hands-on experience with cloud data services (Glue, Lambda, Synapse More ❯

Senior Data Engineer – Contract london (2 days onsite) £500/day (Likely Outside IR35) 3 months initially Bright Purple is delighted to be working with an exciting, product-focused consultancy delivering some of the UK’s most high-profile and widely used consumer applications . Their client list features some of the biggest names in tech and beyond. … We’re seeking an experienced Senior Data Engineer to join their growing Data Practice on a 3-month engagement, helping shape and deliver scalable, cloud-native data solutions for household-name clients. What you’ll be doing Designing, building and maintaining robust data pipelines Automating and orchestrating workflows (AWS Glue, Azure Data … optimising modern data architectures (e.g. Databricks, Snowflake) Collaborating with multidisciplinary teams to deliver real business value What we’re looking for Strong experience with Python, SQL , and pipeline tools such as dbt or Airflow Proven background in data modelling, warehousing, and performance optimisation Hands-on experience with cloud data services (Glue, Lambda, Synapse More ❯

Senior Data Engineer – Contract london (2 days onsite) £500/day (Likely Outside IR35) 3 months initially Bright Purple is delighted to be working with an exciting, product-focused consultancy delivering some of the UK’s most high-profile and widely used consumer applications . Their client list features some of the biggest names in tech and beyond. … We’re seeking an experienced Senior Data Engineer to join their growing Data Practice on a 3-month engagement, helping shape and deliver scalable, cloud-native data solutions for household-name clients. What you’ll be doing Designing, building and maintaining robust data pipelines Automating and orchestrating workflows (AWS Glue, Azure Data … optimising modern data architectures (e.g. Databricks, Snowflake) Collaborating with multidisciplinary teams to deliver real business value What we’re looking for Strong experience with Python, SQL , and pipeline tools such as dbt or Airflow Proven background in data modelling, warehousing, and performance optimisation Hands-on experience with cloud data services (Glue, Lambda, Synapse More ❯

Senior Data Engineer – Contract london (2 days onsite) £500/day (Likely Outside IR35) 3 months initially Bright Purple is delighted to be working with an exciting, product-focused consultancy delivering some of the UK’s most high-profile and widely used consumer applications . Their client list features some of the biggest names in tech and beyond. … We’re seeking an experienced Senior Data Engineer to join their growing Data Practice on a 3-month engagement, helping shape and deliver scalable, cloud-native data solutions for household-name clients. What you’ll be doing Designing, building and maintaining robust data pipelines Automating and orchestrating workflows (AWS Glue, Azure Data … optimising modern data architectures (e.g. Databricks, Snowflake) Collaborating with multidisciplinary teams to deliver real business value What we’re looking for Strong experience with Python, SQL , and pipeline tools such as dbt or Airflow Proven background in data modelling, warehousing, and performance optimisation Hands-on experience with cloud data services (Glue, Lambda, Synapse More ❯

Senior Data Engineer Contract Edinburgh (2 days onsite) £500/day (Likely Outside IR35) 3 months initially Bright Purple is delighted to be working with an exciting, product-focused consultancy delivering some of the UK s most high-profile and widely used consumer applications . Their client list features some of the biggest names in tech and beyond. … We re seeking an experienced Senior Data Engineer to join their growing Data Practice on a 3-month engagement, helping shape and deliver scalable, cloud-native data solutions for household-name clients. What you ll be doing Designing, building and maintaining robust data pipelines Automating and orchestrating workflows (AWS Glue, Azure Data … optimising modern data architectures (e.g. Databricks, Snowflake) Collaborating with multidisciplinary teams to deliver real business value What we re looking for Strong experience with Python, SQL , and pipeline tools such as dbt or Airflow Proven background in data modelling, warehousing, and performance optimisation Hands-on experience with cloud data services (Glue, Lambda, Synapse More ❯

Senior Data Engineer – Contract | Edinburgh (2 days onsite) £500/day (Likely Outside IR35) 3 months initially Bright Purple is delighted to be working with an exciting, product-focused consultancy delivering some of the UK’s most high-profile and widely used consumer applications . Their client list features some of the biggest names in tech and beyond. … We’re seeking an experienced Senior Data Engineer to join their growing Data Practice on a 3-month engagement, helping shape and deliver scalable, cloud-native data solutions for household-name clients. What you’ll be doing Designing, building and maintaining robust data pipelines Automating and orchestrating workflows (AWS Glue, Azure Data … optimising modern data architectures (e.g. Databricks, Snowflake) Collaborating with multidisciplinary teams to deliver real business value What we’re looking for Strong experience with Python, SQL , and pipeline tools such as dbt or Airflow Proven background in data modelling, warehousing, and performance optimisation Hands-on experience with cloud data services (Glue, Lambda, Synapse More ❯

We believe in better. And we make it happen. Better content. Better products. And better careers. Working in Tech, Product or Data at Sky is about building the next and the new. From broadband to broadcast, streaming to mobile, SkyQ to Sky Glass, we never stand still. We optimise and innovate. We turn big ideas into the products … Machine Learning Engineer to advance our personalised recommendation systems by developing efficient, low-latency solutions that serve millions of users globally. The successful candidate will collaborate closely with data scientists, engineers, and product managers to design intelligent content recommendation mechanisms and drive the ongoing advancement of our … Machine Learning Platform. Model Development: Design, train, and optimise machine learning models focused on user personalisation, encompassing recommendation engines, ranking algorithms, user segmentation, and content analysis. Data Pipeline Engineering: Construct and maintain robust and scalable data pipelines for feature engineering and model training utilising both structured and unstructured large-scale datasets. Production Deployment: Deploy and More ❯

We believe in better. And we make it happen. Better content. Better products. And better careers. Working in Tech, Product or Data at Sky is about building the next and the new. From broadband to broadcast, streaming to mobile, SkyQ to Sky Glass, we never stand still. We optimise and innovate. We turn big ideas into the products … Machine Learning Engineer to advance our personalised recommendation systems by developing efficient, low-latency solutions that serve millions of users globally. The successful candidate will collaborate closely with data scientists, engineers, and product managers to design intelligent content recommendation mechanisms and drive the ongoing advancement of our … Machine Learning Platform. Model Development: Design, train, and optimise machine learning models focused on user personalisation, encompassing recommendation engines, ranking algorithms, user segmentation, and content analysis. Data Pipeline Engineering: Construct and maintain robust and scalable data pipelines for feature engineering and model training utilising both structured and unstructured large-scale datasets. Production Deployment: Deploy and More ❯

We believe in better. And we make it happen. Better content. Better products. And better careers. Working in Tech, Product or Data at Sky is about building the next and the new. From broadband to broadcast, streaming to mobile, SkyQ to Sky Glass, we never stand still. We optimise and innovate. We turn big ideas into the products … Machine Learning Engineer to advance our personalised recommendation systems by developing efficient, low-latency solutions that serve millions of users globally. The successful candidate will collaborate closely with data scientists, engineers, and product managers to design intelligent content recommendation mechanisms and drive the ongoing advancement of our … Machine Learning Platform. Model Development: Design, train, and optimise machine learning models focused on user personalisation, encompassing recommendation engines, ranking algorithms, user segmentation, and content analysis. Data Pipeline Engineering: Construct and maintain robust and scalable data pipelines for feature engineering and model training utilising both structured and unstructured large-scale datasets. Production Deployment: Deploy and More ❯

I am recruiting for a Data Engineer to be based in Glasgow 3 days a week, 2 days remote. The role falls inside IR35 so you will need to work through an umbrella company for the duration of the contract. You must have several years of experience developing data pipelines and data warehousing solutions … libraries such as Pandas, NumPy, PySpark, etc. You will also have a number of years hands-on experience with cloud services, especially Databricks, for building and managing scalable data pipelines. ETL process expertise is essential. Proficiency in working with Snowflake or similar cloud-based data warehousing solutions is also essential. Experience in data development … and solutions in highly complex data environments with large data volumes is also required. You will be responsible for collaborating with cross-functional teams to understand data requirements, and design efficient, scalable, and reliable ETL processes using Python and Databricks. You will also develop and deploy ETL jobs that extract data from More ❯

to help the engineering team refactor the ‘research-level’ code into production-ready code. You will be responsible for expanding and optimizing our cloud infrastructure and data pipeline architecture, as well as optimising data flow and collection throughout our custom-built Clinical Management Interface (CMI). You will work closely with the Data … seamless integration with their pipelines and clearly document all your work. Essential requirements: - Minimum 3 years professional software development experience - Expert proficiency in Python in software development and data science/processing - Strong working knowledge of R - Proven experience in refactoring "research-level" code into production-ready software - Proven experience of collaborating effectively in cross-functional team environment More ❯

to help the engineering team refactor the ‘research-level’ code into production-ready code. You will be responsible for expanding and optimizing our cloud infrastructure and data pipeline architecture, as well as optimising data flow and collection throughout our custom-built Clinical Management Interface (CMI). You will work closely with the Data … seamless integration with their pipelines and clearly document all your work. Essential requirements: - Minimum 3 years professional software development experience - Expert proficiency in Python in software development and data science/processing - Strong working knowledge of R - Proven experience in refactoring "research-level" code into production-ready software - Proven experience of collaborating effectively in cross-functional team environment More ❯

Lead Data Engineer Salary: £90,000 - £115,000 Location: West London (3 Days in Office) Using data science at their core, they transform complex property and customer data into insights that enhance targeting, retention, and performance. The Opportunity This flexible, data-driven start-up places Data Science at the heart … of its business, using cutting-edge UK property data to deliver powerful insights for real estate partners and retailers. By leveraging advanced analytics, predictive modelling, and customer segmentation, the company helps clients better understand and engage with their audiences, improving marketing precision, timing, and overall performance. With a strong technical foundation and a collaborative team led by experienced … data leaders, the company is building efficient, automated data pipelines and scalable systems to power intelligent decision-making. This is an exciting opportunity for forward-thinking professionals to join a growing team, shape best practices, and make a real impact in a fast-evolving data and technology environment. The Role and Responsibilities This is More ❯

The Senior Data/BI Engineer will play a pivotal role in developing and optimising data pipelines, ensuring the seamless flow of information across the organisation. The successful candidate will also be heavily involved in supporting our cloud migration project. Client Details Based in Manchester City Centre, we are a leader in our field in the … serve over 35,000 customers with services like electrical testing, asset management, non-destructive testing (NDT), and inspections for infrastructure, manufacturing, and healthcare facilities. Description The Successful Senior Data Engineer will be responsible for but not limited to: · Development and implement data and reporting solutions from our Dynamics, in-house and 3rd party sources, using the … latest Microsoft technologies: Azure Synapse Analytics & Azure Data Factory, Azure Data Lake, Azure SQL Database Support older Microsoft Technologies whilst we are in transition: SSIS and SSRS. Supporting change and migration efforts. Work with other members of the team or directly with business users to understand and document business requirements, Undertake/support the monitoring of More ❯

The Senior Data/BI Engineer will play a pivotal role in developing and optimising data pipelines, ensuring the seamless flow of information across the organisation. The successful candidate will also be heavily involved in supporting our cloud migration project. Client Details Based in Manchester City Centre, we are a leader in our field in the … serve over 35,000 customers with services like electrical testing, asset management, non-destructive testing (NDT), and inspections for infrastructure, manufacturing, and healthcare facilities. Description The Successful Senior Data Engineer will be responsible for but not limited to: · Development and implement data and reporting solutions from our Dynamics, in-house and 3rd party sources, using the … latest Microsoft technologies: Azure Synapse Analytics & Azure Data Factory, Azure Data Lake, Azure SQL Database Support older Microsoft Technologies whilst we are in transition: SSIS and SSRS. Supporting change and migration efforts. Work with other members of the team or directly with business users to understand and document business requirements, Undertake/support the monitoring of More ❯