151 to 175 of 402 Data Pipeline Jobs in England

Manchester, United Kingdom Posted on 10/01/2025 We provide end-to-end IT solutions and services including Applications services, Data & Analytics services, AI/ML Technologies and Professional services in the UK and EU market. Job Description Job Summary The Azure Data Architecture Lead will provide architectural leadership in designing, implementing, and optimizing … large-scale data solutions using Azure Data Factory and related Azure services. This role is responsible for driving enterprise-scale data initiatives, ensuring that solutions are robust, scalable, and aligned with industry best practices. The position is key in shaping data strategy, enforcing governance, and fostering innovation to support organizational objectives. Key … Responsibilities Architect end-to-end data integration and transformation solutions using Azure Data Factory, ensuring scalability, reliability, and compliance with enterprise standards. Design and oversee advanced analytics platforms leveraging Azure Databricks, Azure Synapse Analytics, and Azure Machine Learning to enable data-driven decision-making. Define and implement DevOps practices for Azure-based data More ❯

Data Engineer - London/Nationwide (Hybrid) - SC Eligible - £60,000 A leading digital consultancy is looking to hire a Data Engineer to support the expansion of its Data & AI team. This is an excellent opportunity for someone with up to 3 years of experience (or a strong academic background) to gain hands-on exposure … to cloud technologies and grow in a collaborative, fast-paced environment. What You'll Be Doing: Building and maintaining scalable data pipelines across Azure and GCP Working with tools such as Azure Data Factory, Snowflake, BigQuery, and Cloud Functions Supporting data warehousing and integration across diverse industry projects Collaborating with data professionals … and contributing to cross-functional delivery teams Assisting in reporting and data visualisation (Power BI/Looker - nice to have) What We're Looking For: 0-3 years of experience in a data engineering or similar role (graduates welcome) Proficiency in SQL and Python Exposure to any of the following: Azure Data Factory GCP More ❯

Data Engineer - Azure Specialist London - 5 Days a Week £80,000 - £100,000 + 10% Bonus Company Our client is a well-established insurance group , recognised for its commitment to innovation and technology-driven transformation. They are seeking a skilled Data Engineer (Azure Specialist) to join their high-performing technology team. This is a fantastic opportunity … to work on mission-critical projects with visibility across the entire business, while shaping the group's data strategy and infrastructure for the future. Responsibilities Design, develop, and own robust, high-performance data pipelines and APIs. Build scalable data infrastructure using Azure technologies, SQL, and dbt. Assemble and manage large, complex datasets to meet … business requirements. Define and document cloud solution architectures, technical designs, and testing processes. Take ownership of data quality and drive opportunities for continuous improvement. Partner with stakeholders across the organisation to support business intelligence and data science initiatives. Troubleshoot and resolve data management issues while contributing to architectural decision-making. Experience Needed Bachelor's More ❯

Excellent opportunity for a veteran Data Engineer ready to take the lead on large-scale, data-driven projects. This is an exciting opportunity to step into a senior role where youll design and deliver innovative solutions, mentor teams, and shape the future of data engineering within a forward-thinking environment. What youll be doing … As a Lead Data Engineer, youll play a hands-on role in designing and developing data processing and persistence software components that handle data at scale. Youll work within agile teams, leading engineers, driving best practice, and ensuring robust, high-quality solutions that meet user needs. Your responsibilities will include: Designing, coding, testing, and … troubleshooting complex data components and subsystems. Leading development practices, ensuring performance, security, and operational readiness. Mentoring and coaching team members, helping them to grow and deliver to a high standard. Working closely with architects on data pipelines (ETL/ELT) and distributed data frameworks. Driving technical proposals, advising on effort estimates, and contributing to More ❯

BI Analyst Location: Bristol (Hybrid 2/3 days a week onsite) Salary: £35,000 - £45,000 DOE Role Profile We are seeking a strategic and commercially-minded Data Analyst to lead the data strategy, elevate our reporting capabilities, and deliver impactful insights that influence executive decision-making and drive performance across the organisation. This role … sits at the heart of our transformation journey - reporting directly to the Chief Executive Officer and working closely with C-suite leaders to ensure data is a key driver of operational and strategic success. Key Responsibilities Partner with the CEO, CFO, Head of Risk, Group Sales Director , providing strategic data and insight to support their decision … making. Design, build, and deliver group-wide reporting programmes , primarily using Tableau , aligned with leadership priorities and business strategy. Ensure data quality and governance , maintaining the accuracy, validation, and trustworthiness of insights used across the business. Provide executive-level analytics, including: Forecasting support to the CFO Strategic planning and hypothesis testing with the CPO Bespoke data More ❯

Senior Data Engineer - London (Hybrid 3 to 4 days) Type: Full-time Location: London, UK Help Build the Future of Data in Financial Services A leading financial services client is growing its data engineering team and is looking for experienced engineers who are passionate about building scalable, high-quality data systems. This … that values clean code, continuous learning, and strong engineering principles. What You'll Be Doing You'll be a key contributor to the development of a next-generation data platform, with responsibilities including: Designing and implementing scalable data pipelines using Python and Apache Spark Building and orchestrating workflows using AWS services such as Glue , Lambda , S3 … CI/CD , version control , automated testing , and modular design Supporting the development of a lakehouse architecture using Apache Iceberg Collaborating with product and business teams to deliver data-driven solutions Embedding observability and quality checks into data workflows Participating in code reviews, pair programming, and architectural discussions Gaining domain knowledge in financial data More ❯

Senior Data Engineer - London (Hybrid 3 to 4 days) Type: Full-time Location: London, UK Help Build the Future of Data in Financial Services A leading financial services client is growing its data engineering team and is looking for experienced engineers who are passionate about building scalable, high-quality data systems. This … that values clean code, continuous learning, and strong engineering principles. What You'll Be Doing You'll be a key contributor to the development of a next-generation data platform, with responsibilities including: Designing and implementing scalable data pipelines using Python and Apache Spark Building and orchestrating workflows using AWS services such as Glue , Lambda , S3 … CI/CD , version control , automated testing , and modular design Supporting the development of a lakehouse architecture using Apache Iceberg Collaborating with product and business teams to deliver data-driven solutions Embedding observability and quality checks into data workflows Participating in code reviews, pair programming, and architectural discussions Gaining domain knowledge in financial data More ❯

Databricks Data Engineer - Newcastle (hybrid) - £450/pd Please note - this role will require you to attend the Newcastle-upon-tyne based office 2-3 days per week. To be eligible for this role you must have the unrestricted right to work in the UK - this organisation is not able to offer sponsorship. A leading construction organisation is … seeking an experienced Azure Databricks Data Engineer to support the ongoing development and optimisation of their cloud data platform. Key Responsibilities: Enhance and scale the existing Azure Databricks environment. Collaborate with data architects and analysts to deliver robust data pipelines. Implement best practices for data engineering, performance tuning, and security. … Work closely with stakeholders to understand data requirements and deliver solutions. Required experience: Strong hands-on experience with Azure Databricks and Spark. Solid understanding of Azure Data Lake, Data Factory, and Synapse. Proven track record in building scalable data pipelines and ETL processes. Experience working in hybrid environments and cross-functional teams. More ❯

Data Engineer Snowflake 3 months + Remote (UK located) £400 per day Outside IR35 This is an excellent opportunity for someone keen to get involved in the early stages of strategic data projects and deliver robust data solutions. You must have experience working with Snowflake for this role. The company is a well-known … into a customer-centric, digitally driven business. Now is the perfect time to join them on this exciting journey as they look to drive forward innovation through meaningful data insights. As a Data Engineer you will join a newly established team as the embark on new business improvement projects. You will build scalable and reliable data pipelines to power analytics, reporting, and strategic decision making. As a Data Engineer you will have the opportunity to utilise cutting edge data tools to design and implement robust data solutions. Required Skills/Experience: At least 2 years' experience working with Snowflake At least 2 years experience working with Azure, AWS More ❯

Job Title: Data Engineer Apprentice Location: Crawley, UK Contract: Apprenticeship Are you passionate about data and looking to launch your career in data engineering? This apprenticeship offers the opportunity to gain hands-on experience with leading cloud technologies while being mentored by experienced professionals in a global technology environment. The Role As a Data Engineer Apprentice, you'll play a key role in designing and maintaining data pipelines that make business insights possible. You'll work with a range of Microsoft Azure tools including Data Factory, Databricks, and SQL Server, as well as develop dashboards and KPIs using Power BI. Day-to-day, you'll: Support the design … build, and maintenance of data pipelines. Assist with data ingestion, transformation, and storage in Azure SQL Database and other cloud solutions. Collaborate with senior team members to translate business needs into technical solutions. Document workflows, processes, and best practices. Troubleshoot issues and suggest improvements. Take ownership of your learning journey while contributing to real projects. What More ❯

Job Title: Data Engineer Apprentice Location: Crawley, UK Contract: Apprenticeship Are you passionate about data and looking to launch your career in data engineering? This apprenticeship offers the opportunity to gain hands-on experience with leading cloud technologies while being mentored by experienced professionals in a global technology environment. The Role As a Data Engineer Apprentice, you'll play a key role in designing and maintaining data pipelines that make business insights possible. You'll work with a range of Microsoft Azure tools including Data Factory, Databricks, and SQL Server, as well as develop dashboards and KPIs using Power BI. Day-to-day, you'll: Support the design … build, and maintenance of data pipelines. Assist with data ingestion, transformation, and storage in Azure SQL Database and other cloud solutions. Collaborate with senior team members to translate business needs into technical solutions. Document workflows, processes, and best practices. Troubleshoot issues and suggest improvements. Take ownership of your learning journey while contributing to real projects. What More ❯

Job Title: Data Engineer Apprentice Location: Crawley, UK Contract: Apprenticeship Are you passionate about data and looking to launch your career in data engineering? This apprenticeship offers the opportunity to gain hands-on experience with leading cloud technologies while being mentored by experienced professionals in a global technology environment. The Role As a Data Engineer Apprentice, you'll play a key role in designing and maintaining data pipelines that make business insights possible. You'll work with a range of Microsoft Azure tools including Data Factory, Databricks, and SQL Server, as well as develop dashboards and KPIs using Power BI. Day-to-day, you'll: Support the design … build, and maintenance of data pipelines. Assist with data ingestion, transformation, and storage in Azure SQL Database and other cloud solutions. Collaborate with senior team members to translate business needs into technical solutions. Document workflows, processes, and best practices. Troubleshoot issues and suggest improvements. Take ownership of your learning journey while contributing to real projects. What More ❯

Senior Data Engineer - £65,000-£70,000 + Excellent Benefits Location: Leeds (Hybrid, 2 days on-site) Are you ready to take on large-scale data engineering challenges within a business that's investing heavily in cloud transformation, AI, and advanced analytics? This is a unique opportunity to join a rapidly growing, tech-driven organisation that … s modernising its entire data landscape and gearing up for a massive change and innovation portfolio in 2026. You'll play a pivotal role in designing and building scalable data platforms on AWS, supporting everything from cloud migrations and Salesforce integrations to AI-ready data pipelines and petabyte-scale projects. What You'll Be … Doing Designing and developing modern data architectures to support advanced analytics and automation. Leading end-to-end data migration projects from on-prem SQL Server to AWS cloud environments. Building and optimising data pipelines using Python, Lambda, and SQL, ensuring performance, quality, and scalability. Collaborating across DevOps, analytics, and engineering teams to deliver cloud More ❯

on-site 6 Months + UMBRELLA only- Inside IR35 Key Responsibilities Design, develop, and maintain ETL/ELT pipelines using Airflow for orchestration and scheduling. Build and manage data transformation workflows in DBT running on Databricks . Optimize data models in Delta Lake for performance, scalability, and cost efficiency. Collaborate with analytics, BI, and data … governance standards. Manage and monitor Databricks clusters & SQL Warehouses to support workloads. Contribute to CI/CD practices for data pipelines (version control, testing, deployments). Troubleshoot pipeline failures, performance bottlenecks, and scaling challenges. Document workflows, transformations, and data models for knowledge sharing. Required Skills & Qualifications 3-6 years of experience as a Data … job clusters, SQL Warehouses). Strong SQL skills and understanding of data modeling (Kimball, Data Vault, or similar) . Proficiency in Python for scripting and pipeline development. Experience with CI/CD tools (e.g., GitHub Actions, GitLab CI, Azure DevOps). Familiarity with cloud platforms (AWS, Azure, or GCP). Strong problem-solving skills and More ❯

Data Management Developer Lead page is loaded Data Management Developer Leadlocations: Londontime type: Full timeposted on: Posted Todayjob requisition id: RThe purpose of this role is to oversee the development of our database marketing solutions, using database technologies such as Microsoft SQL Server/Azure, Amazon Redshift, Google BigQuery. The role will be involved in design … the company has put people at the heart of its approach to digital business transformation. As the only integrated experience consultancy in the world with a heritage in data science and business performance, Merkle delivers holistic, end-to-end experiences that drive growth, engagement, and loyalty. Merkle's expertise has earned recognition as a "Leader" by top industry … analyst firms, in categories such as digital transformation and commerce, experience design, engineering and technology integration, digital marketing, data science, CRM and loyalty, and customer data management. With more than 16,000 employees, Merkle operates in 30+ countries throughout the Americas, EMEA, and APAC. For more information, visit . Role Purpose The purpose of this role More ❯

Data Engineer - Manchester (Hybrid) - £55,000 + Bonus The Opportunity: Lorien Global is proud to partner with a leading North West brand that's redefining its global strategy through data. Following a major migration project, the business is now investing heavily in cutting-edge data solutions to power smarter tech, innovative products, and world-class customer … experiences. The Role: We're on the lookout for a Data Engineer who's passionate about building scalable, cloud-native data platforms. You'll be a key player in a growing team, helping to shape the future of data infrastructure using AWS, PySpark, Iceberg, and more. From designing high-performance pipelines to supporting a … full-scale migration from SQL Server to AWS, this role offers the chance to work on real-time data solutions that drive impact across the business. You'll work with analysts, data scientists, and engineers to unlock automation, enable BI, and increase advanced analytics. What You'll Be Doing: Designing and building robust data More ❯

consumer finance, asset management, and leasing. MUFG offers stability in an ever-changing market, providing services to high-profile clients worldwide.You will be part of AMD-Architecture, Middleware, Data Management & Enterprise Service department. The AMD pillar of Technology has a department focused on Data Engineering & Business Intelligence (DEBI), who are responsible for designing & developing enterprise standard … technologies that support data integration, processing, warehousing, mining & reporting.Main Purpose of the Role : To architect, design and develop EDM and data solutions as part of multiple initiatives across International MUFG entities (MUFG Securities, MUFG Bank and MUMSS) To drive innovation and automation so that we can deliver the highest quality service to our business partners. To … in accordance with business requirement and architectural guidelines. To mentor and lead other EDM resources that augment the team.Key Responsibilities:In this role, you will be responsible for data across MUFG's banking arm and securities business under a dual-hat arrangement. Under this arrangement, you will act and make decisions on behalf of both the bank and More ❯

Contract Data Engineer - Azure, Databricks, SQL, Python 6 Weeks Outside IR35 £500-£600 per day Remote (occasional travel to Manchester or Glasgow) My client is delivering a high-impact data migration programme and is looking for a Data Engineer to join on a 6-week contract. This role will suit someone who thrives in … fast-moving delivery environments and has hands-on experience with modern data engineering tools. The Role Join a focused delivery team to execute a business-critical data migration Build, optimise and maintain data pipelines using Azure and Databricks Translate business requirements into technical solutions that ensure data accuracy, integrity and security Deliver … a strong focus on clean, reliable code and documentation Skills & Experience Required Advanced SQL skills for querying, modelling and migration tasks Ability to write production-quality Python for data engineering and automation Proven experience of data migration projects, ideally at enterprise scale Experience with Databricks and the wider Azure stack (Data Lake, Synapse, ADF More ❯

Contract Data Engineer - Azure, Databricks, SQL, Python 6 Weeks | Outside IR35 | £500-£600 per day | Remote (occasional travel to Manchester or Glasgow) My client is delivering a high-impact data migration programme and is looking for a Data Engineer to join on a 6-week contract. This role will suit someone who thrives in … fast-moving delivery environments and has hands-on experience with modern data engineering tools. The Role Join a focused delivery team to execute a business-critical data migration Build, optimise and maintain data pipelines using Azure and Databricks Translate business requirements into technical solutions that ensure data accuracy, integrity and security Deliver … a strong focus on clean, reliable code and documentation Skills & Experience Required Advanced SQL skills for querying, modelling and migration tasks Ability to write production-quality Python for data engineering and automation Proven experience of data migration projects, ideally at enterprise scale Experience with Databricks and the wider Azure stack (Data Lake, Synapse, ADF More ❯

Regulatory Data Engineer - Contract (3 Months) Location: London (Onsite, 5 days per week) Rate: £850 per day (Inside IR35) We are seeking a highly experienced Regulatory Data Engineer to join a trading desk team on a three-month engagement. This is an excellent opportunity to work within a front-office environment and contribute to critical data initiatives. Key Responsibilities Design, develop, and implement data pipelines to support regulatory and trading desk requirements. Collaborate with traders and business stakeholders to translate requirements into technical solutions. Ensure accuracy, integrity, and quality of data across systems. Contribute hands-on to the data infrastructure, maintaining high coding and engineering standards. Work closely … with engineers and data specialists to improve reporting and analytics capabilities. Essential Skills and Experience Minimum 5 years' experience in global markets, hedge funds, or investment banking. Advanced expertise in Python with strong hands-on development experience. Proficiency in data tools (e.g., ETL tools, data visualisation tools). Proven data engineering skills More ❯

Senior Data Engineer (AI & MLOps) - Software - Newcastle/Hybrid or Remote Day rate: £300 - £500 (Inside IR35) Duration: 6 months Start: ASAP My new client is looking for a Senior Data Engineer with expertise in AI, MLOps, and AWS architecture to design and deliver production-grade machine learning pipelines. The ideal candidate will be passionate about … Engineer, MLOps Engineer, or similar role. Strong background in data structures, algorithms, and software engineering principles. Advanced proficiency in Python for data wrangling, pipeline automation, and ML workflows. Expertise in AWS services, including databases (RDS, DynamoDB, Redshift) and machine learning/AI (SageMaker, AI/ML frameworks). Hands-on experience with ML … pipeline orchestration, CI/CD, and deployment automation. Deep understanding of ML Ops practices, including monitoring, scaling, and retraining strategies. Familiarity with AIOps concepts and tools for operational automation. Preferred Skills Experience with data science and machine learning model development. Knowledge of containerization (Docker, Kubernetes, EKS). Exposure to infrastructure-as-code (Terraform, CloudFormation). Strong problem More ❯

Senior Data Engineer UK Remote, £500/day, Outside IR35 A Senior Data Engineer is required by fast-growing HealthTech/Tech for Good start-up. This is a 3-month contract initially with scope for extension. In this role, you will design, build, and maintain robust data pipelines using streaming and batch processing … technologies and help to lead the modernization of their data lakehouse. You will implement and manage secure cloud infrastructure using AWS, championing a culture of data quality, security, and technical excellence. Key Responsibilities: Architect and maintain data pipelines (streaming & batch) Build and manage scalable data storage solutions (data lakehouse) Design … and maintain secure AWS cloud infrastructure Implement and manage CI/CD and DevOps pipelines Champion data quality, security, and best practices Collaborate with cross-functional teams Implement and manage MLOps capabilities Essential Skills: Advanced Python programming skills Expertise in data engineering tools and frameworks (Apache Flink) Hands-on AWS experience (Serverless, CloudFormation, CDK) Strong understanding More ❯

of specialists ensuring they are motivated, collaborating and working well. Essential knowledge and skills Proven experience in delivering complex, large-scale technical projects, with a strong focus on data platforms, analytics, or data-driven solutions. Deep understanding of data technologies, including data pipelines, ETL processes, data warehousing, and cloud-based … data platforms (e.g., AWS, Azure). Strong technical background, with the ability to engage with engineers, architects, and data scientists to ensure alignment on technical solutions. Demonstrated ability to manage cross-functional teams, including engineering, product, and data teams, to deliver high-quality outcomes. Strong communication and influencing skills to engage senior stakeholders Experience More ❯

Your new company A Principal Data Engineer/Architect is required on a permanent basis for a forward-thinking organisation at the heart of Leeds. The Data Services team are on a mission to unlock the value of data by delivering high-quality, secure, and accessible data services. With a focus on … modern cloud-based technologies and strong partnerships, they help colleagues navigate the complexities of a data-driven world. Your new role As a Principal Data Engineer, you will be instrumental in shaping the organisations strategic cloud data platform. You'll lead the design and implementation of scalable data pipelines, drive innovation in … data-centric products, and champion automation and predictive analytics. This is a senior technical leadership role where you'll establish best practices, ensure compliance, and deliver smart, customer-focused solutions. What you'll need to succeed You'll bring extensive experience in data engineering within Azure environments, with a strong track record in modernisation and large More ❯

Job Title: Business Intelligence Data Analyst Location: Middlesex 12 Month Contract Salary up to £65k + Benefits Job Type: Full-time Department: Sales Operations/Business Analytics Job Overview Are you passionate about transforming data into strategic insights that drive business success? We are looking for a Business Intelligence Data Analyst to support our … Global Virtual Sales (GVS) team across EMEA. This role is essential to enabling data-driven decision-making that maximizes sales effectiveness and enhances operational efficiency. As part of a dynamic and collaborative environment, you'll work closely with cross-functional teams-including Sales, Finance, Operations, Strategy & Planning, and IT-to analyze complex datasets and deliver actionable insights. Your … work will directly influence how we execute our Sales Go-to-Market strategy and shape the future of virtual selling Key Responsibilities Enable and support data-driven decision-making across sales operations to enhance productivity and revenue growth. Conduct advanced analysis of complex datasets spanning sales, operations, marketing, and planning. Design and develop reports and dashboards to track More ❯