1 to 25 of 261 Permanent Data Lake Jobs in the UK

At Dufrain , we don't just build data solutions-we shape the future of data-driven decision-making. Do you thrive on solving complex challenges and turning data into powerful insights that drive transformation? We're looking for Data Engineers who have a broad range of data engineering skills, with … a focus on Microsoft Databricks . Experience with Microsoft Fabric or Snowflake is also highly desirable. ABOUT THE ROLE As a Data Engineer, you will play a key role in delivering high-quality data solutions for our clients. This role offers the opportunity to work across a variety of technologies and contribute to the growth of … and engineering practices. Key competencies include: Databricks Platform Expertise : Proven experience designing and delivering data solutions using Databricks on Azure or AWS. Databricks Components : Proficient in Delta Lake, Unity Catalog, MLflow, and other core Databricks tools. Programming & Query Languages : Strong skills in SQL and Apache Spark (Scala or Python). Relational Databases : Experience with on-premises and More ❯

Job summary Are you a data engineer with a passion for public service? Then maybe you're looking for us! We are seeking an experienced and skilled Advanced Data Engineer to be part of a growing team that delivers on our mission of protecting and improving the health and wellbeing of the people of Wales. You … ll be involved in improving our data practices, cataloguing our data, automating our data processing, analytical pipelines and developing our overall data maturity. Working with us you'll collaborate with other technical specialists (such as developers, database administrators, data analysts and data scientists) on the development, implementation, maintenance … do things. Accountabilities Implement enterprise data models, pipelines, and metadata management Develop and maintain data platform components (e.g. data catalogue, data lake, warehouse, ETL frameworks) Follow best practice in data security, compliance, and the ethical use of data in accordance with standards and requirements such as GDPR More ❯

Job summary We are looking for an experienced Senior Data Engineer with strong technical expertise and innovative mindset to drive our team forward. Main duties of the job Data Engineer will be involved in the entire data pipeline from data acquisition to data extraction, ensuring that the data is accurate, reliable, and accessible, and fit-for-purpose for business intelligence, analytics, and data science functions. Data Engineer will be collaborating with cross-functional teams to support the delivery of impactful analytics work for our clients. About us We are the official research support team for the NHS Benchmarking Network, working with over … healthcare organisations across the UK to identify opportunities for service development and quality improvement. We also serve as the primary source of benchmarking data for the NHS England Workforce, Training and Education team, providing in-depth analysis across national mental health services. Our unique and comprehensive datasets support strategic decision-making and resource allocation across the UK healthcare More ❯

Job summary Are you looking for a pivotal role, where you can leverage your extensive experience in data engineering and architecture to design and implement robust data solutions that drive business insights and decision-making? We are looking for a Principal Data Engineer to lead a growing team and to own and help deliver … our Digital and Data strategy. Your leadership will be instrumental in setting technical direction and mentoring engineers to optimize our data pipelines and infrastructure. You will collaboratively develop and maintain data models, architecture, and integration strategies so that the organisation can use its data well and safely. You are equally as comfortable … to: Define and implement enterprise data models, pipelines, and metadata management Develop and maintain data platform components (e.g. data catalogue, data lake, warehouse, ETL frameworks) Ensure data security, compliance, and the ethical use of data in accordance with standards and requirements such as GDPR and Cyber Essentials More ❯

Broster Buchanan are partnered global business within the food industry with a turnover of £8Bn+ who are looking for a Fabric Data Engineer to join as they set up an exciting new project team within the group. This would suit an experienced Azure Data Engineer with good Fabric experience looking to join a new, growing team … Collaborate with their wider technology and business teams to ensure goals and visions are aligned and achieved Help with setting up the data warehousing and lake processes, including collaborating with their architecture teams to figure out what that will look like At times, guide and mentor the junior data engineers within the team … end project lifecycle Strong experience with setting up and using big data storage solutions, including data warehouses, data lakes or data lake houses Able to build and demonstrate a solid understanding of the concepts of real time data streaming, batch data processing and data transformation More ❯

As a Senior Data Engineer specializing in Natural Gas and Power, you will support the design, development, and optimization of advanced data infrastructure and products that support critical trading, risk, and market analytics functions. You will collaborate closely with traders, quantitative analysts, risk managers, and IT stakeholders to ensure data systems are scalable, secure … ETL/ELT frameworks and orchestration tools (e.g., Airflow, Azure Data Factory, Dagster). Proficient in cloud platforms (preferably Azure) and services such as Data Lake, Synapse, Event Hubs, and Functions. Authoring reports and dashboards with either open source or commercial products (e.g. PowerBI, Plot.ly, matplotlib) Programming OOP DevOps Web technologies HTTP/S REST … with time-series databases (e.g., InfluxDB, kdb+, TimescaleDB) and real-time data processing. Familiarity with distributed computing and data warehousing technologies (e.g., Spark, Snowflake, Delta Lake). Strong understanding of data governance, master data management, and data quality frameworks. Solid grasp of web technologies and APIs (REST, JSON, XML More ❯

Peaple Talent has partnered with a global digital engineering consultancy who are recruiting for a Lead Data Engineer on a permanent basis based out of their Sunderland office. You will have a great understanding of technology consultancy services along with fantastic communication and stakeholder engagement skills as this role will be client facing. Skills and experience: Proven experience … as a Lead Data Engineer with a focus on Azure cloud services. Experience of managing small teams whilst also being hands-on. Strong database fundamentals including SQL/TSQL, performance and schema design. Experience architecting and building data … applications using Azure, specifically a Data Warehouse and/or Data Lake. Technologies : Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, Azure Databricks and Power BI. Experience with creating low-level designs for data platform implementations. ETL pipeline development for the integration with data sources and More ❯

At Dufrain , we don't just build data solutions-we shape the future of data-driven decision-making. Do you thrive on solving complex challenges and turning data into powerful insights that drive transformation? We're looking for Data Engineers who have a broad range of data engineering skills, with … a focus on Microsoft Fabric . ABOUT THE ROLE As a Data Engineer, you will play a key role in delivering high-quality data solutions for our clients. This role offers the opportunity to work across a variety of technologies and contribute to the growth of our Consulting practice. ROLE RESPONSIBILITIES Develop strong working relationships with … clients on projects, demonstrating interpersonal skills with both business and technical colleagues. Experience in developing performant end-to-end data solutions in a collaborative team environment. Deliver high-quality work, escalate problems when necessary, and propose solutions. Support the growth of the Consulting practice through contributions to initiatives, knowledge sharing, and data services. ESSENTIAL TECHNICAL SKILLS More ❯

to technology, delivering bespoke software solutions and expert advice. Our clients are increasingly looking to us to help them make the best use of their data. In building data platforms and pipelines, our data engineers create the foundation for diverse data & analytics solutions, including data science and AI. They build data lakes and warehouses, create the processes to extract or access operational data, and transform siloed datasets into integrated data models that allow insight into business performance and problems or training of ML models. These are hands-on, client-facing roles, with openings at senior or lead level to suit your experience. You may be … or solving tough engineering challenges. You'd also be expected to spend some time on-site with clients in the London area on an ad-hoc basis. Our data engineers combine a strong software engineering approach with solid data fundamentals and experience with modern tools. We're technology agnostic, and we're open minded when it More ❯

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data Mesh, lakehouse, data vault and data warehouses. Our data engineers create pipelines that support our data scientists and power our front-end applications. This means we do data-intensive work for both OLTP and OLAP use cases. Our environments are primarily cloud-native spanning AWS, Azure and GCP More ❯

Senior Data Engineer - London/Hertfordshire - Hybrid Are you a technically strong Azure Data Engineer looking to join a forward-thinking, innovative company? Here's your chance to work with a leading construction and infrastructure company who foster a culture of collaboration, support, and strong communication. You'll be responsible for ensuring seamless integration and optimization … of diverse business data streams - from CRM, finance, procurement to fleet and commercial data - enabling reliable, high-quality data for analytics and business insights. While this role won't be client-facing, strong communication and collaboration skills within their agile team are essential. Requirements: Experience orchestrating … data pipelines with Azure Data Factory (ADF) Proficiency managing API data ingestion and orchestration using Azure Databricks Working with Azure Data Lake (ADL) Storage, on-prem SQL, and Python Vast Azure data engineering experience Proficient in ADF, Databricks, ADL, SQL, Python Desirable: Exposure to D365 or similar CRM environments More ❯

management technical knowledge so as to be able to work across full data cycle. • Proven Experience working with AWS data technologies (S3, Redshift, Glue, Lambda, Lake formation, Cloud Formation), GitHub, CI/CD • Coding experience in Apache Spark, Iceberg or Python (Pandas) • Experience in change and release management. • Experience in Database Warehouse design and data … ETL/ELT development in AWS preferred with experience in the following: • Strong experience in some of the AWS services like Redshift, Lambda,S3,Step Functions, Batch, Cloud formation, Lake Formation, Code Build, CI/CD, GitHub, IAM, SQS, SNS, Aurora DB • Good experience with DBT, Apache Iceberg, Docker, Microsoft BI stack (nice to have) • Experience in data warehouse design (Kimball and lake house, medallion and data vault) is a definite preference as is knowledge of other data tools and programming languages such as Python & Spark and Strong SQL experience. • Experience is building Data lake and building CI/CD data pipelines • A candidate is expected More ❯

support EQ's transition into a stronger, more profitable, product-led business, driven by real insights and the needs of its customers. We are looking for a skilled Data Engineer to join our Data team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data pipelines … cloud-based data pipelines and architectures Strong expertise in Microsoft Fabric and Databricks, including data pipeline development, data warehousing, and data lake management Proficiency in Python, SQL, Scala, or Java Experience with data processing frameworks such as Apache Spark, Apache Beam, or Azure Data Factory Strong understanding … Azure and or AWS Strong collaboration and communication skills, with the ability to work effectively with cross-functional teams Desirable: Experience with Azure Synapse Analytics, Azure Data Lake Storage, or other Azure data services Experience with agile development methodologies and version control systems such as Git Certification in Microsoft Azure, Databricks, or other relevant technologies More ❯

Data Solutions Architect – Any Cloud (Banking) Location: London, United Kingdom (Flexible Hybrid Working) Type: Permanent, Full-Time A leading global consultancy is seeking an experienced Data Solutions Architect to join its growing Data Management practice. This is a strategic, client-facing role focused on modernising data platforms within the banking sector , enabling … large-scale cloud transformation and data-driven decision-making. You will lead architecture design, influence enterprise data strategies, and guide technical delivery across cloud ecosystems like Azure , AWS , or GCP . This role also involves significant pre-sales engagement—translating client requirements into solution blueprints and supporting proposal development. Key Responsibilities Architect and oversee the delivery … of enterprise-scale data platforms (data lakes, lakehouses, warehouses) using tools like Databricks , Snowflake , Synapse , and Azure Fabric Define and execute cloud migration strategies, leveraging CI/CD pipelines, Terraform, Azure DevOps, and GitHub Support RFP/RFI responses, conduct client workshops, and shape pre-sales proposals (40–50% of the role) Advise senior client stakeholders More ❯

Data Solutions Architect – Any Cloud (Banking) Location: London, United Kingdom (Flexible Hybrid Working) Type: Permanent, Full-Time A leading global consultancy is seeking an experienced Data Solutions Architect to join its growing Data Management practice. This is a strategic, client-facing role focused on modernising data platforms within the banking sector , enabling … large-scale cloud transformation and data-driven decision-making. You will lead architecture design, influence enterprise data strategies, and guide technical delivery across cloud ecosystems like Azure , AWS , or GCP . This role also involves significant pre-sales engagement—translating client requirements into solution blueprints and supporting proposal development. Key Responsibilities Architect and oversee the delivery … of enterprise-scale data platforms (data lakes, lakehouses, warehouses) using tools like Databricks , Snowflake , Synapse , and Azure Fabric Define and execute cloud migration strategies, leveraging CI/CD pipelines, Terraform, Azure DevOps, and GitHub Support RFP/RFI responses, conduct client workshops, and shape pre-sales proposals (40–50% of the role) Advise senior client stakeholders More ❯

Lead Data Engineer - Manchester - Hybrid - £75k - £80k This is an excellent opportunity for a highly experienced data engineer to join a growing tech consultancy and contribute to the leadership of their data engineering practice. If you are an experienced and capable data engineer with a history of using Microsoft products and working … of clients to gather requirements and deliver solutions. Be willing to engage and assist in pre-sales activities, bids, proposals etc. Use key techniques such as Governance, Architecture, Data Modelling, ETL/ELT, Data Lakes, Data Warehousing, Master Data, and BI. Consistently utilise key processes in the engineering delivery cycle including Agile … and DevOps, Git, APIs, Containers, Microservices and Data Pipelines. Assist in building out, developing, and training the data engineering function. What do I need to apply Strong MS data engineering expertise (Fabric, ADF, ADL, Synapse, SQL) Expert use of Databricks Strong Python experience Consultancy experience Leadership experience My client are looking to book in More ❯

Lead Data Engineer - Manchester - Hybrid - £75k - £80k This is an excellent opportunity for a highly experienced data engineer to join a growing tech consultancy and contribute to the leadership of their data engineering practice. If you are an experienced and capable data engineer with a history of using Microsoft products and working … of clients to gather requirements and deliver solutions. Be willing to engage and assist in pre-sales activities, bids, proposals etc. Use key techniques such as Governance, Architecture, Data Modelling, ETL/ELT, Data Lakes, Data Warehousing, Master Data, and BI. Consistently utilise key processes in the engineering delivery cycle including Agile … and DevOps, Git, APIs, Containers, Microservices and Data Pipelines. Assist in building out, developing, and training the data engineering function. What do I need to apply Strong MS data engineering expertise (Fabric, ADF, ADL, Synapse, SQL) Expert use of Databricks Strong Python experience Consultancy experience Leadership experience My client are looking to book in More ❯

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data Mesh, lakehouse, data vault and data warehouses. Our data engineers create pipelines that support our data scientists and power our front-end applications. This means we do data-intensive work for both OLTP and OLAP use cases. Our environments are primarily cloud-native spanning AWS, Azure and GCP More ❯

A Manchester based FMCG is seeking a highly skilled Senior/LeadData Engineer to join their growing data team. This role will be instrumental in designing, building, and maintaining a modern data warehouse and data infrastructure to support advanced analytics and reporting needs for a fast-growing organisation. Key Responsibilities: Design … develop, and maintain scalable data pipelines using SQL and Python (PySpark) . Ingest, transform, and curate data from multiple sources into Azure Data Lake and Delta Lake formats. Build and optimize datasets for performance and reliability in Azure Databricks . Collaborate with analysts and business stakeholders to translate data requirements … in SQL for data transformation and performance tuning. Solid experience with Python , ideally using PySpark in Azure Databricks . Hands-on experience with Azure Data Lake Storage Gen2 . Understanding of data warehouse concepts , dimensional modelling , and data architecture . Experience working with Delta Lake and large-scale data More ❯

The Role We are looking for a hands-on Azure Data Engineer who will lead the final phase of our Client's cloud migration and design the enterprise-grade data platform from the ground up. This is a hybrid role with a strong technical focusblending architecture, automation, and data engineeringto empower s next generation … across Europe, the US, and APAC. As they scale globally and accelerate their AI capabilities, they are completing their transition to the cloud and building a company-wide data platform to power insight-driven transformation for their consultants and clients Required Skills & Experience Must-Haves: 3+ years of hands-on Azure engineering experience (IaaS ? PaaS), including Infra as … data SMEguiding system administrators and upskilling junior technical team members. What You'll Achieve in Year 1 Months 312: Design and build their Azure data lake using Synapse , Fabric , or an alternative strategy. Ingest data from core platforms: NetSuite , HubSpot , and client RFP datasets. Automate data pipelines using ADF , Fabric Dataflows More ❯

The Role We are looking for a hands-on Azure Data Engineer who will lead the final phase of our Client's cloud migration and design the enterprise-grade data platform from the ground up. This is a hybrid role with a strong technical focus—blending architecture, automation, and data engineering—to empower s … across Europe, the US, and APAC. As they scale globally and accelerate their AI capabilities, they are completing their transition to the cloud and building a company-wide data platform to power insight-driven transformation for their consultants and clients Required Skills & Experience Must-Haves: 3+ years of hands-on Azure engineering experience (IaaS PaaS), including Infra as … SME—guiding system administrators and upskilling junior technical team members. What You'll Achieve in Year 1 Months 3-12: Design and build their Azure data lake using Synapse , Fabric , or an alternative strategy. Ingest data from core platforms: NetSuite , HubSpot , and client RFP datasets. Automate data pipelines using ADF , Fabric Dataflows More ❯

Senior Data Engineer - London/Hertfordshire - Hybrid Are you a technically strong Azure Data Engineer looking to join a forward-thinking, innovative company? Here's your chance to work with a leading construction and infrastructure company who foster a culture of collaboration, support, and strong communication. You'll be responsible for ensuring seamless integration and optimization … of diverse business data streams - from CRM, finance, procurement to fleet and commercial data - enabling reliable, high-quality data for analytics and business insights. While this role won't be client-facing, strong communication and collaboration skills within their agile team are essential. Requirements: Experience orchestrating … data pipelines with Azure Data Factory (ADF) Proficiency managing API data ingestion and orchestration using Azure Databricks Working with Azure Data Lake (ADL) Storage, on-prem SQL, and Python Vast Azure data engineering experience Proficient in ADF, Databricks, ADL, SQL, Python Desirable: Exposure to D365 or similar CRM environments More ❯

Senior Data Engineer - London/Hertfordshire - Hybrid Are you a technically strong Azure Data Engineer looking to join a forward-thinking, innovative company? Here's your chance to work with a leading construction and infrastructure company who foster a culture of collaboration, support, and strong communication. You'll be responsible for ensuring seamless integration and optimization … of diverse business data streams - from CRM, finance, procurement to fleet and commercial data - enabling reliable, high-quality data for analytics and business insights. While this role won't be client-facing, strong communication and collaboration skills within their agile team are essential. Requirements: Experience orchestrating … data pipelines with Azure Data Factory (ADF) Proficiency managing API data ingestion and orchestration using Azure Databricks Working with Azure Data Lake (ADL) Storage, on-prem SQL, and Python Vast Azure data engineering experience Proficient in ADF, Databricks, ADL, SQL, Python Desirable: Exposure to D365 or similar CRM environments More ❯

UK Head Office - Manchester Hybrid/PERMANENT/SALARY: COMPETITIVE Job Purpose The Data Lead is responsible for leading and growing MPM's data capabilities. Managing the data team, overseeing the design, delivery, and governance of data infrastructure, pipelines, and analytics solutions. The Data Lead is also responsible for setting … and executing the vision for data within the organisation, enabling accurate, timely, and actionable insights through data practices, collaboration, and strong leadership. This role deals with a wide variety of tasks including, but not limited to the following: Manage, mentor, and grow the newly formed Data Team to deliver high-quality solutions that meet … evolving business objectives. Define and maintain the roadmap for data infrastructure, tools, and quality standards. Lead a data-first culture across the organisation, championing data literacy and self-service capabilities. Represent the Data Team in leadership discussions and ensure alignment with business priorities. Continuously improve data operations, governance, and platform More ❯

people too - creating new challenges and opportunities all the time. We're a business that never stands still. Join us on our journey to as a Head of Data Engineering & Insight, and you won't either. As part of our accelerated digital transformation and the implementation of our new Next Generation BI platform, we are looking for a … years' experience in Business Intelligence, with 5+ years in a BI leadership role in a global or matrixed organisation . Proven expertise in modern BI architecture (Data Lake, EDW, Streaming, APIs, Real-Time & Batch Processing). Demonstrated experience delivering cloud-based analytics platforms (Azure, AWS, GCP). Strong knowledge of data governance, cataloguing, security, automation … Engineering & Insight will work within a modern, cloud-based BI ecosystem , including: Data Integration: Fivetran , HVR, Databricks , Apache Kafka, Google BigQuery , Google Analytics 4 Data Lake & Storage: Databricks Delta Lake, Amazon S3 Data Transformation: dbt Cloud Data Warehouse: Snowflake Analytics & Reporting: Power BI, Excel, Snowflake SQL REST API Advanced Analytics More ❯