1 to 25 of 44 Data Lake Jobs in the North of England

Job summary We are looking for an experienced Senior Data Engineer with strong technical expertise and innovative mindset to drive our team forward. Main duties of the job Data Engineer will be involved in the entire data pipeline from data acquisition to data extraction, ensuring that the data is accurate, reliable, and accessible, and fit-for-purpose for business intelligence, analytics, and data science functions. Data Engineer will be collaborating with cross-functional teams to support the delivery of impactful analytics work for our clients. About us We are the official research support team for the NHS Benchmarking Network, working with over … healthcare organisations across the UK to identify opportunities for service development and quality improvement. We also serve as the primary source of benchmarking data for the NHS England Workforce, Training and Education team, providing in-depth analysis across national mental health services. Our unique and comprehensive datasets support strategic decision-making and resource allocation across the UK healthcare More ❯

Senior Data Ops Data EngineerLeeds 2/3 days per week£55,000-£67,000+ Excellent Benefits18 Months FTC with long term potential Your New Company Our large public sector organisation are seeking a senior Data Platform DataOps Engineer to serve as our clients first DataOps specialist in a growing team of Data … and DevOps teams to align operational strategies with technical and business requirements. Optimize operational performance and cost management for services including Azure Data Factory, Azure Databricks, Delta Lake, and Azure Data Lake Storage. Serve as the domain expert in DataOps by providing strategic guidance, mentoring colleagues, and driving continuous process improvements. What you will … within the Microsoft Azure ecosystem. Hands-on expertise with Azure Data Platform with components such as Azure Data Factory, Azure Databricks, Azure Data Lake Storage, Delta Lake, Azure SQL, Purview and APIM. Proficiency in developing CI/CD data pipelines and strong programming skills in Python, SQL, Bash, and PySpark More ❯

Social network you want to login/join with: We are looking for an experienced Data Governance Lead to drive enterprise-wide data governance initiatives with a focus on modern cloud data platforms, specifically Databricks and Microsoft Azure. This role will define and enforce data policies, standards, and best practices that ensure … business objectives and compliance requirements. Collaborate with data engineering and platform teams to integrate data governance within Databricks and Azure environments, including Data Lake, Synapse, and Purview. Define and enforce data classification, access management, and lineage tracking using Azure-native and third-party tools. Lead the Data Governance Council … governance, data management, or data architecture. Hands-on experience with Databricks and/or Microsoft Azure data services (e.g., Data Lake Storage, Synapse, Purview, Azure Data Factory). Strong understanding of data governance frameworks such as DAMA-DMBOK. Familiarity with data privacy, security, and More ❯

management technical knowledge so as to be able to work across full data cycle. • Proven Experience working with AWS data technologies (S3, Redshift, Glue, Lambda, Lake formation, Cloud Formation), GitHub, CI/CD • Coding experience in Apache Spark, Iceberg or Python (Pandas) • Experience in change and release management. • Experience in Database Warehouse design and data … ETL/ELT development in AWS preferred with experience in the following: • Strong experience in some of the AWS services like Redshift, Lambda,S3,Step Functions, Batch, Cloud formation, Lake Formation, Code Build, CI/CD, GitHub, IAM, SQS, SNS, Aurora DB • Good experience with DBT, Apache Iceberg, Docker, Microsoft BI stack (nice to have) • Experience in data warehouse design (Kimball and lake house, medallion and data vault) is a definite preference as is knowledge of other data tools and programming languages such as Python & Spark and Strong SQL experience. • Experience is building Data lake and building CI/CD data pipelines • A candidate is expected More ❯

that adventure. Our employees are at the heart of delivering impactful and meaningful work for our clients and helping them to reach for and realise their vision. Snowflake Data Architect Slalom is seeking an experienced Snowflake Data Architect to contribute to its growing Data Capability. The ideal candidate is a professional … with experience in designing, developing, validating and communicating enterprise data solutions using Snowflake. Deep experience in developing enterprise data management strategies including data lake/warehouse implementations, data movement, data services, data acquisition, data conversion, and archive/recovery. What will you do? Lead as …/advise clients on reference architectures Requirements: Expert in capturing end user requirements and aligning technical solutions to the business objectives Experience in leading, designing and implementing Cloud Data strategies, including designing multi-phased implementation roadmaps Significant experience (and ideally certified) in Snowflake and acted as lead data architect on at least two Snowflake implementations. Expertise More ❯

Senior Data Engineer Hybrid 2x per week (Leeds, West Yorkshire) Salary to £60,000 + benefits About the Role: Our client, a tech for good organisation, is investing heavily in digital transformation and building a cloud-first, data-led approach to support major environmental goals. Think scalable pipelines, secure platforms, and data systems that … directly contribute to a greener future. As a Senior Data Engineer, you'll play a key role in designing and delivering robust, cloud-based data infrastructure. You'll work with large datasets, build scalable pipelines, and help ensure data is high quality, secure, and accessible, while mentoring more junior engineers and working closely with … Cloud data platform development and deployment. Expertise across data warehouse and ETL/ELT development in AWS preferred Experience is building a Data Lake as well as building CI/CD data pipelines Excellent communication and problem-solving skills. What's on Offer: Salary to £60,000 + benefits Hybrid working More ❯

Databricks Data Engineer (roles available at Lead, Senior and Mid Level), £ 50000 - 85000 + benefits. SQL, ETL, Data Warehousing, Databricks etc. Home Based with one day a month at the office in Nottingham. Strong commercial knowledge of Databricks is required for this role. Expanding SaaS product company are looking for a number of Data Engineer as they continue to grow. In these hands-on roles you will be part of the team responsible for designing, creating, deploying and managing the companies data assets and you will guide and influence other members of the data engineering team with the ultimate goal of writing excellent quality, clean and high performant code. … Excellent knowledge of the SDLC, including testing and delivery in an agile environment. Excellent knowledge of SQL and ETL Experience of data warehousing and data lake solutions. Experience working in an Agile environment. This is an excellent opportunity to join a company as it continues to grow and expand its data team. In More ❯

Azure Data Engineer - Leeds (Hybrid) - £65,000 Are you an experienced Azure Data Engineer with a passion for building scalable, cloud-native data solutions? We're recruiting on behalf of a dynamic organisation in Leeds that's investing in its data platform - and they're looking for someone with strong Databricks … expertise to join their team. About the role: Designing and developing robust data pipelines using Azure Data Factory, Databricks, and Synapse Analytics. Working with Delta Lake and Azure Data Lake Storage to manage and optimise large datasets. Collaborating with data analysts, engineers, and business stakeholders to deliver clean, reliable data. … Azure-based architecture Ensuring best practices in data governance, security, and performance tuning Requirements: Proven experience with Azure Data Services (ADF, Synapse, Data Lake) Strong hands-on experience with Databricks (including PySpark or SQL) Solid SQL skills and understanding of data modelling and ETL/ELT processes Familiarity with Delta Lake More ❯

Principal Data Engineer - hyrbid (Leeds) - £70k-£80k Are you an experienced Azure Data Engineer looking to make an impact in an established financial insitution. We are seeking a hands-on professional to join a high-performing team tasked with building and enhancing a modern Azure data platform, with an expertise in Databricks. About the … architectural decisions, and support the migration of reporting services to Azure. Key Responsibilities: Design, build, and maintain ETL pipelines using Azure Data Factory , Azure Data Lake , Synapse , and Databricks . Design and build a greenfield Azure data platform to support business-critical data needs. Collaborate with stakeholders across the organization to … outputs. Opportunity to transition into a leadership role, supporting the development of junior team members. Key Skills & Experience: 5+ years of experience in Azure (ADF, Data Lake, Synapse). Strong SQL skills for data integration and reporting. Experience with Power BI for visualization and analytics. Familiarity with Python (preferred but not currently in use More ❯

Data Engineer - Leading Pharmaceutical Company - Manchester(Tech Stack: Data Engineer, Databricks, Python, Power BI, Azure, TSQL, ETL, Agile Methodologies)About the Role: We are seeking a talented and experienced Data Engineer on behalf of our client, a leading Software House. This is a fully remote position, offering the opportunity to work with cutting-edge … technologies and contribute to exciting projects in a collaborative environment.About the Role: Our client is seeking an experienced Data Engineer to join their team in Manchester. This hybrid position involves working within the pharmaceutical industry, focusing on the design, development, and maintenance of data pipelines, ETL processes, and databases. The role is ideal for someone passionate … in an agile team.Experience working in a highly regulated industry and with highly sensitive data.Exposure to large data solutions like Snowflake, Trino, Synapse, Azure Data Lake, and Databricks.Experience in data science using R, Stata, or Python.Familiarity with Atlassian tools such as JIRA, Confluence, and BitBucket.Understanding of clinical trials, GCP, and GxP.What We Offer More ❯

Lead Data Engineer, £ 70000 - 80000 + benefits. SQL, ETL, Data Warehousing, Databricks etc. Home Based with one day a month at the office in Nottingham. Expanding SaaS product company are looking for a Lead Data Engineer as they continue to grow. In this hands-on role you will be part of the team responsible … for designing, creating, deploying and managing the companies data assets and you will guide and influence other members of the data engineering team with the ultimate goal of writing excellent quality, clean and high performant code. Responsibilities include - Work with the Data Architects to determine technical delivery and functionality. Design data solutions … technical solutions. Excellent knowledge of the SDLC, including testing and delivery in an agile environment. Excellent knowledge of SQL. Experience of data warehousing and data lake solutions. Good knowledge of ETL processes and tools. Knowledge of databricks. This is an excellent opportunity to join a company as it continues to grow and expand its data More ❯

Data Engineer - Leading Pharmaceutical Company - Manchester (Tech Stack: Data Engineer, Databricks, Python, Power BI, Azure, TSQL, ETL, Agile Methodologies) About the Role: We are seeking a talented and experienced Data Engineer on behalf of our client, a leading Software House. This is a fully remote position, offering the opportunity to work with cutting-edge … technologies and contribute to exciting projects in a collaborative environment. About the Role: Our client is seeking an experienced Data Engineer to join their team in Manchester. This hybrid position involves working within the pharmaceutical industry, focusing on the design, development, and maintenance of data pipelines, ETL processes, and databases. The role is ideal for someone … agile team. Experience working in a highly regulated industry and with highly sensitive data. Exposure to large data solutions like Snowflake, Trino, Synapse, Azure Data Lake, and Databricks. Experience in data science using R, Stata, or Python. Familiarity with Atlassian tools such as JIRA, Confluence, and BitBucket. Understanding of clinical trials, GCP, and More ❯

Prestigious opportunity for a Data Platform Engineer within a pioneering technology team based in Manchester. Following a period of significant growth, investment and a cloud transformation, we are inviting you to join our success story! As our Azure Data Platform Engineer, you will be responsible for:- Designing, building, and maintaining scalable data solutions on … Microsoft Azure. Designing and implementing scalable data pipelines using Azure Data Factory, Databricks, Synapse Analytics, and other Azure services. Leading technical workstreams and supporting project delivery. Acting as the subject-matter expert for cloud-based data engineering. Ensure data governance, security, and compliance standards are met. Monitoring and troubleshooting data … you possess a combination of some of the following skills, then LETS TALK! Proven experience with Azure data services (SQL, Data Factory, Data Lake, Synapse, Databricks, Azure SQL and Cosmos DB). Strong proficiency in designing and operating scalable data solutions and pipelines. Platform engineering - Terraform Familiar with Cloud security, performance More ❯

Data Engineer Join a Trailblazing Team in Clinical Research Data ManagementStep into a role that places you at the cutting edge of clinical trials, leveraging Electronic Health Record (EHR) data to revolutionise patient care and research. Based in the innovative Manchester Science Park, this position offers the chance to be part of a world … is usually on Tuesdays and Thursdays but can be subject to change. Strong benefits package This role focuses on the development, performance, management, and troubleshooting of ETL processes, data pipelines, and data infrastructure. The successful candidate will ensure the effective and reliable operation of these systems, adopting new tools and technologies to stay ahead of industry … database testing; unit, performance, stress, security.- Experience in agile teams and highly regulated industries.- Exposure to large data solutions like Snowflake, Trino, Synapse, Azure Data Lake, and Databricks.- Data science experience using R, Stata, or Python.- Familiarity with Atlassian tools; JIRA, Confluence, BitBucket.- Understanding of clinical trials, GCP, and GxP.Personal Attributes:- Strong collaboration More ❯

Haywards Heath, Home Office (Remote) or Manchester Salary: £55,000 - £65,000 depending on experience Department: Finance We're 1st Central, a market-leading insurance company utilising smart data and technology at pace. Rapid growth has been based on giving our 1.4 million customers exactly what they want: great value insurance with an excellent service. And that's … Lead Finance Solutions Developer 1st Central are on the lookout for a technically strong and analytically minded Lead Finance Solutions Developer who can bridge the gap between finance, data engineering, and actuarial analytics. If you're passionate about delivering high-quality, scalable data solutions to support business-critical decisions, and you enjoy working across multidisciplinary teams … related field. Strong SQL and data modelling skills; experience with relational databases. Skilled in working with cloud platforms (e.g., AWS, Azure, or GCP) and data lake environments. Experience with actuarial models, principles, and data structures (e.g., Actuarial reserving techniques, cashflow projections under Solvency II and IFRS 17) life tables, claims triangles, policy reserves More ❯

We are on the lookout for a Data Engineer with a robust background in SQL Server and ideally Azure Data Factory. The ideal candidate will have a wealth of experience in business intelligence, data modelling, and visualisation, and will be proficient in Agile Scrum Development methodology. This pivotal role involves leading the development of … based in Middlesbrough, with on-site presence 2-3 times weekly. Day to Day of the role: Lead the development of BI Solution Architecture, Replication Warehouse, and secure data sets using the Microsoft application stack. Create and maintain secure data sets for systems, enabling Key users to produce reports. Perform ETL integrations and support tasks using … a related role. Proficiency with SQL Management Server, SSRS, SSIS, and other related systems. Experience with Microsoft Azure stack and technologies such as ADF, Logic Apps, Data Lake, Data Bricks, Power Query, etc. Experience in low code MS related technologies, SharePoint, Nintex, Power Apps, Power Bi, etc. ETL related experience using SSIS, ADF, or similar More ❯

We are on the lookout for a Data Engineer with a robust background in SQL Server and ideally Azure Data Factory. The ideal candidate will have a wealth of experience in business intelligence, data modelling, and visualisation, and will be proficient in Agile Scrum Development methodology. This pivotal role involves leading the development of … based in Goldthorpe, with on-site presence 2-3 times weekly. Day to Day of the role: Lead the development of BI Solution Architecture, Replication Warehouse, and secure data sets using the Microsoft application stack. Create and maintain secure data sets for systems, enabling Key users to produce reports. Perform ETL integrations and support tasks using … a related role. Proficiency with SQL Management Server, SSRS, SSIS, and other related systems. Experience with Microsoft Azure stack and technologies such as ADF, Logic Apps, Data Lake, Data Bricks, Power Query, etc. Experience in low code MS related technologies, SharePoint, Nintex, Power Apps, Power Bi, etc. ETL related experience using SSIS, ADF, or similar More ❯

As a Senior Data Engineer , you will be an integral part of a cross-functional feature development squad, dedicated to delivering innovative data solutions on our leading analytics platform. You will utilize your diverse skill set to contribute to all aspects of data engineering, with a core focus in designing scalable and secure data architectures for data retrieval. Collaborating closely with other engineers, a scrum master, and business stakeholders, you will ensure a continuous delivery of value in an agile environment. Collaborate within the squad to define and deliver scalable, secure data features aligned with business goals. Architect high-performance data retrieval solutions (APIs, web scrapers … in data feature development and stay updated on industry trends, implementing innovative tools and solutions. Proficient in Microsoft Azure (DevOps, Data Factory, Data Lake, Functions, Azure Container Instances, RBAC and Entra) and DevOps CI/CD. Proficient in Infrastructure as Code (Terraform), secure FTP configurations (SFTP/FTPS), and remediation of security vulnerabilities More ❯

Simpson Associates transforms raw data into actionable insights that drive positive change. Our Microsoft data expertise, our specialist sector knowledge, plus our innovative and trusted advice and guidance are just some of the reasons clients choose to work with us. Our mission is to help purpose-led organisations from within the public and private sectors to … harness data as a lever for change and enable them to realise business value more quickly. We provide the full range of services to support organisations on their data transformation journey. From advisory support and data strategy, to developing Data & AI solutions, right through to providing a range of managed services. We … key data science models and how they relate to achieving business benefits. Advantageous Qualifications and Skills Microsoft Azure expertise including the use of DevOps, Data Lake, Azure Functions, Power BI. Experience in project delivery and consultancy. Prince 2 Agile or similar. Experience in Blue Light, NHS, Local Government or Social Housing sectors. Experience with MLOps More ❯

Data Engineer- Outside Ir35 - £500 daily rate- Manchester - 6 Month Role Overview: We're seeking an experienced Contract Data Engineer with a strong background in Databricks and cloud data platforms to support a high-impact data transformation programme. This hybrid role requires 2 days per week onsite in our Manchester office (non … negotiable) , with the remainder remote. You'll join a fast-paced, delivery-focused data team responsible for building and optimising scalable, production-grade data pipelines and infrastructure. Key Responsibilities: Design and implement robust, scalable ETL/ELT pipelines using Databricks and Apache Spark Ingest, transform, and manage large volumes of data from diverse sources … based projects Strong hands-on skills with Databricks , Apache Spark , and Python or Scala Proficient in SQL and working with large-scale data environments Experience with Delta Lake , Azure Data Lake , or similar technologies Familiarity with version control, CI/CD, and infrastructure-as-code tools Ability to deliver independently in a contractor capacity More ❯

Data Engineer- Outside Ir35 - £500 daily rate- Manchester - 6 Month Role Overview: We're seeking an experienced Contract Data Engineer with a strong background in Databricks and cloud data platforms to support a high-impact data transformation programme. This hybrid role requires 2 days per week onsite in our Manchester office (non … negotiable) , with the remainder remote. You'll join a fast-paced, delivery-focused data team responsible for building and optimising scalable, production-grade data pipelines and infrastructure. Key Responsibilities: Design and implement robust, scalable ETL/ELT pipelines using Databricks and Apache Spark Ingest, transform, and manage large volumes of data from diverse sources … based projects Strong hands-on skills with Databricks , Apache Spark , and Python or Scala Proficient in SQL and working with large-scale data environments Experience with Delta Lake , Azure Data Lake , or similar technologies Familiarity with version control, CI/CD, and infrastructure-as-code tools Ability to deliver independently in a contractor capacity More ❯

Our Senior Data Engineers enable public sector organisations to embrace a data-driven approach by providing high-quality, cost-efficient data platforms and services tailored to clients' needs. They develop, operate, and maintain these services, ensuring maximum value for data consumers, including analysts, scientists, and business stakeholders. Key responsibilities As a Senior … Data Engineer, you may assume multiple roles based on our clients' needs. The role is highly hands-on, supporting project delivery as a senior contributor and upskilling client team members. You might also take on a technical architect role, collaborating with the MadeTech team to identify growth opportunities within the account. You'll need a drive to deliver … criteria. Enthusiasm for learning and self-development Proficiency in Git (including Github Actions) and understanding of branch strategies Experience gathering and meeting requirements from clients and users on data projects Strong experience in Infrastructure as Code (IaC) and deploying infrastructure across environments Managing cloud infrastructure with a DevOps approach Handling and transforming various data types (JSON More ❯

Our Senior Data Engineers enable public sector organisations to embrace a data-driven approach by providing high-quality, cost-efficient data platforms and services tailored to clients' needs. They develop, operate, and maintain these services, ensuring maximum value for data consumers, including analysts, scientists, and business stakeholders. Key responsibilities As a Senior … Data Engineer, you may assume multiple roles based on our clients' needs. The role is highly hands-on, supporting project delivery as a senior contributor and upskilling client team members. You might also take on a technical architect role, collaborating with the MadeTech team to identify growth opportunities within the account. You'll need a drive to deliver … criteria. Enthusiasm for learning and self-development Proficiency in Git (including Github Actions) and understanding of branch strategies Experience gathering and meeting requirements from clients and users on data projects Strong experience in Infrastructure as Code (IaC) and deploying infrastructure across environments Managing cloud infrastructure with a DevOps approach Handling and transforming various data types (JSON More ❯

AWS Data Engineer - IT Consultancy - Leeds - Hybrid Are you a data engineer with a passion for AWS and solving real-world business problems? I am working with a fast-growing IT consultancy based in Leeds that's delivering cutting-edge cloud and data solutions to clients across the UK. They're looking for their … next AWS Data Engineer to join the team. As an AWS Data Engineer, you'll be working on a variety of client projects, helping to design, build, and optimise scalable data pipelines and cloud-native architectures. You'll collaborate with cross-functional teams to deliver high-impact solutions that drive business value. Requirements: Experience … Python SQL & NoSQL databases CI/CD tools : GitHub Actions or CodePipeline Infrastructure as Code (Terraform is preferable) Experience building ETL/ELT pipelines Familiarity with data lake and data warehouse concepts Excellent communication and stakeholder engagement skills Benefits: Hybrid working (one day per week in Leeds office) Clear career progression in a growing consultancy More ❯

At Amazon, data is critical to everything we do, from understanding product inventories across our global fulfillment centers to predicting what products to recommend to customers. Data ceaselessly informs our flywheel, helping us to understand customer needs, onboard new sellers and selection, and identify opportunities for cost savings. Business Data Technologies (BDT) provides trustworthy … intuitive, and cost-efficient solutions for Amazonians to secure, store, analyze, and transform this data at unparalleled scale. BDT owns, develops, and operates Andes, one of the largest data lakes in the world, storing more than 2 exabytes of data, read/written to by 100k+ data pipelines, with >60k monthly active … that is compliant with Amazon policies and law of the land. The BDT Data Governance organization works to ensure that users of the Andes data lake and policy owners have a consistent and seamless experience complying with Amazon's data handling policies. This across multiple applications, UIs, APIs, CDK and CLIs. The Data More ❯