1 to 25 of 379 Data Lake Jobs

At Dufrain , we don't just build data solutions-we shape the future of data-driven decision-making. Do you thrive on solving complex challenges and turning data into powerful insights that drive transformation? We're looking for Data Engineers who have a broad range of data engineering skills, with … a focus on Microsoft Databricks . Experience with Microsoft Fabric or Snowflake is also highly desirable. ABOUT THE ROLE As a Data Engineer, you will play a key role in delivering high-quality data solutions for our clients. This role offers the opportunity to work across a variety of technologies and contribute to the growth of … and engineering practices. Key competencies include: Databricks Platform Expertise : Proven experience designing and delivering data solutions using Databricks on Azure or AWS. Databricks Components : Proficient in Delta Lake, Unity Catalog, MLflow, and other core Databricks tools. Programming & Query Languages : Strong skills in SQL and Apache Spark (Scala or Python). Relational Databases : Experience with on-premises and More ❯

Job summary Are you a data engineer with a passion for public service? Then maybe you're looking for us! We are seeking an experienced and skilled Advanced Data Engineer to be part of a growing team that delivers on our mission of protecting and improving the health and wellbeing of the people of Wales. You … ll be involved in improving our data practices, cataloguing our data, automating our data processing, analytical pipelines and developing our overall data maturity. Working with us you'll collaborate with other technical specialists (such as developers, database administrators, data analysts and data scientists) on the development, implementation, maintenance … do things. Accountabilities Implement enterprise data models, pipelines, and metadata management Develop and maintain data platform components (e.g. data catalogue, data lake, warehouse, ETL frameworks) Follow best practice in data security, compliance, and the ethical use of data in accordance with standards and requirements such as GDPR More ❯

Job summary We are looking for an experienced Senior Data Engineer with strong technical expertise and innovative mindset to drive our team forward. Main duties of the job Data Engineer will be involved in the entire data pipeline from data acquisition to data extraction, ensuring that the data is accurate, reliable, and accessible, and fit-for-purpose for business intelligence, analytics, and data science functions. Data Engineer will be collaborating with cross-functional teams to support the delivery of impactful analytics work for our clients. About us We are the official research support team for the NHS Benchmarking Network, working with over … healthcare organisations across the UK to identify opportunities for service development and quality improvement. We also serve as the primary source of benchmarking data for the NHS England Workforce, Training and Education team, providing in-depth analysis across national mental health services. Our unique and comprehensive datasets support strategic decision-making and resource allocation across the UK healthcare More ❯

Salary: 50.000 - 60.000 € per year Requirements: • 3+ years of hands-on experience as a Data Engineer working with Databricks and Apache Spark • Strong programming skills in Python, with experience in data manipulation libraries (e.g., PySpark, Spark SQL) • Experience with core components of the Databricks ecosystem: Databricks Workflows, Unity Catalog, and Delta Live Tables • Solid understanding of … data warehousing principles, ETL/ELT processes, data modeling and techniques, and database systems • Proven experience with at least one major cloud platform (Azure, AWS, or GCP) • Excellent SQL skills for data querying, transformation, and analysis • Excellent communication and collaboration skills in English and German (min. B2 levels) • Ability to work independently as well … ingest, transform, and load data from various sources (databases, APIs, streaming platforms) into cloud-based data lakes and warehouses • Leveraging the Databricks ecosystem (SQL, Delta Lake, Workflows, Unity Catalog) to deliver reliable and performant data workflows • Integrating with cloud services such as Azure, AWS, or GCP to enable secure, cost-effective data More ❯

European locations. We believe in the power of agile and cross-functional collaboration, bringing together people from various backgrounds to create outstanding digital solutions. Aufgaben As a Data Engineer with a focus on Databricks , you will play a key role in building modern, scalable, and high-performance data solutions for our clients. You'll be part … of our growing Data & AI team and work hands-on with the Databricks platform, supporting clients in solving complex data challenges. Your Job's Key Responsibilities Are: Designing, developing, and maintaining robust data pipelines using Databricks, Spark, and Python Building efficient and scalable ETL processes to … ingest, transform, and load data from various sources (databases, APIs, streaming platforms) into cloud-based data lakes and warehouses Leveraging the Databricks ecosystem (SQL, Delta Lake, Workflows, Unity Catalog) to deliver reliable and performant data workflows Integrating with cloud services such as Azure, AWS, or GCP to enable secure, cost-effective data More ❯

European locations. We believe in the power of agile and cross-functional collaboration, bringing together people from various backgrounds to create outstanding digital solutions. Aufgaben As a Data Engineer with a focus on Databricks , you will play a key role in building modern, scalable, and high-performance data solutions for our clients. You'll be part … of our growing Data & AI team and work hands-on with the Databricks platform, supporting clients in solving complex data challenges. Your Job's Key Responsibilities Are: Designing, developing, and maintaining robust data pipelines using Databricks, Spark, and Python Building efficient and scalable ETL processes to … ingest, transform, and load data from various sources (databases, APIs, streaming platforms) into cloud-based data lakes and warehouses Leveraging the Databricks ecosystem (SQL, Delta Lake, Workflows, Unity Catalog) to deliver reliable and performant data workflows Integrating with cloud services such as Azure, AWS, or GCP to enable secure, cost-effective data More ❯

European locations. We believe in the power of agile and cross-functional collaboration, bringing together people from various backgrounds to create outstanding digital solutions. Aufgaben As a Data Engineer with a focus on Databricks , you will play a key role in building modern, scalable, and high-performance data solutions for our clients. You'll be part … of our growing Data & AI team and work hands-on with the Databricks platform, supporting clients in solving complex data challenges. Your Job's Key Responsibilities Are: Designing, developing, and maintaining robust data pipelines using Databricks, Spark, and Python Building efficient and scalable ETL processes to … ingest, transform, and load data from various sources (databases, APIs, streaming platforms) into cloud-based data lakes and warehouses Leveraging the Databricks ecosystem (SQL, Delta Lake, Workflows, Unity Catalog) to deliver reliable and performant data workflows Integrating with cloud services such as Azure, AWS, or GCP to enable secure, cost-effective data More ❯

Job summary Are you looking for a pivotal role, where you can leverage your extensive experience in data engineering and architecture to design and implement robust data solutions that drive business insights and decision-making? We are looking for a Principal Data Engineer to lead a growing team and to own and help deliver … our Digital and Data strategy. Your leadership will be instrumental in setting technical direction and mentoring engineers to optimize our data pipelines and infrastructure. You will collaboratively develop and maintain data models, architecture, and integration strategies so that the organisation can use its data well and safely. You are equally as comfortable … to: Define and implement enterprise data models, pipelines, and metadata management Develop and maintain data platform components (e.g. data catalogue, data lake, warehouse, ETL frameworks) Ensure data security, compliance, and the ethical use of data in accordance with standards and requirements such as GDPR and Cyber Essentials More ❯

Broster Buchanan are partnered global business within the food industry with a turnover of £8Bn+ who are looking for a Fabric Data Engineer to join as they set up an exciting new project team within the group. This would suit an experienced Azure Data Engineer with good Fabric experience looking to join a new, growing team … Collaborate with their wider technology and business teams to ensure goals and visions are aligned and achieved Help with setting up the data warehousing and lake processes, including collaborating with their architecture teams to figure out what that will look like At times, guide and mentor the junior data engineers within the team … end project lifecycle Strong experience with setting up and using big data storage solutions, including data warehouses, data lakes or data lake houses Able to build and demonstrate a solid understanding of the concepts of real time data streaming, batch data processing and data transformation More ❯

About Exsolvæ: Exsolvæ is a Brussels-based consultancy at the forefront of Data and Artificial Intelligence innovation. We specialize in providing expert Consultancy, Audit, and Solution Development services to empower our partners in overcoming unique challenges. Our holistic approach fosters collaboration across diverse domains, including Data Engineering, Advanced Analytics, and AI. At Exsolvæ, our "Solvers" are … work experience Experience: On-Premise AND Cloud hands on Data Engineering experience Azure Expertise : Extensive experience with Azure Data Factory, Synapse Analytics, Data Lake Storage, and Azure Databricks. Programming Skills : Proficiency in Python and SQL, with experience in big data tools like Apache Spark. Cloud Proficiency : Hands-on experience with Azure … to streamline data processing and storage. Data Ecosystem Implementation : Design and implement solutions leveraging Azure Synapse Analytics, Data Factory, Databricks, Data Lake Storage, and related tools. Pipeline Optimization : Develop and maintain robust ETL/ELT pipelines to ensure efficient data ingestion and transformation. Technical Implementation Data Modeling More ❯

Peaple Talent has partnered with a global digital engineering consultancy who are recruiting for a Lead Data Engineer on a permanent basis based out of their Sunderland office. You will have a great understanding of technology consultancy services along with fantastic communication and stakeholder engagement skills as this role will be client facing. Skills and experience: Proven experience … as a Lead Data Engineer with a focus on Azure cloud services. Experience of managing small teams whilst also being hands-on. Strong database fundamentals including SQL/TSQL, performance and schema design. Experience architecting and building data … applications using Azure, specifically a Data Warehouse and/or Data Lake. Technologies : Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, Azure Databricks and Power BI. Experience with creating low-level designs for data platform implementations. ETL pipeline development for the integration with data sources and More ❯

At Dufrain , we don't just build data solutions-we shape the future of data-driven decision-making. Do you thrive on solving complex challenges and turning data into powerful insights that drive transformation? We're looking for Data Engineers who have a broad range of data engineering skills, with … a focus on Microsoft Fabric . ABOUT THE ROLE As a Data Engineer, you will play a key role in delivering high-quality data solutions for our clients. This role offers the opportunity to work across a variety of technologies and contribute to the growth of our Consulting practice. ROLE RESPONSIBILITIES Develop strong working relationships with … clients on projects, demonstrating interpersonal skills with both business and technical colleagues. Experience in developing performant end-to-end data solutions in a collaborative team environment. Deliver high-quality work, escalate problems when necessary, and propose solutions. Support the growth of the Consulting practice through contributions to initiatives, knowledge sharing, and data services. ESSENTIAL TECHNICAL SKILLS More ❯

to technology, delivering bespoke software solutions and expert advice. Our clients are increasingly looking to us to help them make the best use of their data. In building data platforms and pipelines, our data engineers create the foundation for diverse data & analytics solutions, including data science and AI. They build data lakes and warehouses, create the processes to extract or access operational data, and transform siloed datasets into integrated data models that allow insight into business performance and problems or training of ML models. These are hands-on, client-facing roles, with openings at senior or lead level to suit your experience. You may be … or solving tough engineering challenges. You'd also be expected to spend some time on-site with clients in the London area on an ad-hoc basis. Our data engineers combine a strong software engineering approach with solid data fundamentals and experience with modern tools. We're technology agnostic, and we're open minded when it More ❯

Senior Data Engineer - London/Hertfordshire - Hybrid Are you a technically strong Azure Data Engineer looking to join a forward-thinking, innovative company? Here's your chance to work with a leading construction and infrastructure company who foster a culture of collaboration, support, and strong communication. You'll be responsible for ensuring seamless integration and optimization … of diverse business data streams - from CRM, finance, procurement to fleet and commercial data - enabling reliable, high-quality data for analytics and business insights. While this role won't be client-facing, strong communication and collaboration skills within their agile team are essential. Requirements: Experience orchestrating … data pipelines with Azure Data Factory (ADF) Proficiency managing API data ingestion and orchestration using Azure Databricks Working with Azure Data Lake (ADL) Storage, on-prem SQL, and Python Vast Azure data engineering experience Proficient in ADF, Databricks, ADL, SQL, Python Desirable: Exposure to D365 or similar CRM environments More ❯

support EQ's transition into a stronger, more profitable, product-led business, driven by real insights and the needs of its customers. We are looking for a skilled Data Engineer to join our Data team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data pipelines … cloud-based data pipelines and architectures Strong expertise in Microsoft Fabric and Databricks, including data pipeline development, data warehousing, and data lake management Proficiency in Python, SQL, Scala, or Java Experience with data processing frameworks such as Apache Spark, Apache Beam, or Azure Data Factory Strong understanding … Azure and or AWS Strong collaboration and communication skills, with the ability to work effectively with cross-functional teams Desirable: Experience with Azure Synapse Analytics, Azure Data Lake Storage, or other Azure data services Experience with agile development methodologies and version control systems such as Git Certification in Microsoft Azure, Databricks, or other relevant technologies More ❯

Robert Half Technology are assisting a market leading financial services organisation to recruit a Junior Data Engineer. Hybrid working - London based Role The Junior Data Engineer will assist in the design, development, and maintenance of scalable data pipelines and ETL/ELT processes in Azure. Write efficient and reliable SQL queries for data extraction, transformation, and analysis. Support data integration from various sources (internal systems, third-party vendors) into centralised data platforms. Monitor and troubleshoot data jobs and pipelines to ensure consistent and reliable delivery. Collaborate with analytics and reporting teams to deliver high-quality datasets for business intelligence. Participate in code reviews, documentation, and … querying, data manipulation, and performance tuning. Hands-on experience with Azure Data Services (such as Azure Data Factory, Azure Synapse, Data Lake, or Logic Apps). Strong problem-solving skills and attention to detail. Excellent communication skills and a collaborative mindset. Nice To Have Experience with Python for data More ❯

Data Solutions Architect – Any Cloud (Banking) Location: London, United Kingdom (Flexible Hybrid Working) Type: Permanent, Full-Time A leading global consultancy is seeking an experienced Data Solutions Architect to join its growing Data Management practice. This is a strategic, client-facing role focused on modernising data platforms within the banking sector , enabling … large-scale cloud transformation and data-driven decision-making. You will lead architecture design, influence enterprise data strategies, and guide technical delivery across cloud ecosystems like Azure , AWS , or GCP . This role also involves significant pre-sales engagement—translating client requirements into solution blueprints and supporting proposal development. Key Responsibilities Architect and oversee the delivery … of enterprise-scale data platforms (data lakes, lakehouses, warehouses) using tools like Databricks , Snowflake , Synapse , and Azure Fabric Define and execute cloud migration strategies, leveraging CI/CD pipelines, Terraform, Azure DevOps, and GitHub Support RFP/RFI responses, conduct client workshops, and shape pre-sales proposals (40–50% of the role) Advise senior client stakeholders More ❯

Data Solutions Architect – Any Cloud (Banking) Location: London, United Kingdom (Flexible Hybrid Working) Type: Permanent, Full-Time A leading global consultancy is seeking an experienced Data Solutions Architect to join its growing Data Management practice. This is a strategic, client-facing role focused on modernising data platforms within the banking sector , enabling … large-scale cloud transformation and data-driven decision-making. You will lead architecture design, influence enterprise data strategies, and guide technical delivery across cloud ecosystems like Azure , AWS , or GCP . This role also involves significant pre-sales engagement—translating client requirements into solution blueprints and supporting proposal development. Key Responsibilities Architect and oversee the delivery … of enterprise-scale data platforms (data lakes, lakehouses, warehouses) using tools like Databricks , Snowflake , Synapse , and Azure Fabric Define and execute cloud migration strategies, leveraging CI/CD pipelines, Terraform, Azure DevOps, and GitHub Support RFP/RFI responses, conduct client workshops, and shape pre-sales proposals (40–50% of the role) Advise senior client stakeholders More ❯

Job Description Data Solutions Architect Any Cloud (Banking) Location: London, United Kingdom (Flexible Hybrid Working) Type: Permanent, Full-Time A leading global consultancy is seeking an experienced Data Solutions Architect to join its growing Data Management practice. This is a strategic, client-facing role focused on modernising data platforms within the banking … sector , enabling large-scale cloud transformation and data-driven decision-making. You will lead architecture design, influence enterprise data strategies, and guide technical delivery across cloud ecosystems like Azure , AWS , or GCP . This role also involves significant pre-sales engagementtranslating client requirements into solution blueprints and supporting proposal development. Key Responsibilities Architect and oversee the … delivery of enterprise-scale data platforms (data lakes, lakehouses, warehouses) using tools like Databricks , Snowflake , Synapse , and Azure Fabric Define and execute cloud migration strategies, leveraging CI/CD pipelines, Terraform, Azure DevOps, and GitHub Support RFP/RFI responses, conduct client workshops, and shape pre-sales proposals (4050% of the role) Advise senior client stakeholders More ❯

Lead Data Engineer - Manchester - Hybrid - £75k - £80k This is an excellent opportunity for a highly experienced data engineer to join a growing tech consultancy and contribute to the leadership of their data engineering practice. If you are an experienced and capable data engineer with a history of using Microsoft products and working … of clients to gather requirements and deliver solutions. Be willing to engage and assist in pre-sales activities, bids, proposals etc. Use key techniques such as Governance, Architecture, Data Modelling, ETL/ELT, Data Lakes, Data Warehousing, Master Data, and BI. Consistently utilise key processes in the engineering delivery cycle including Agile … and DevOps, Git, APIs, Containers, Microservices and Data Pipelines. Assist in building out, developing, and training the data engineering function. What do I need to apply Strong MS data engineering expertise (Fabric, ADF, ADL, Synapse, SQL) Expert use of Databricks Strong Python experience Consultancy experience Leadership experience My client are looking to book in More ❯

Lead Data Engineer - Manchester - Hybrid - £75k - £80k This is an excellent opportunity for a highly experienced data engineer to join a growing tech consultancy and contribute to the leadership of their data engineering practice. If you are an experienced and capable data engineer with a history of using Microsoft products and working … of clients to gather requirements and deliver solutions. Be willing to engage and assist in pre-sales activities, bids, proposals etc. Use key techniques such as Governance, Architecture, Data Modelling, ETL/ELT, Data Lakes, Data Warehousing, Master Data, and BI. Consistently utilise key processes in the engineering delivery cycle including Agile … and DevOps, Git, APIs, Containers, Microservices and Data Pipelines. Assist in building out, developing, and training the data engineering function. What do I need to apply Strong MS data engineering expertise (Fabric, ADF, ADL, Synapse, SQL) Expert use of Databricks Strong Python experience Consultancy experience Leadership experience My client are looking to book in More ❯

Job Description Role : Data Architect (GCP) Location : London Salary : £80,000 to £130,000 (Multiple grades) + bonus & benefits We are supporting a global technology services provider in the strategic expansion of their Google Cloud Data and AI practice, with multiple opportunities available for experienced professionals across architecture and engineering. This unit is driving major transformation … initiatives with flagship enterprise clients, with a focus on intelligent data platforms, AI-driven analytics, and scalable cloud- solutions. Role Overview This is a strategic and hands-on opportunity for an experienced Data Architect to lead the development of a robust, scalable data architecture on Google Cloud Platform (GCP). You’ll be responsible … for defining the platform strategy, leading a team of data engineers, and ensuring high standards of data quality, compliance, and performance across enterprise-grade data environments. Key Responsibilities: Design and implement enterprise-level data architecture strategies on GCP. Gather client requirements and translate business challenges into scalable data solutions. Act More ❯

your job looks like You are part of the IT Data & AI Team and play a key role in shaping, implementing, and enhancing our Data Lake using Azure Databricks Source Integration : Establish connections to diverse systems via APIs , ODBC , Event Hubs , and sFTP protocols. ETL/ELT Pipelines : Design and optimize data pipelines … using Azure Data Factory and Databricks Medallion Architecture : Implement Bronze, Silver, and Gold layers using formats like Delta , Parquet , and JSON for data transformation. Data Modeling : Develop and optimize data models using star schema and slowly changing dimensions for analytics and operations. Data Governance : Ensure robust data security … Based Access Control (RBAC) and Unity Catalog . Performance Optimization : Tune pipelines, optimize queries, and implement CI/CD pipelines using Azure DevOps . Collaboration : Work closely with data scientists, IT team, and business stakeholders to deliver end-to-end solutions. As Azure Sr. Data Engineer you'll collaborate with our partners and BME, our parent More ❯

Get AI-powered advice on this job and more exclusive features. Direct message the job poster from Tenth Revolution Group Data & Analytics Recruitment Specialist | Connecting Top Data Talent with Innovative Organisations in the UK Data Solutions Architect – Any Cloud (Banking) Location: London, United Kingdom (Flexible Hybrid Working) Type: Permanent, Full-Time A leading global … consultancy is seeking an experienced Data Solutions Architect to join its growing Data Management practice. This is a strategic, client-facing role focused on modernising data platforms within the banking sector , enabling large-scale cloud transformation and data-driven decision-making. You will lead architecture design, influence enterprise data strategies … This role also involves significant pre-sales engagement—translating client requirements into solution blueprints and supporting proposal development. Key Responsibilities Architect and oversee the delivery of enterprise-scale data platforms (data lakes, lakehouses, warehouses) using tools like Databricks , Snowflake , Synapse , and Azure Fabric Define and execute cloud migration strategies, leveraging CI/CD pipelines, Terraform, Azure More ❯

Senior Azure Data Engineer Remote Working - UK Home-based with occasional travel into the office 39,784 - 49,477 (National) 45,456- 55,149 (London within the M25) Additional allowance for exceptional candidates - Max Salary - 62,000 (National) and 67,000 (London within the M25) Homeworking allowance of 581 per annum Job Ref: J12964 The role of the … Senior Azure Data Engineer is to design, build, test and maintain data on the enterprise data platform, allowing accessibility of the data that meets business and end user needs. The role holder will be responsible for maximising the automations, scalability, reliability and security of data services, focusing on opportunities for … re-use, adaptation and efficient engineering. Accountabilities: Design, build and test data pipelines and services, based on feeds from multiple systems using a range of different storage technologies and/or access methods provided by the Enterprise Data Platform, with a focus on creating repeatable and reusable components and products. Design, write and iterate code from More ❯