1 to 25 of 280 Data Lake Jobs in the UK

seeking a meticulous and experienced Data Architect to play a crucial role in designing, documenting, and ensuring the quality of our next-generation media data lake on the Microsoft Azure platform. This role will focus on developing and upholding the architectural integrity of the data lake, with a strong emphasis on the … is scalable, reliable, and delivers high-quality data. WHAT YOU WILL DO • Design and document significant components of the end-to-end architecture for the media data lake on Azure, ensuring alignment with the medallion architecture (Bronze, Silver, Gold layers). • Develop, implement, and oversee Quality Assurance strategies, test plans, and processes for data pipelines … Client Experiences by Cultivating Trust. • Lead Optimistically by Championing Growth and Development to Mobilize the Enterprise. • Extensive experience with Microsoft Azure data services (Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, Azure Purview). • Deep understanding and practical experience in designing and implementing medallion data lake architectures. • Strong More ❯

About Us: ATREIDES is a leader in advanced geospatial technologies, big data solutions, and data analytics. We specialize in providing cutting-edge software platforms that enable our clients to process, analyze, and visualize large-scale geospatial and sensor data for critical decision-making. Our mission is to deliver innovative, scalable, and high-performance platforms … that help organizations unlock the full potential of their data for various industries, including defense, national security, environmental monitoring, and urban planning. We are seeking a highly skilled Senior Data Engineer with a strong focus on big data, data analytics, and geospatial data intelligence to join our engineering team. In … this role, you will help design and develop robust, scalable platforms that handle vast amounts of geospatial data and provide actionable insights through data analytics. Job Overview: As a Senior Data Engineer at ATREIDES, you will be responsible for developing and optimizing our core software platform, with an emphasis on integrating and processing big More ❯

Role Summary and Impact We are looking for a detail-oriented and analytical Technical Analyst to join our team building a state-of-the-art media data lake on Azure. This role will bridge the gap between business requirements and technical implementation, working closely with stakeholders, data engineers, and data architects. The Technical … solutions. Proficiency in SQL for data querying and analysis. Experience with data modelling concepts and data warehousing principles. Familiarity with data lake concepts and architectures, preferably medallion architecture. Excellent communication, documentation, and interpersonal skills. Ability to work effectively with both technical and non-technical stakeholders. Experience with data visualization … tools (e.g., Power BI, Tableau) is a plus. Experience working with Azure data services (e.g., Azure Data Lake Storage, Azure Databricks). Understanding of ETL/ELT processes. Familiarity with Agile development methodologies. Knowledge of the media or advertising industry. Life at WPP Media & Benefits Our passion for shaping the next era of media includes More ❯

optimising for performance, scalability, and security. Collaborate with business stakeholders, data engineers, and analytics teams to ensure solutions are fit for purpose. Implement and optimise Databricks Delta Lake, Medallion Architecture, and Lakehouse patterns for structured and semi-structured data. Ensure best practices in Azure networking, security, and federated data access . Key Skills & Experience 5+ … optimising for performance, scalability, and security. Collaborate with business stakeholders, data engineers, and analytics teams to ensure solutions are fit for purpose. Implement and optimise Databricks Delta Lake, Medallion Architecture, and Lakehouse patterns for structured and semi-structured data. Ensure best practices in Azure networking, security, and federated data access . Key Skills & Experience 5+ … years of experience in data architecture, solution architecture, or cloud data engineering roles. Strong expertise in Azure data services , including: Azure Databricks (Delta Lake, Unity Catalog, MLflow) Azure Event Hub and Azure Data Explorer for real-time and streaming data pipelines Azure Storage (ADLS Gen2, Blob Storage) for scalable More ❯

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data Mesh, lakehouse, data vault and data warehouses. Our data engineers create pipelines that support our data scientists and power our front-end applications. This means we do data-intensive work for both OLTP and OLAP use cases. Our environments are primarily cloud-native spanning AWS, Azure and GCP More ❯

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data Mesh, lakehouse, data vault and data warehouses. Our data engineers create pipelines that support our data scientists and power our front-end applications. This means we do data-intensive work for both OLTP and OLAP use cases. Our environments are primarily cloud-native spanning AWS, Azure and GCP More ❯

condition, or are neurodiverse. If you'd like to apply and need adjustments made, you can let us know in your application. About the role As an Azure Data Engineer, your role involves designing, implementing, and managing data solutions on the Microsoft Azure cloud platform. Your responsibilities will include, but not limited to: Data … grow! Data Management and Storage: You'll work with various data storage options provided by Microsoft Azure, including Azure SQL Database, Azure Data Lake Storage, and Azure Blob Storage. Your task is to design data storage systems that scale, perform efficiently, and meet the organisation's requirements. Essential Skills Proficiency in … SQL, SQL Server Understand parallel processing and data architecture patterns Azure Data Factory (ADF) Azure Synapse Analytics Azure Data Lake Storage Ability to design and implement pipelines in ADF. Build complex efficient processes for transformation using SQL & ADF Pipelines Ensure secure, organised, and reliable data pipelines Troubleshoot all data More ❯

ADF Data Engineer ADF Data Engineer - SQL Server 2022, ADF, ETL Pipelines, Azure Synapse Analytics, Azure SQL, Azure Data Lake, SSIS, SSAS, SSRS, Power BI, Fabric, Data Warehousing, Agile - Hybrid (Warrington) - £50k + excellent bonus + excellent bens An experienced ADF Data Engineer is urgently required by my client … with strong experience of the Microsoft BI stack and Azure Services (SQL Server 2022, ADF, ETL Pipelines, Azure Synapse Analytics, Azure SQL, Azure Data Lake, SSIS, SSAS, SSRS, Power BI, Fabric). You will be responsible for helping to build a new, best practice-based data warehouse solution which will function Enterprise-wide. You will … role on a product development programme using the Microsoft BI stack and Azure Services (SQL Server 2022, ADF, ETL Pipelines, Azure Synapse Analytics, Azure SQL, Azure Data Lake, SSIS, SSAS, SSRS, Power BI, Fabric) - Work collaboratively within an Agile team of Developers, Testers and BA/Product Owners on a set of new greenfield projects - Observe object More ❯

Key Responsibilities: Data Pipelines and ETL Processes ETL Design: Design and implement ETL processes within MaPS architectural patterns to extract, transform, and load data from various source systems into our reporting solutions. Pipeline Development: Develop and configure meta-data driven data pipelines using data orchestration tools such as Azure Data … unusual data profiles and establish recovery processes to maintain data integrity. Azure Services Utilise Azure Cloud based data technology platforms, Data Lake based storage & Cloud data pipeline/orchestration tools for efficient data storage, processing, and transfer. Ensure the scalability and reliability of data solutions … implementing ETL processes. Experience in using SQL to query and manipulate data. Proficiency in Azure data tooling such as Synapse Analytics, Microsoft Fabric, Azure Data Lake Storage/One Lake, and Azure Data Factory. Understanding of data extraction from vendor REST APIs. Spark/Pyspark or Python skills a bonus More ❯

ll be part of a diverse, ambitious team that celebrates creativity and collaboration. Ready to make your mark? Join us and be part of something bigger. As a Data Architect (Manager) in Kainos, you'll be responsible for providing SME guidance in traditional data architecture disciplines around data structures, data flows, data sourcing and data governance. Data Architects work closely with clients to understand their data requirements and take responsibility for ensuring solutions are fit for purpose. They also provide technical leadership for the rest of the team in the area of data. Data Architects may also work at the solution or … defining information handling models and capacity planning across heterogeneous data store technologies. Experience of establishing data governance processes Experience of architecting a data lake and solutions that reside within a data lake ecosystem Enterprise Data Architecture experience. Embracing our differences At Kainos, we believe in the power of More ❯

Senior Data Engineer - (Azure/Databricks) page is loaded Senior Data Engineer - (Azure/Databricks) Apply locations London - Scalpel time type Full time posted on Posted 7 Days Ago job requisition id REQ05851 This is your opportunity to join AXIS Capital - a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our … data solutions are secure, efficient, and optimized. Key Responsibilities: Design and implement data solutions using Azure services, including Azure Databricks, ADF, and Data Lake Storage. Develop and maintain ETL/ELT pipelines to process structured and unstructured data from multiple sources. - Automate loads using Databricks workflows and Jobs Develop, test and … warehousing, and real-time streaming. Knowledge of developing and processing full and incremental loads. Experience of automated loads using Databricks workflows and Jobs Expertise in Azure Databricks, including Delta Lake, Spark optimizations, and MLflow. Strong experience with Azure Data Factory (ADF) for data integration and orchestration. Hands-on experience with Azure DevOps, including pipelines, repos More ❯

Senior Data Ops Data EngineerLeeds 2/3 days per week£55,000-£67,000+ Excellent Benefits18 Months FTC with long term potential Your New Company Our large public sector organisation are seeking a senior Data Platform DataOps Engineer to serve as our clients first DataOps specialist in a growing team of Data … and DevOps teams to align operational strategies with technical and business requirements. Optimize operational performance and cost management for services including Azure Data Factory, Azure Databricks, Delta Lake, and Azure Data Lake Storage. Serve as the domain expert in DataOps by providing strategic guidance, mentoring colleagues, and driving continuous process improvements. What you will … within the Microsoft Azure ecosystem. Hands-on expertise with Azure Data Platform with components such as Azure Data Factory, Azure Databricks, Azure Data Lake Storage, Delta Lake, Azure SQL, Purview and APIM. Proficiency in developing CI/CD data pipelines and strong programming skills in Python, SQL, Bash, and PySpark More ❯

expanding the existing mission critical ecosystem of Data and BI systems for analytics and decision making, using Primarily Data Warehouse/Mart, Data Lake, Self Service BI Tools and Data Science/ML platform using Oracle, Snowflake and AWS tech stack. Qualifications We are looking for a skilled Lead Data … pipeline/ELT development, including performance tuning Very good SQL skills Past experience and solid understanding of end to end Data Warehouse, Data Lake and BI design and delivery at scale (high level and detailed) and various architectural strategies Solid information architecture skills/experience: Data ingestions, and Data Warehouse and Date Lake modelling experience (Star Schema, Snowflake Schema, Data Vault etc.) Past hands-on development experience in at least one enterprise analytics database like Oracle/Terradata/SQL Server/Snowflake Understanding of AWS cloud and fundamental services like IAM, Lambda, S3 etc. Experience of liaising with both in-house and third More ❯

expanding the existing mission critical ecosystem of Data and BI systems for analytics and decision making, using Primarily Data Warehouse/Mart, Data Lake, Self Service BI Tools and Data Science/ML platform using Oracle, Snowflake and AWS tech stack. Qualifications We are looking for a skilled Lead Data … pipeline/ELT development, including performance tuning Very good SQL skills Past experience and solid understanding of end to end Data Warehouse, Data Lake and BI design and delivery at scale (high level and detailed) and various architectural strategies Solid information architecture skills/experience: Data ingestions, and Data Warehouse and Date Lake modelling experience (Star Schema, Snowflake Schema, Data Vault etc.) Past hands-on development experience in at least one enterprise analytics database like Oracle/Terradata/SQL Server/Snowflake Understanding of AWS cloud and fundamental services like IAM, Lambda, S3 etc. Experience of liaising with both in-house and third More ❯

ll be part of a diverse, ambitious team that celebrates creativity and collaboration. Ready to make your mark? Join us and be part of something bigger. As a Data Architect (Manager) in Kainos, you'll be responsible for providing SME guidance in traditional data architecture disciplines around data structures, data flows, data sourcing and data governance. Data Architects work closely with clients to understand their data requirements and take responsibility for ensuring solutions are fit for purpose. They also provide technical leadership for the rest of the team in the area of data. Data Architects may also work at the solution or … defining information handling models and capacity planning across heterogeneous data store technologies. Experience of establishing data governance processes Experience of architecting a data lake and solutions that reside within a data lake ecosystem At Kainos, we believe in the power of diversity, equity and inclusion. We are committed to building More ❯

ll be part of a diverse, ambitious team that celebrates creativity and collaboration. Ready to make your mark? Join us and be part of something bigger. As a Data Architect (Manager) in Kainos, you'll be responsible for providing SME guidance in traditional data architecture disciplines around data structures, data flows, data sourcing and data governance. Data Architects work closely with clients to understand their data requirements and take responsibility for ensuring solutions are fit for purpose. They also provide technical leadership for the rest of the team in the area of data. Data Architects may also work at the solution or … defining information handling models and capacity planning across heterogeneous data store technologies. Experience of establishing data governance processes Experience of architecting a data lake and solutions that reside within a data lake ecosystem At Kainos, we believe in the power of diversity, equity and inclusion. We are committed to building More ❯

ll be part of a diverse, ambitious team that celebrates creativity and collaboration. Ready to make your mark? Join us and be part of something bigger. As a Data Architect (Manager) in Kainos, you'll be responsible for providing SME guidance in traditional data architecture disciplines around data structures, data flows, data sourcing and data governance. Data Architects work closely with clients to understand their data requirements and take responsibility for ensuring solutions are fit for purpose. They also provide technical leadership for the rest of the team in the area of data. Data Architects may also work at the solution or … defining information handling models and capacity planning across heterogeneous data store technologies. Experience of establishing data governance processes Experience of architecting a data lake and solutions that reside within a data lake ecosystem At Kainos, we believe in the power of diversity, equity and inclusion. We are committed to building More ❯

is a strong culture where people are encouraged to be present, bold and curious, allowing them to achieve their individual goals. We are currently looking for a Lead Data Architect, to be based in London on a full-time basis, reporting into the Data & Transformation Strategy Lead. Responsibilities: Collaborate with business stakeholders, data scientists … analysts, and IT teams end to end to understand business needs and translate them into effective data architecture solutions. Design and implement scalable and efficient data architectures that support business goals, including data warehousing, data lakes, and real-time data processing. Lead the evaluation, selection, and integration of new data technologies and tools to optimise data management processes. Support the development of data products, through robust architecture, data governance policies, ensuring data quality, integrity, security, and compliance with relevant regulations (GDPR). Lead a team to develop and maintain data models, schemas, and data dictionaries to More ❯

Engineer to join our Data Platform Team in the Data Intelligence Camp. You'll build, enhance and optimise a key data lake platform that will enable storage and preparation of data across billions of data points, powering analytics platforms and reporting systems throughout our organization and helping … data, platform engineering, and product development. As part of this team of amazing humans, You will Architect for scale and performance : Design and implement Data Lake platforms that support storage and processing of billions of rows, while maintaining cost efficiency and a pragmatic approach to delivery of solutions and architecture. Drive platform evolution : Lead technical … decisions that shape Data Lake platforms, balancing performance, security, and developer experience. Partner with product teams : Collaborate closely with product teams and engineering teams, data science teams, product managers, key vendors, security and privacy experts to understand their data needs and requirements and agree on solutions to build. Champion technical excellence : Drive standards More ❯

Engineer to join our Data Platform Team in the Data Intelligence Camp. You'll build, enhance and optimise a key data lake platform that will enable storage and preparation of data across billions of data points, powering analytics platforms and reporting systems throughout our organization and helping … data, platform engineering, and product development. As part of this team of amazing humans, You will Architect for scale and performance : Design and implement Data Lake platforms that support storage and processing of billions of rows, while maintaining cost efficiency and a pragmatic approach to delivery of solutions and architecture. Drive platform evolution : Lead technical … decisions that shape Data Lake platforms, balancing performance, security, and developer experience. Partner with product teams : Collaborate closely with product teams and engineering teams, data science teams, product managers, key vendors, security and privacy experts to understand their data needs and requirements and agree on solutions to build. Champion technical excellence : Drive standards More ❯

Engineer to join our Data Platform Team in the Data Intelligence Camp. You'll build, enhance and optimise a key data lake platform that will enable storage and preparation of data across billions of data points, powering analytics platforms and reporting systems throughout our organization and helping … data, platform engineering, and product development. As part of this team of amazing humans, You will Architect for scale and performance : Design and implement Data Lake platforms that support storage and processing of billions of rows, while maintaining cost efficiency and a pragmatic approach to delivery of solutions and architecture. Drive platform evolution : Lead technical … decisions that shape Data Lake platforms, balancing performance, security, and developer experience. Partner with product teams : Collaborate closely with product teams and engineering teams, data science teams, product managers, key vendors, security and privacy experts to understand their data needs and requirements and agree on solutions to build. Champion technical excellence : Drive standards More ❯

Data Lead/BI Business Intelligence & Insights Manager/Principal Data Engineer who has proven leadership in a technology environment, managing teams of Data Engineers, BI Developers and Data Analysts is required for a well-established company based in Bedford, Bedfordshire. IS THIS YOU? Can you lead the companys data … be based in either UK or Portugal Local Salary Ranges Will Apply JOB TYPE: Full-Time, Permanent JOB OVERVIEW We have a fantastic new job opportunity for a Data Lead/BI Business Intelligence & Insights Manager/Principal Data Engineer who has proven leadership in a technology environment, managing teams of Data Engineers, BI … approach. Experience in SaaS or platform-based businesses is a plus Experience in data roles (data engineering, BI, analytics or architecture). Data lake storage, SQL database, NoSQL Sound understanding of database architecture and relevant underpinning concepts, including relational data modelling, frameworks, warehouse and data lake design, performance More ❯

Data Lead/BI Business Intelligence & Insights Manager/Principal Data Engineer who has proven leadership in a technology environment, managing teams of Data Engineers, BI Developers and Data Analysts is required for a well-established company based in Bedford, Bedfordshire. IS THIS YOU? Can you lead the companys data … be based in either UK or Portugal Local Salary Ranges Will Apply JOB TYPE: Full-Time, Permanent JOB OVERVIEW We have a fantastic new job opportunity for a Data Lead/BI Business Intelligence & Insights Manager/Principal Data Engineer who has proven leadership in a technology environment, managing teams of Data Engineers, BI … approach. Experience in SaaS or platform-based businesses is a plus Experience in data roles (data engineering, BI, analytics or architecture). Data lake storage, SQL database, NoSQL Sound understanding of database architecture and relevant underpinning concepts, including relational data modelling, frameworks, warehouse and data lake design, performance More ❯

Data Lead/BI Business Intelligence & Insights Manager/Principal Data Engineer who has proven leadership in a technology environment, managing teams of Data Engineers, BI Developers and Data Analysts is required for a well-established company based in Bedford, Bedfordshire. IS THIS YOU? Can you lead the companys data … be based in either UK or Portugal Local Salary Ranges Will Apply JOB TYPE: Full-Time, Permanent JOB OVERVIEW We have a fantastic new job opportunity for a Data Lead/BI Business Intelligence & Insights Manager/Principal Data Engineer who has proven leadership in a technology environment, managing teams of Data Engineers, BI … approach. Experience in SaaS or platform-based businesses is a plus Experience in data roles (data engineering, BI, analytics or architecture). Data lake storage, SQL database, NoSQL Sound understanding of database architecture and relevant underpinning concepts, including relational data modelling, frameworks, warehouse and data lake design, performance More ❯

Data Lead/BI Business Intelligence & Insights Manager/Principal Data Engineer who has proven leadership in a technology environment, managing teams of Data Engineers, BI Developers and Data Analysts is required for a well-established company based in Bedford, Bedfordshire. IS THIS YOU? Can you lead the company’s data … be based in either UK or Portugal – Local Salary Ranges Will Apply JOB TYPE: Full-Time, Permanent JOB OVERVIEW We have a fantastic new job opportunity for a Data Lead/BI Business Intelligence & Insights Manager/Principal Data Engineer who has proven leadership in a technology environment, managing teams of Data Engineers, BI … approach. Experience in SaaS or platform-based businesses is a plus Experience in data roles (data engineering, BI, analytics or architecture). Data lake storage, SQL database, NoSQL Sound understanding of database architecture and relevant underpinning concepts, including relational data modelling, frameworks, warehouse and data lake design, performance More ❯