76 to 100 of 146 Data Lake Jobs in England

superior financial strength ratings from each of the major rating agencies. General Reinsurance, London Branch (a subsidiary of General Reinsurance AG) is offering an opportunity for a Senior Data Engineer in our Data and Governance domain. Main Purpose of the Role We are seeking a detail-oriented senior data engineer to support the implementation … of our Data and AI Governance initiatives that reports into the Enterprise Data and AI Services (EDAS) team. This role will play a key part in supporting the team of governance processes, audits, managing risks and ensuring data privacy in close collaboration with the Security, Legal and Privacy teams. The post holder will also … contribute to the Implementation, integration, development and maintenance of relevant governance, MDM and lineage tools. The ideal candidate will have a strong foundation in data engineering and report configurations in a modern landscape and a keen interest in expanding their expertise in governance, audit, and privacy. This role offers the opportunity to gain hands-on experience in applying More ❯

Technology Product Owner, AI Data Analytics, Microsoft Stack, Azure, Data Engineering, ML, Azure, Mainly Remote Technology Product Owner required to join a global Professional Services business based in Central London. However, this is practically a remote role, but when travel is required (to London, Europe and the States on occasions. We need someone who has come … background with a hardened skillset in Microsoft Stack Technologies (C# .NET Core) who has then transitioned into Product Ownership. We need someone highly analytical who can understand large Data Sets, Data Bricks and is able to bring Proof of Concepts to the table and help with the execution. The platform primarily serves two key personas: Data … for more details... Experience required: Development FIRST mentality. You must have been a Developer before moving into Product Ownership Technical proficiency: Familiarity with Azure services (e.g., Data Lake, Synapse, Fabric) and Databricks for data engineering, analytics, performance optimisation, and governance. Experience with implementing and optimising scalable cloud infrastructure is highly valued. Backlog management: Demonstrated expertise More ❯

Technology Product Owner, AI Data Analytics, Microsoft Stack, Azure, Data Engineering, ML, Azure, Remote with Travel Technology Product Owner required to join a global Professional Services business based in Central London. However, this is practically a remote role, but when travel is required (to London, Europe and the States on occasions. We need someone who has … background with a hardened skillset in Microsoft Stack Technologies (C# .NET Core) who has then transitioned into Product Ownership. We need someone highly analytical who can understand large Data Sets, Data Bricks and is able to bring Proof of Concepts to the table and help with the execution. The platform primarily serves two key personas: Data … for more details Experience required: Development FIRST mentality. You must have been a Developer before moving into Product Ownership Technical proficiency:Familiarity with Azure services (e.g., Data Lake, Synapse, Fabric) and Databricks for data engineering, analytics, performance optimisation, and governance. Experience with implementing and optimising scalable cloud infrastructure is highly valued. Backlog management:Demonstrated expertise More ❯

Technical Lead - Data - £85,000 Location: UK - Remote Employment Type: Full-time, Permanent About the Role A Microsoft Partnered Consultancy is looking for a Technical Data Team Lead - to join a dynamic, agile team delivering innovative data solutions for clients. This is a hands-on leadership role where you'll mentor technical consultants, oversee … project delivery, and contribute to designing scalable data solutions in cloud environments. Key Responsibilities: Lead and mentor a team of technical consultants, driving engagement and growth. Manage resource planning, scheduling, and performance development. Collaborate with pre-sales and project teams to scope and deliver data projects. Support technical delivery, including cloud migrations, data lakes … using Microsoft/Azure technologies (Fabric, Databricks, Synapse, Data Factory, SQL Server). Expertise in ETL/ELT processes, SQL, and Python. Knowledge of data lake implementation and medallion lakehouse architecture. Solid understanding of BI, data warehousing concepts, and optimization techniques for large-scale solutions. What's on Offer Competitive salary and benefits More ❯

Technical Lead - Data - £85,000 Location: UK - Remote Employment Type: Full-time, Permanent About the Role A Microsoft Partnered Consultancy is looking for a Technical Data Team Lead - to join a dynamic, agile team delivering innovative data solutions for clients. This is a hands-on leadership role where you'll mentor technical consultants, oversee … project delivery, and contribute to designing scalable data solutions in cloud environments. Key Responsibilities: Lead and mentor a team of technical consultants, driving engagement and growth. Manage resource planning, scheduling, and performance development. Collaborate with pre-sales and project teams to scope and deliver data projects. Support technical delivery, including cloud migrations, data lakes … using Microsoft/Azure technologies (Fabric, Databricks, Synapse, Data Factory, SQL Server). Expertise in ETL/ELT processes, SQL, and Python. Knowledge of data lake implementation and medallion lakehouse architecture. Solid understanding of BI, data warehousing concepts, and optimization techniques for large-scale solutions. What's on Offer Competitive salary and benefits More ❯

Senior Data Engineer (Snowflake SQL Python) London/WFH to £85k Are you a skilled data technologist with strong leadership and stakeholder management skills? You could be progressing your career in a senior, hands-on Data Engineer position at a global tech company that provide data centric software solutions to major blue … chip and government organisations to enable them to discover and analyse data and customer feedback. As a Senior Data Engineer you will plan and lead data engineering activities across multiple programmes of work to deliver secure, robust and scalable data engineering solutions for complex data analytics products. You'll implement … modern data engineering practices, build complex data pipelines and provide guidance to other team members to ensure optimal code performance is achieved, championing best practices. Beyond this you'll seek to monetise the database, collaborating closely with business leaders. Location/WFH: You can work from home most of the time, meeting up with colleagues in More ❯

Interim Data Architect - £700 per day (Outside IR35) Central London | Creative Agency | ASAP Start A leading creative agency in Central London is looking for an experienced Interim Data Architect to … help shape and deliver their data roadmap. You'll be responsible for connecting and optimising core business systems (NetSuite, BambooHR, SugarCRM, etc.), building a data lake , and supporting plans to implement Microsoft Fabric . This is a hands-on role where you'll work closely with a Data Analyst to cleanse and structure … business data, while also guiding and influencing senior stakeholders on best practice. What you'll need: Proven experience as a Data Architect in complex environments Strong knowledge of data lakes and modern data platforms Ability to influence senior leadership and drive data strategy Comfortable rolling up your sleeves and working More ❯

About the RoleWe are seeking a Databricks Data Engineer with strong expertise in designing and optimising large-scale data engineering solutions within the Databricks Data Intelligence Platform. This role is ideal for someone passionate about building high-performance data pipelines and ensuring robust data governance across … modern cloud environments.Key Responsibilities Design, build, and maintain scalable data pipelines using Databricks Notebooks, Jobs, and Workflows for both batch and streaming data. Optimise Spark and Delta Lake performance through efficient cluster configuration, adaptive query execution, and caching strategies. Conduct performance testing and cluster tuning to ensure cost-efficient, high-performing workloads. Implement data quality … with Databricks Unity Catalogue and governance best practices. Develop PySpark applications for ETL, data transformation, and analytics, following modular and reusable design principles. Create and manage Delta Lake tables with ACID compliance, schema evolution, and time travel for versioned data management. Integrate Databricks solutions with Azure services such as Azure Data Lake More ❯

About the RoleWe are seeking a Databricks Data Engineer with strong expertise in designing and optimising large-scale data engineering solutions within the Databricks Data Intelligence Platform. This role is ideal for someone passionate about building high-performance data pipelines and ensuring robust data governance across modern … cloud environments. Key Responsibilities Design, build, and maintain scalable data pipelines using Databricks Notebooks, Jobs, and Workflows for both batch and streaming data. Optimise Spark and Delta Lake performance through efficient cluster configuration, adaptive query execution, and caching strategies. Conduct performance testing and cluster tuning to ensure cost-efficient, high-performing workloads. Implement data quality … with Databricks Unity Catalogue and governance best practices. Develop PySpark applications for ETL, data transformation, and analytics, following modular and reusable design principles. Create and manage Delta Lake tables with ACID compliance, schema evolution, and time travel for versioned data management. Integrate Databricks solutions with Azure services such as Azure Data Lake More ❯

Data Engineering Consultant - £50,000 - Hybrid As a Data Engineer, you will work within an agile team to deliver high-value data solutions for clients. You'll be hands-on across the full lifecycle, using and developing your technical and project skills to create high-quality outcomes. While we follow a Winning from Anywhere … approach, some travel to client sites, company conferences, and events may be required. What You'll Deliver End-to-end data solutions covering acquisition, engineering, modelling, analysis, and visualisation. Client workshops and communication at both technical and business levels. Design and implementation of robust ETL/ELT solutions using the Microsoft/Azure ecosystem (Fabric/Databricks). … Server technologies. Skills in developing ETL/ELT pipelines using Azure Synapse, Data Factory, Databricks, or Fabric, with SQL and Python. Strong understanding of data lake and lakehouse architectures. Experience working with large, complex datasets from diverse sources. Strong SQL and Python capabilities, including stored procedures, notebooks, and query optimisation. Familiarity with Power BI, Analysis More ❯

Data Engineering Consultant - £50,000 - Hybrid As a Data Engineer, you will work within an agile team to deliver high-value data solutions for clients. You'll be hands-on across the full lifecycle, using and developing your technical and project skills to create high-quality outcomes. While we follow a Winning from Anywhere … approach, some travel to client sites, company conferences, and events may be required. What You'll Deliver End-to-end data solutions covering acquisition, engineering, modelling, analysis, and visualisation. Client workshops and communication at both technical and business levels. Design and implementation of robust ETL/ELT solutions using the Microsoft/Azure ecosystem (Fabric/Databricks). … Server technologies. Skills in developing ETL/ELT pipelines using Azure Synapse, Data Factory, Databricks, or Fabric, with SQL and Python. Strong understanding of data lake and lakehouse architectures. Experience working with large, complex datasets from diverse sources. Strong SQL and Python capabilities, including stored procedures, notebooks, and query optimisation. Familiarity with Power BI, Analysis More ❯

Senior Data Developer – Hedge Fund | New York (Hybrid) Are you an experienced data developer who loves building fast, reliable systems that fuel quantitative trading and risk management ? A global multi-strategy fund is expanding its New York data team to build the next generation of its market data platform , working directly with … PMs, and traders to power insights that drive investment returns. You’ll: Develop scalable ETL pipelines in Python for high-volume financial data. Build cloud-based data lake environments (AWS/Databricks). Implement robust data validation and quality checks. Work with researchers to design pipelines for modeling and back-testing. Document and maintain the … firm’s core data architecture. Requirements: 5+ years of Python development (Pandas, SQL, Linux environment). Hands-on experience building data solutions for trading, research, or risk. Deep technical curiosity and strong problem-solving skills. Strong knowledge of market data and financial time-series. Bachelor’s or higher in a quantitative or technical discipline. More ❯

We are looking for a skilled data engineer to help build and maintain our data infrastructure. The ideal candidate will have expertise working with cloud data platforms and architecting robust and scalable data pipelines. This role is for contractors Tasks As a Senior Data Engineer, you will work cross-functionally … to intake, process, transform, and store large amounts of data from various sources. You will build and maintain the core data infrastructure that powers the organisation's advanced analytics, machine learning applications, and data-driven decision-making. This role requires strong technical skills, problem-solving abilities, and excellent communication with stakeholders at all levels. … Requirements 10+ years of experience with building and maintaining data warehouses, data lakes, and other large-scale data storage systems Knowledge of ETL processes and data transformation techniques Ability to design and develop data aggregation and integration processes across different systems and environments Understanding of data architecture patterns More ❯

Data Engineering team lead - £70,000 - Hybrid Overview We're looking for a Lead Technical Consultant to guide a team of Technical Consultants, oversee delivery of high-quality data solutions, and collaborate across the business to ensure successful project outcomes. This role combines leadership, resource management, and hands-on technical expertise within Microsoft and Azure data … deliver successful projects. Technical Delivery & Innovation Support technical delivery when needed, including designing scalable Azure/Microsoft data solutions. Drive innovation through cloud migrations, data lake implementations, and robust ETL/ELT pipeline development. Skills & Experience You'll thrive in this role if you have: Leadership Proven ability to build strong relationships and foster an … inclusive, collaborative environment. Demonstrated leadership experience, acting as a role model for company values. Technical Expertise Strong background in Data Engineering or Data Warehouse development using Microsoft Fabric, Azure Databricks, Synapse, Data Factory, and/or SQL Server. Expertise in ETL/ELT design with SQL and Python. Solid understanding of data More ❯

Data Engineering team lead - £70,000 - Hybrid Overview We're looking for a Lead Technical Consultant to guide a team of Technical Consultants, oversee delivery of high-quality data solutions, and collaborate across the business to ensure successful project outcomes. This role combines leadership, resource management, and hands-on technical expertise within Microsoft and Azure data … deliver successful projects. Technical Delivery & Innovation Support technical delivery when needed, including designing scalable Azure/Microsoft data solutions. Drive innovation through cloud migrations, data lake implementations, and robust ETL/ELT pipeline development. Skills & Experience You'll thrive in this role if you have: Leadership Proven ability to build strong relationships and foster an … inclusive, collaborative environment. Demonstrated leadership experience, acting as a role model for company values. Technical Expertise Strong background in Data Engineering or Data Warehouse development using Microsoft Fabric, Azure Databricks, Synapse, Data Factory, and/or SQL Server. Expertise in ETL/ELT design with SQL and Python. Solid understanding of data More ❯

Data Engineer - Active SC, Databricks, PySpark Up to £475 per day (Inside IR35) Remote/London 6 months My client is an International Consultancy who are recruiting for an Data Engineer with Active Security Clearance (SC) and strong Databricks and Azure experience to deliver … and optimise Data engineering solutions. Key requirements: Proven experience as a Data Engineer with Active Security Clearance (SC) Strong experience with Databricks, PySpark and Delta Lake Expertise in Jobs & Workflows, cluster tuning, and performance optimisation Solid understanding of Data governance (Unity Catalog, Lineage, Access Policies) Hands-on with Azure services: Data Lake Storage (Gen2), Key Vault, Azure Functions Familiarity with CI/CD for Databricks deployments Strong troubleshooting in distributed Data environments Excellent communication and collaboration skills Nice to have: Immediate availability Experience with enterprise-scale Databricks environments Knowledge of Azure Synapse, Power BI and cost optimisation strategies If you're interested in this role, click More ❯

Data Engineer - Active SC, Databricks, PySpark Up to £475 per day (Inside IR35) Remote/London 6 months My client is an International Consultancy who are recruiting for an Data Engineer with Active Security Clearance (SC) and strong Databricks and Azure experience to deliver … and optimise Data engineering solutions. Key requirements: Proven experience as a Data Engineer with Active Security Clearance (SC) Strong experience with Databricks, PySpark and Delta Lake Expertise in Jobs & Workflows, cluster tuning, and performance optimisation Solid understanding of Data governance (Unity Catalog, Lineage, Access Policies) Hands-on with Azure services: Data Lake Storage (Gen2), Key Vault, Azure Functions Familiarity with CI/CD for Databricks deployments Strong troubleshooting in distributed Data environments Excellent communication and collaboration skills Nice to have: Immediate availability Experience with enterprise-scale Databricks environments Knowledge of Azure Synapse, Power BI and cost optimisation strategies If you're interested in this role, click More ❯

of and provide hands-on technical leadership in the delivery of proofs-of-concept or proofs-of-architecture for our customers. • Collaborate with application engineers, product managers, and data scientists to deliver solutions that solve real-world problems • Deeply embedded with our most strategic platform customers, serving as their technical thought partner in ideating and building novel applications … AI Solution Architects are data and analytics professionals with at least 5 years of experience with knowledge of Transformer (and other Deep Learning) Architectures, data lake architectures, data integration, and data governance, and at least 2 years of experience with cloud-based AI/ML technologies (such as tools from AWS … POC to create SML • Expertise in optimizing AI solutions for runtime cost. • Expertise in conversational system architecture • Extensive knowledge on API's & Integrations, Patterns. • Extensive knowledge of big data technologies • Knowledge on Statistical Methods • OTHER SKILLS WE'D APPRECIATE 1. Understanding of NLP engines, Artificial Intelligence, Machine Learning frameworks etc. EDUCATION QUALIFICATION • Graduate in Engineering OR master's More ❯

Data Consultant Salary: Up to £70,000 I am working with a leading Microsoft partner that is currently recruiting for a Data Consultant to join their growing team. This organisation is driving digital transformation for a wide range of businesses across the UK, specialising in delivering innovative solutions using the Microsoft Power Platform, Azure and emerging … technologies like Microsoft Fabric. Their mission is to help organisations unlock the full potential of their data through modern, cloud-first architectures. They are known for their consultative approach, working closely with clients to design tailored solutions that improve efficiency, enable self-sufficiency and accelerate growth. With a strong focus on scalable analytics and advanced business intelligence, they … are modernising data platforms to deliver future-ready solutions. This is a hands-on role where you will work across the full data stack, collaborating with architects, analysts and stakeholders to build platforms that drive insight and decision-making. In this role, you will be responsible for: Building and managing data pipelines using Azure More ❯

Data Consultant Salary: Up to £70,000 I am working with a leading Microsoft partner that is currently recruiting for a Data Consultant to join their growing team. This organisation is driving digital transformation for a wide range of businesses across the UK, specialising in delivering innovative solutions using the Microsoft Power Platform, Azure and emerging … technologies like Microsoft Fabric. Their mission is to help organisations unlock the full potential of their data through modern, cloud-first architectures. They are known for their consultative approach, working closely with clients to design tailored solutions that improve efficiency, enable self-sufficiency and accelerate growth. With a strong focus on scalable analytics and advanced business intelligence, they … are modernising data platforms to deliver future-ready solutions. This is a hands-on role where you will work across the full data stack, collaborating with architects, analysts and stakeholders to build platforms that drive insight and decision-making. In this role, you will be responsible for: Building and managing data pipelines using Azure More ❯

Your New Role We're looking for an experienced Principal Data Engineer to lead the development and evolution of a modern enterprise data platform. This strategic, hands-on position involves architecting and optimising scalable data pipelines to support advanced analytics, AI/ML initiatives, and actionable insights across the organisation.You'll take full ownership … Acting as the technical authority, you'll define best practices, establish governance frameworks, and mentor engineers and analysts to maximise platform value.This is an opportunity to shape the data landscape and deliver solutions that empower decision-making and innovation. Key Responsibilities Platform Leadership: Design, implement, and manage Snowflake as the enterprise data hub, ensuring scalability, security … and performance. Data Architecture & Strategy: Define frameworks for ingestion, replication, storage, and transformation across diverse data sources. Pipeline Development: Build efficient ELT pipelines using tools such as DBT and Python, integrating operational, financial, and network data. Performance Optimisation: Configure Snowflake warehouses and partitioning strategies for cost efficiency and speed. Governance & Compliance: Implement data quality More ❯

Data Engineer (Python) - Leading Asset Manager | London (Hybrid) Brand-new team. High autonomy. Big impact and high bonus! A top Asset Manager is building a new data function from scratch , and they need a sharp Data Engineer to help define and deliver their future data strategy. You'll work directly with senior … Hybrid: 3 days in London office, 2 days WFH (flexible when needed). The Role Build and optimise scalable data pipelines for a new Data Lake Work with Python, SQL, Spark, AWS, Docker, CI/CD Orchestrate workflows using Airflow, Prefect, or Dagster Support ETL, API integrations, and high-quality data validation Translate … business needs into technical solutions with full ownership What You Need STEM degree (Computer Science, Engineering, Maths, etc.) 2+ years in data engineering/analytics/software Strong Python + SQL 2+ years ETL, APIs, CI/CD, Docker, AWS Ideally but not essential experience with Airflow/Prefect/Dagster Background in Financial Services, FinTech, Insurance, PE More ❯

humanity's enduring challenges. We are seeking to recruit a Full Stack Software Engineer to help design, build, and maintain user facing components of our scalable, high-performance data platform supporting our bio-informaticians and science teams. You'll collaborate closely as part of a cross functional team working with architects, platform and data engineers to … deliver platform tools to help manage the full data lifecycle in the platform, monitor sequencing device telemetry and provide the tooling our science teams need to discover and use the data. Key Responsibilities Design, develop, and support web applications deployed to support the data platform. Own the full stack development for tools that support data management across the full lifecycle, including ingest monitoring, curation, managing releases, monitoring data quality, and user access control management. Build and support components to help manage ingest from and monitor distributed next-generation sequencing devices, including device telemetry (Kafka, MQTT streaming from Oxford Nanopore NGS devices). Design, develop, and maintain platform tools to support our More ❯

client is a specialist insurer and reinsurer operating globally, with a focus on complex commercial and industrial risks. The business combines deep sector expertise with advanced use of data, analytics, and modern technology to deliver informed, efficient, and fair underwriting decisions. The company is in a period of significant growth and is committed to maintaining a collaborative, inclusive … technology landscape. This is a pivotal position within the companys scaling and modernisation programme. The successful candidate will act as the bridge between internally developed underwriting platforms, the data lakehouse and analytics estate, and a range of commercial off-the-shelf (COTS) applications used across operations, finance, and support functions. You will be responsible for defining standards, blueprints … understanding of the .NET/C# ecosystem and modern front-end frameworks (ideally Angular). Experience with data architecture concepts including relational, NoSQL, and data lake technologies. Knowledge of cloud-based AI architectures, including LLMs, Generative AI, and Agentic AI. Experience integrating proprietary systems with COTS finance or ERP platforms, particularly within general ledger, financial More ❯

client is a specialist insurer and reinsurer operating globally, with a focus on complex commercial and industrial risks. The business combines deep sector expertise with advanced use of data, analytics, and modern technology to deliver informed, efficient, and fair underwriting decisions.The company is in a period of significant growth and is committed to maintaining a collaborative, inclusive, and … technology landscape.This is a pivotal position within the company’s scaling and modernisation programme. The successful candidate will act as the bridge between internally developed underwriting platforms, the data lakehouse and analytics estate, and a range of commercial off-the-shelf (COTS) applications used across operations, finance, and support functions.You will be responsible for defining standards, blueprints, and … understanding of the .NET/C# ecosystem and modern front-end frameworks (ideally Angular). Experience with data architecture concepts including relational, NoSQL, and data lake technologies. Knowledge of cloud-based AI architectures, including LLMs, Generative AI, and Agentic AI. Experience integrating proprietary systems with COTS finance or ERP platforms, particularly within general ledger, financial More ❯