26 to 50 of 71 Data Lake Jobs in London

Cleared Data Engineers & Data Architects UK | Hybrid | Multiple levels At Datatech Analytics, we’re supporting a leading UK technology and consulting organisation delivering mission critical data and AI platforms across defence and national security programmes. We’re looking to speak with DV-cleared Data … modern data engineering tooling Spark, Databricks and distributed data platforms AWS, Azure and Google Cloud Modern data lake and data warehouse architectures Who we’re keen to speak with Active DV clearance is essential. We’re interested in strong engineers ...

Python for data engineering Experience delivering solutions on cloud data platforms ( AWS preferred , Azure acceptable) Understanding of data lake and lakehouse architectures Ability to work autonomously and collaboratively within delivery squads Experience working within Central Government departments (highly desirable) Desirable Azure Data Lake, Synapse, Delta Lake Streaming technologies (Kafka, Event Hubs ...

Data Architect (AI Data Foundations) Location - Erskine, London, Farnborough or Newcastle Candidates are required to be eligible for clearance We are seeking an AI Data Architect to design and implement AI-ready data foundations. You will define data models, storage … patterns, pipelines, and governance frameworks that support machine learning and analytics. Working closely with data engineers and ML engineers, you will ensure data is well-structured, accessible, secure, and scalable for AI use cases. Key Responsibilities Design and maintain data models and schemas (relational ...

engineers to deliver robust and reusable data products Manage data lakes and warehouses using formats such as Delta Lake and Parquet Implement best practices for data governance, performance, and security Continuously evaluate and adopt new technologies to evolve the organisation's data … junior engineers and contribute to team capability building Technical Stack Core: Azure Data Factory Azure Synapse Analytics Azure Data Lake Storage Gen2 SQL Server Databricks Enhancements: Python (PySpark, Pandas) CI/CD (Azure DevOps) Infrastructure as Code (Terraform, Bicep) REST APIs GitHub Actions Desirable: Microsoft ...

engineers to deliver robust and reusable data products Manage data lakes and warehouses using formats such as Delta Lake and Parquet Implement best practices for data governance, performance, and security Continuously evaluate and adopt new technologies to evolve the organisation’s data … junior engineers and contribute to team capability building Technical Stack Core: Azure Data Factory Azure Synapse Analytics Azure Data Lake Storage Gen2 SQL Server Databricks Enhancements: Python (PySpark, Pandas) CI/CD (Azure DevOps) Infrastructure as Code (Terraform, Bicep) REST APIs GitHub Actions Desirable: Microsoft ...

Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time on-site when required Expertise required in AWS, Data Pipelines, ETL, Data … analytics and reporting. Develop ETL/ELT workflows to move data efficiently into data warehouses, data lakes and lake houses using open-source and AWS tooling. Apply DevOps practices, including CI/CD, infrastructure as code and automation, to improve and streamline data ...

Data Engineering Product Owner, AI Data Analytics, Microsoft Stack, Azure, Data Bricks, ML, Azure, Mainly Remote Data Engineering/Technology Product Owner required to join a global Professional Services business based in Central London. However, this is practically a remote role … must have been working within Data Engineering before moving into Product Ownership Technical proficiency:Familiarity with Azure services (e.g., Data Lake, Synapse, Fabric) and Databricks for data engineering, analytics, performance optimisation, and governance Development Framework experience within Microsoft Stack Technologies Experience with implementing ...

Aker Systems was founded in 2017 by a team of experienced technology professionals who recognised an opportunity to provide highly secure enterprise data platforms to large organisations. Aker builds and operates ground-breaking, ultra-secure, high performance, cloud-based data infrastructure for the enterprise. Our proprietary … Platforms designing modern data platforms with appropriate architecture coving Data Mesh, Data Fabric, Data Lake etc. Data/Analytics - solving enterprise data and analytics problems using technology Cloud - hands-on Public Cloud experience in either ...

Senior and Principal Data Architect (Two opportunities) Location: UK (Hybrid) Permanent role, Salary from £80k - £117k Overview We're partnering with a leading financial services consultancy to hire a Senior and Principal Data Architect . This is a senior leadership role focused on shaping and delivering … cloud-based data platforms (AWS, Azure, or GCP) Strong knowledge of data modelling (conceptual, logical, physical) and data lake design Experience with data cataloguing, metadata management, and governance tools Understanding of data security, privacy, and access frameworks Excellent stakeholder ...

Data Delivery & Technical Engagement Lead Key skills : Experience with insurance domain, AWS, data engineering, data architecture, Lloyd’s & London Market and Specialty Insurance, Experience : 10+ years Location : London, UK Job Type : Permanent (Full time) We at Coforge are seeking “Data Delivery & Technical … development, PySpark, Glue Catalog) Apache Spark AWS Redshift (modelling, optimisation) AWS Lambda AWS RDS AWS Step Functions Experience with Open Table Formats: Delta Lake Apache Hudi Strong knowledge of S3-based data lakes and modern cloud-native data architectures Data Engineering & Architecture Skills ...

Senior Data Consultant - London Our client, a fast-growing, innovative company based in London is seeking a Senior Data Consultant to lead key client accounts and deliver high-impact data transformation projects. This is an exciting opportunity to work across the full project lifecycle … from proposal development to delivery, in collaboration with data engineers, data scientists, and DevOps teams. The ideal candidate is a self-starter with natural curiosity, a proactive approach, and a drive to get things done, both personally and by enabling others. You will be a catalyst ...

skills. Technical Stack: Languages: Python (primary), SQL, Bash Cloud: Azure, AWS Tools: MLflow, Azure ML, Airflow, Docker, Fargate, FastAPI Data: Snowflake, Delta Lake, Redis, Azure Data Lake Infra & Ops: Terraform, GitHub Actions, Azure DevOps, Grafana, Azure Monitor Desirable skills/knowledge/experience: Knowledge … data platforms (e.g. Snowflake, Azure Data Lake). Experience deploying models as APIs (FastAPI, Azure Functions). Understanding of monitoring, model performance tracking, and observability best practices. ...

Databricks Data Engineer Up to £75,000 I am currently recruiting for a Databricks Data Engineer to support the delivery of modern data platform and engineering programmes within a growing data focused consultancy. Working across a range of enterprise clients, this role … focuses on designing and delivering scalable data solutions using the Databricks Data Platform. This position sits at the intersection of data architecture, engineering delivery and client engagement. You will be responsible for building robust data pipelines, supporting cloud data platforms ...

Description: Cybersecurity Consultant (Cloud, Data, AI & Microsoft Security) Department: Information Security - (Data & AI team) Duration: Duration 4 months Location : London (Hybrid) Role Overview We are looking for a highly skilled Cybersecurity Consultant with strong expertise across Microsoft Security , Cloud Security , Data Protection , and emerging … into clear business language and actionable recommendations. Desirable Skills Experience with secure data platforms (Azure Data Factory, Data Lake, SQL, or similar). Knowledge of API, container security, Kubernetes, and infrastructure-as-code security. Familiarity with PCI, GDPR, data privacy requirements ...

Data Engineer (Pharma Background Essential) £550 per day - INSIDE IR35 6 Months Hybrid - 1 day per week in London Role Purpose We are currently looking for a Data Engineer to be responsible for designing, building and maintaining our clients data pipelines and platform infrastructure … written and verbal communication skills, with the ability to explain technical concepts to non-technical audiences Desirable Skills Experience with Azure Data Lake Storage, Azure Synapse Analytics or Azure Data Factory prior to Fabric migration. Familiarity with Microsoft Purview for data governance ...

Senior Data Engineer Location: London (Hybrid) Function: Data Engineering/Data Platform Salary: Up to £140,000 base + bonus + benefits About the Role Our client is a global commodities and energy trading firm operating at the intersection of quantitative research, technology … significantly expanded its capabilities in renewable power optimisation, analytics, and physical gas trading across international markets. They are now hiring a Senior Data Engineer to join their Technology Data, AI & UX team in London. This team sits close to the trading floor and is responsible ...

time. Your Role: Design, develop, and optimise large-scale cloud data platforms using Azure Data Factory, Data Lake, Key Vault, Azure Functions, Databricks, Delta Lake, PySpark, and Unity Catalog. Collaborate with underwriting, actuarial, Delegated Authority, bordereaux, exposure management, reinsurance, finance, and Solvency … Your Skills: Expert-level hands-on experience with Azure Data Services and Databricks in enterprise, highly regulated environments. Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns, distributed compute, data modelling, and performance optimisation. Strong programming skills in Python, PySpark, and Spark SQL, with experience ...

Principal Data Architect - Hybrid - London - No Sponsorship Available About the Role We are seeking an experienced and visionary Principal Data Architect to join a leading consultancy in the financial services sector. This strategic role involves leading complex, high-profile data initiatives that drive innovation … Strong expertise in cloud technologies (AWS, Azure, GCP) with relevant certifications highly desirable. In-depth knowledge of data modeling, data lake design, and metadata management tools. Proven ability to design secure, scalable, and cost-effective data solutions in cloud ecosystems. Experience ...

Senior Data Engineer | £600 per day Outside IR35 | 6 Month Contract A PE-backed asset/wealth management firm managing multi-billion pounds in assets is seeking a Senior Data Engineer to design and deliver a new Azure-based data warehouse. The organisation … take a hands-on role building a robust data platform from ingestion through to consumption. Tech Environment: Azure Data Lake Storage Azure Synapse Medallion Architecture ETL/data ingestion pipelines Dimensional modelling Strong governance, security, and error handling Ideal Experience: Azure-based data ...

Data Engineer - Databricks - Hybrid (London) - Azure/AWS - Up to £75,000 If you're a Data Engineer who enjoys working close to the business, solving real problems and seeing the impact of what you build, this is worth a look. You'll be joining … supportive, down-to-earth team What they're looking for: Solid experience with Databricks (Azure or AWS) Good understanding of tools like Delta Lake, Unity Catalog, MLflow Strong SQL and Spark (Python or Scala) Experience working with data pipelines and modern data platforms A good ...

companies globally – with offices in New York, London, Madrid and Singapore. We are at an early and exciting stage of embedding AI and data capabilities across the firm. This hire is central to that effort. The Role: We are looking for a commercially minded AI and data … into the business in a practical, considered way Data Infrastructure and Reporting Help maintain and extend our Snowflake-based data lake, supporting data quality and accessibility across the business Build and maintain Power BI dashboards and reports that drive commercial decision-making across ...

engineering best practice and guide delivery across cross-functional teams. Responsibilities Build and optimise Azure data platforms using ADF, Data Lake, Azure Functions and Databricks Deliver reliable, high-quality data pipelines Work with stakeholders to ensure solutions meet Lloyd's of London regulatory … Services and Databricks in enterprise environments. Proven delivery experience within the Lloyd's of London or London Market insurance sector Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns and data modelling. Strong Python, PySpark and Spark SQL capability Solid understanding of data governance Proven ...

Your new company Working for a renowned financial services organisation Your new role Seeking a Data Engineer to help design and maintain scalable batch and near-real-time ingestion pipelines, modernizing legacy ETL/ELT processes into Azure and Snowflake, and implementing best-practice patterns such … data warehouses, ideally Snowflake Proficiency in Python for data processing, automation, and pipeline utilities Good understanding of data lake/lakehouse concepts and ingestion patterns Infrastructure-as-Code exposure (Terraform) and CI/CD (Azure DevOps) Able to prototype quickly while adhering ...

Principal Data Architect, Secure Government and Defence Programmes Location: UK, hybrid, client site as required Security Requirement: Developed Vetting (DV) clearance is mandatory. This role is only suitable for candidates who currently hold active DV clearance. Overview A leading UK consulting organisation delivering mission-critical digital and data transformation across defence, national security, and sensitive government environments is seeking a Principal Data Architect. This is a senior, client-facing role focused on defining and delivering secure, scalable data platforms within highly classified programmes. Due to the nature of the work and access ...

existing Netezza structures. Produce the EDW logical and physical data models, dimensional models for the Semantic Layer, and Data Lake zone architecture (Bronze/Silver/Gold). Define Enterprise Semantic Layer design, including pre-computed aggregations, dimensional models, and Tableau connectivity specifications. Lead source … with at least 3 years leading enterprise data model design. Experience designing Lakehouse architecture: medallion architecture (Bronze/Silver/Gold), Delta Lake, or equivalent. Proficiency in dimensional modelling (Kimball methodology) and enterprise data modelling. Experience with Databricks (Unity Catalog, Delta Lake) or Snowflake ...