10 of 10 Remote Data Lake Jobs in the City of London

analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage. Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools … Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

Data Engineer - £350PD - Remote Required Technical Skills Data Pipeline & ETL Design, build, and maintain robust ETL/ELT pipelines for structured and unstructured data Hands-on experience with AWS Glue and AWS Step Functions Implementation of data validation, data quality … production pipelines Experience with incremental data processing patterns (CDC, watermarking, upserts) AWS Data Services Amazon S3: data lake architectures, partitioning strategies, lifecycle policies DynamoDB: data modeling, secondary indexes, streams, and performance optimization Amazon Redshift: foundational querying, integrations, and performance considerations ...

granular financial datasets. You’ll play a central role in shaping large-scale tick-data architecture - building pipelines, improving data-lake structures, and refining storage formats to support demanding research and trading workflows... Key Responsibilities Architect, build, and refine high-throughput pipelines capable of processing … Develop resilient, well-structured Python/OOP solutions underpinning tick-data ingestion, normalisation, storage, and replay Design and evolve data-lake patterns using Parquet and related columnar formats; introduce Iceberg-style table modelling where beneficial Implement cloud-backed processing and storage patterns - leveraging services from ...

About the Opportunity This is a unique opportunity for a Data Engineer to work at the intersection of cutting-edge technology and elite sports performance. You’ll join a collaborative, high-standard engineering team with a culture built around creativity, innovation, and excellence. In this role … scientists to enable end-to-end data workflows Own the architecture and maintenance of a GCP-based data lake/lakehouse environment Your Background 4+ years of industry experience in Data Engineering roles Advanced-level Python for data applications ...

Data Engineer | Hybrid | London | Databricks | Azure | 85k I'm working with a global powerhouse that's been setting the standard for excellence for over 60 years. With more than 1,000 projects delivered worldwide and a combined value exceeding $150 billion, they've earned a reputation … forefront of the data engineering field. You'll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards and analytics ETL & Data Modelling: T-SQL, metadata-driven pipelines Design ...

Role : Senior Data Architect Domain: Commodities Trading Location : London (hybrid 3 days per week) Contract : Initial 12 months Rate : £750-850/d inside IR35 X4 Technology are partnered with a global energy trading firm embarking on a major data transformation to enable AI-driven capabilities … confidently consumed by AI and analytics solutions by driving data productisation, enterprise & cross-portfolio modelling, business rule definition, Azure data lake maturity, data quality assurance and advanced concepts like ontology and data mesh. Your work will support large-scale trading ...

Technology Product Owner, AI Data Analytics, Microsoft Stack, Azure, Data Engineering, ML, Azure, Remote with Travel Technology Product Owner required to join a global Professional Services business based in Central London. However, this is practically a remote role, but when travel is required (to London, Europe … Development FIRST mentality. You must have been a Developer before moving into Product Ownership Technical proficiency:Familiarity with Azure services (e.g., Data Lake, Synapse, Fabric) and Databricks for data engineering, analytics, performance optimisation, and governance. Experience with implementing and optimising scalable cloud infrastructure is highly ...

Director of Data & AI | Microsoft Azure Stack | AI-Driven Legal Platform Workplace Type: Hybrid Location: London Salary: 130-150k The Role This is a senior leadership role for someone who wants to sit at the heart of a business where AI is already embedded into live products … platform is built on a Microsoft stack with centrally stored data: Azure Machine Learning Microsoft Fabric Databricks Centralised data lake architecture Microsoft Dynamics CRM AI-powered legal tools focused on due diligence and contract review Nice to Haves Experience working on established AI products rather ...

daily. Join us to grow your skills, shape bold ideas, and help build the future of tech. We’re looking for a Principal Data Archite ct to help shape how high-growth startups build and scale on AWS. In this role, you’ll design and deliver … data and analytics solutions - from architecture and pipelines to visualization and insights - guiding customers from concept through production. You’ll work closely with startup founders, technical leaders, and account executives to create scalable, cost-efficient architectures that drive real business impact. 📍 Work location - hybrid from London ...

definition through to high-level architecture and platform design. The role covers areas such as Predictive AI, Machine Learning and Generative AI, MLOps, data pipelines and modern AI platforms, across both cloud and private infrastructure. Alongside customer work, you’ll contribute to the development of AI offerings, standards … other architects within the wider team. Key Responsibilities • Act as the AI subject matter expert in customer engagements, working directly with business stakeholders, data teams and IT operations. • Lead discovery and design of AI strategies, governance models and platform architectures aligned to business outcomes. • Design ...