24 of 24 Data Lake Jobs in the City of London

developers (quants and strategists) in a unique hands-on opportunity within a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with … What We’re Looking For 5 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing ...

unique opportunity to remain hands-on in engineering while also leading a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with … What We’re Looking For 10 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing ...

analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage. Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools … Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

Data Engineer - £350PD - Remote Required Technical Skills Data Pipeline & ETL Design, build, and maintain robust ETL/ELT pipelines for structured and unstructured data Hands-on experience with AWS Glue and AWS Step Functions Implementation of data validation, data quality … production pipelines Experience with incremental data processing patterns (CDC, watermarking, upserts) AWS Data Services Amazon S3: data lake architectures, partitioning strategies, lifecycle policies DynamoDB: data modeling, secondary indexes, streams, and performance optimization Amazon Redshift: foundational querying, integrations, and performance considerations ...

granular financial datasets. You’ll play a central role in shaping large-scale tick-data architecture - building pipelines, improving data-lake structures, and refining storage formats to support demanding research and trading workflows... Key Responsibilities Architect, build, and refine high-throughput pipelines capable of processing … Develop resilient, well-structured Python/OOP solutions underpinning tick-data ingestion, normalisation, storage, and replay Design and evolve data-lake patterns using Parquet and related columnar formats; introduce Iceberg-style table modelling where beneficial Implement cloud-backed processing and storage patterns - leveraging services from ...

doing: Design, build, and optimise scalable data pipelines and architecture Work hands-on with SQL, PySpark, Databricks, Azure, and Data Lake to deliver high-performance solutions Translate business needs into technical requirements and data-driven solutions Ensure best practice in data … Engineer or Senior Data Engineer with some team leadership responsibilities Strong technical expertise with PySpark, Databricks, Azure, and Data Lake Deep understanding of data architecture and ETL processes in cloud environments The ability to balance coding and technical delivery with people management ...

About the Opportunity This is a unique opportunity for a Data Engineer to work at the intersection of cutting-edge technology and elite sports performance. You’ll join a collaborative, high-standard engineering team with a culture built around creativity, innovation, and excellence. In this role … scientists to enable end-to-end data workflows Own the architecture and maintenance of a GCP-based data lake/lakehouse environment Your Background 4+ years of industry experience in Data Engineering roles Advanced-level Python for data applications ...

Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time on-site when required Expertise required in AWS, Data Pipelines, ETL, Data … analytics and reporting. Develop ETL/ELT workflows to move data efficiently into data warehouses, data lakes and lake houses using open-source and AWS tooling. Apply DevOps practices, including CI/CD, infrastructure as code and automation, to improve and streamline data ...

Data Engineer | Hybrid | London | Databricks | Azure | 85k I'm working with a global powerhouse that's been setting the standard for excellence for over 60 years. With more than 1,000 projects delivered worldwide and a combined value exceeding $150 billion, they've earned a reputation … forefront of the data engineering field. You'll Work With Azure Data Services: Data Factory, Data Lake, SQL Databricks: Spark, Delta Lake Power BI: Advanced dashboards and analytics ETL & Data Modelling: T-SQL, metadata-driven pipelines Design ...

Role : Senior Data Architect Domain: Commodities Trading Location : London (hybrid 3 days per week) Contract : Initial 12 months Rate : £750-850/d inside IR35 X4 Technology are partnered with a global energy trading firm embarking on a major data transformation to enable AI-driven capabilities … confidently consumed by AI and analytics solutions by driving data productisation, enterprise & cross-portfolio modelling, business rule definition, Azure data lake maturity, data quality assurance and advanced concepts like ontology and data mesh. Your work will support large-scale trading ...

troubleshooting Familiar with the Agile. Well communication. Good English listening and speaking for communicating requirements and development tasks/issues Hands-on experience with lake houses, dataflows, pipelines, and semantic models Ability to build ETL workflows Familiarity with time-series data, market feeds, transactional records, and risk … work with and manage stakeholders Desirable skills/knowledge/experience: Ability to prepare and process datasets for Power BI usage Experience with One Lake, Azure Data Lake, and distributed computing environments Understanding of financial regulations such as GDPR, SOX etc. Spark application performance tuning Knowledge ...

cutting-edge solutions – we help companies across industries get ahead and stay ahead. Powered by curiosity, courage, and innovation, our teams implement data, technology, and AI to create tomorrow, today. Get to know us at genpact.com and on LinkedIn, X, YouTube, and Facebook. Inviting Applications for the role … MLOps concepts, model registration, deployment (AKS, ACI), and monitoring. • Hands-on experience with Azure data services such as Azure Data Lake Storage, Azure Data Factory, Azure Databricks, or Azure Synapse Analytics. • Solid understanding of software engineering principles, including version control (Git), testing ...

Technology Product Owner, AI Data Analytics, Microsoft Stack, Azure, Data Engineering, ML, Azure, Remote with Travel Technology Product Owner required to join a global Professional Services business based in Central London. However, this is practically a remote role, but when travel is required (to London, Europe … Development FIRST mentality. You must have been a Developer before moving into Product Ownership Technical proficiency:Familiarity with Azure services (e.g., Data Lake, Synapse, Fabric) and Databricks for data engineering, analytics, performance optimisation, and governance. Experience with implementing and optimising scalable cloud infrastructure is highly ...

build, and maintain scalable data pipelines using Databricks Notebooks, Jobs, and Workflows for both batch and streaming data. Optimise Spark and Delta Lake performance through efficient cluster configuration, adaptive query execution, and caching strategies. Conduct performance testing and cluster tuning to ensure cost-efficient, high-performing workloads. … governance best practices. Develop PySpark applications for ETL, data transformation, and analytics, following modular and reusable design principles. Create and manage Delta Lake tables with ACID compliance, schema evolution, and time travel for versioned data management. Integrate Databricks solutions with Azure services such as Azure ...

Title: Azure Databricks Data Engineer Primary skills: Advanced SQL, Azure Databricks, Azure Data Factory, Azure Datalake. Secondary skills: Azure SQL, PySpark, Azure Synapse. Experience: 10+ Years of Experience Hybrid - 3 Days/Week onsite is MUST About the job We are looking for an experienced Databricks … Data Engineer to design, develop, and manage data pipelines using Azure services such as Databricks, Data Factory, and Datalake. The role involves building scalable ETL solutions, collaborating with cross-functional teams, and processing large volumes of data. You will work closely with business ...

Azure range of products, which enables us to deliver powerful, connected business solutions that drive significant client value. The Role: Pre-Sales Data Solutions Architect – Microsoft Business Solutions (MBS) The Pre-Sales Data Solutions Architect will define and articulate data and AI strategies within … Microsoft AI-powered business solutions. This strategic role combines deep technical expertise in Microsoft’s data stack with the ability to translate business requirements into end-to-end Microsoft architectures, operating at the intersection of Microsoft Data, AI, and Application technologies. Responsibilities: Engaging with clients ...

Senior Data Scientist Employment Type: Permanent Location: [Specify location or remote flexibility] Role Overview: We are seeking a highly skilled Senior Data Scientist with expertise in multi-cloud environments and a strong background in consulting . The ideal candidate will lead advanced analytics projects, design scalable … data solutions, and provide strategic insights to clients across diverse industries. This role requires a blend of technical proficiency, business acumen, and excellent communication skills. Key Responsibilities: Design and implement data science models (predictive, prescriptive, and descriptive analytics) to solve complex business problems. Work across multi ...

Data Engineering Team Lead - Remote - Databricks - Azure - 80k Join a Leading Microsoft Consultancy Driving Data Innovation I'm working with a well-established Microsoft Partner with an incredible project pipeline, rapid growth, and a reputation for delivering Tech for Good. They're working on cutting-edge … projects using emerging technologies like Microsoft Fabric and Azure Databricks. We're looking for a Data engineering team lead who combines hands-on technical expertise with leadership skills to mentor a talented team and deliver exceptional solutions for our clients. What You'll Do Lead and mentor ...

Your Role We are looking for strong Azure Solution Architects who are passionate and focused on data solutions and Microsoft technology and who ideally have skills in many of the following areas: Partners with other solution architects to assess solution alignment to the overall architectural blueprint – and drive … Serve as technical liaison between Sales team, Clients, Delivery & support teams up to and including Contract negotiations Have experience of designing architecture for data focused Microsoft Azure projects, Cooperate with sales team to formulate/execute a sales strategy to exceed revenue objectives Working on Microsoft Azure PaaS ...

daily. Join us to grow your skills, shape bold ideas, and help build the future of tech. We’re looking for a Principal Data Archite ct to help shape how high-growth startups build and scale on AWS. In this role, you’ll design and deliver … data and analytics solutions - from architecture and pipelines to visualization and insights - guiding customers from concept through production. You’ll work closely with startup founders, technical leaders, and account executives to create scalable, cost-efficient architectures that drive real business impact. 📍 Work location - hybrid from London ...

numi is proud to partner with a growing global technology consultancy seeking a 5x Senior Data Engineers to join their team in London on a hybrid basis. This consultancy empowers some of the world's biggest businesses by delivering advanced data, AI, cloud, and digital solutions. … make a real impact, this is an excellent opportunity to expand your personal and professional skills across various industries and technologies. As the Data Engineer, you will be the core builder responsible for designing, constructing, and maintaining robust and reliable data infrastructure. Your key focus will ...

definition through to high-level architecture and platform design. The role covers areas such as Predictive AI, Machine Learning and Generative AI, MLOps, data pipelines and modern AI platforms, across both cloud and private infrastructure. Alongside customer work, you’ll contribute to the development of AI offerings, standards … other architects within the wider team. Key Responsibilities • Act as the AI subject matter expert in customer engagements, working directly with business stakeholders, data teams and IT operations. • Lead discovery and design of AI strategies, governance models and platform architectures aligned to business outcomes. • Design ...

Paragon Alpha are partnered with one of the worlds most prominent multi-strat hedge funds, looking to make a Data hire in London to support quant equities strategies. This data team helps to build cloud based and scalable data-lakes … quant teams, ingest vast amounts of data into the company, which can be used for quant research and model development. Stack: Python, AWS, S3, Airflow, SQL The role needs a talented data engineer who can build a scalable and bespoke data platform ...

world-leading global quantitative hedge fund are looking for a Data Engineer to support their systematic trading and research platform, with a focus on building, managing, and scaling datasets used directly by quants and traders. You will be responsible for developing automated ETL pipelines, integrating new and alternative … checks, and contributing to the firm’s core data framework while working closely with research and trading teams. Stack: Python, Databases/Datalakes, Git The firm offers a highly collaborative, global environment with strong ownership, close interaction with quants and traders, and the opportunity to work on data ...