1 to 25 of 620 Data Lake Jobs in London

provides its customers with clear advantage, and its technology-led service provides access to all major exchanges, order-flow management via screen, voice and DMA, plus award-winning data, insights and analytics. The Technology Department delivers differentiation, scalability and security for the business. Reporting to the COO, Technology provides digital tools, software services and infrastructure globally to all … design. The IT Group runs our enterprise-wide services to end users and actively manages the firm's infrastructure and data. Within IT, Marex Technology has established a Data team that enables the firm to leverage data assets to increase productivity and improve business decisions, as well as maintain data compliance. The Data … a regulated environment and knowledge of the risk and compliance requirements associated with this. • Oracle Database • MongoDB • Cloud Data Technologies (Mainly Azure - SQL Database, Data Lake, HD Insight, Data Factory etc.) • A collaborative team player, approachable, self-efficient and influences a positive work environment • Demonstrates curiosity • Resilient in a challenging, fast-paced environment More ❯

Senior Java Software Engineer AWS Python Data Finance London Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager … Trading, Front Office finance Spark Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Java Software Engineer (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front … used across the Front Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Java, Python and AWS. If you have any experience in Iceberg, Dremio More ❯

About The Role North Northamptonshire Council are working to modernise their data management and visualisation processes. We are working to deliver our data analytics strategy. This role will support the design and delivery of the data warehousing, data infrastructure and (where required) the migration of existing data sources (data … role A deep understanding of data engineering principles – ideally in an Azure environment Proven experience as a Data Engineer/Specialist in data lake/warehouse and data migration projects Experience building and maintaining ETL processes Experience producing data models and understanding where to use different types of data … years + working with Azure data engineering tools e.g: Azure Data factory Azure Synapse Azure SQL Azure DataBricks Microsoft Fabric Azure data lake Exposure to other data engineering and storage tools: Snowflake AWS tools – Kinesis/Glue/Redshift Google tools – BigQuery/Looker Experience working with open datasets – ingesting data More ❯

Senior Data Engineer - (Azure/Databricks) Axis Capital London, United Kingdom Apply now Posted 15 days ago Permanent Competitive Senior Data Engineer - (Azure/Databricks) Axis Capital London, United Kingdom Apply now This is your opportunity to join AXIS Capital - a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our outstanding … data solutions are secure, efficient, and optimized. Key Responsibilities: Design and implement data solutions using Azure services, including Azure Databricks, ADF, and Data Lake Storage. Develop and maintain ETL/ELT pipelines to process structured and unstructured data from multiple sources. - Automate loads using Databricks workflows and Jobs Develop, test and … warehousing, and real-time streaming. Knowledge of developing and processing full and incremental loads. Experience of automated loads using Databricks workflows and Jobs Expertise in Azure Databricks, including Delta Lake, Spark optimizations, and MLflow. Strong experience with Azure Data Factory (ADF) for data integration and orchestration. Hands-on experience with Azure DevOps, including pipelines, repos More ❯

Senior Data Engineer - (Azure/Databricks) page is loaded Senior Data Engineer - (Azure/Databricks) Apply locations London - Scalpel time type Full time posted on Posted 15 Days Ago job requisition id REQ05851 This is your opportunity to join AXIS Capital - a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our … data solutions are secure, efficient, and optimized. Key Responsibilities: Design and implement data solutions using Azure services, including Azure Databricks, ADF, and Data Lake Storage. Develop and maintain ETL/ELT pipelines to process structured and unstructured data from multiple sources. - Automate loads using Databricks workflows and Jobs Develop, test and … warehousing, and real-time streaming. Knowledge of developing and processing full and incremental loads. Experience of automated loads using Databricks workflows and Jobs Expertise in Azure Databricks, including Delta Lake, Spark optimizations, and MLflow. Strong experience with Azure Data Factory (ADF) for data integration and orchestration. Hands-on experience with Azure DevOps, including pipelines, repos More ❯

Join to apply for the Senior Data Analytics Developer role at Coveo Join to apply for the Senior Data Analytics Developer role at Coveo Are you the expert who transforms data into strategic insights to drive our success? As a Senior Data Developer, you will be at the heart of developing our … data platform. Your main role? Designing fundamental data models, creating data warehouses to facilitate business analysis, and improving data monitoring and observability solutions. Using advanced technologies, you will build efficient and scalable data pipelines. What’s exciting about this role? Your efforts will have a direct impact on uncovering essential … with hands-on experience in Python, dbt, Snowflake, and/or Databricks. You have in-depth knowledge of SQL, ETL processes, data warehousing, and Data Lake architecture. You have experience with automated testing frameworks and a commitment to code quality, including unit testing, integration testing, and code coverage metrics. You excel at working collaboratively with More ❯

for the Databricks Lakehouse Platform. Architectural Design: Lead the end-to-end design of the Databricks Lakehouse architecture (Medallion architecture), including data ingestion patterns, storage layers (Delta Lake), processing frameworks (Spark), and consumption mechanisms. Technology Selection: Evaluate and recommend optimal Databricks features and integrations (e.g., Unity Catalog, Photon, Delta Live Tables, MLflow) and complementary cloud services (e.g. … Azure Data Factory, Azure Data Lake Storage, Power BI). Security & Governance Frameworks: Design robust data governance, security, and access control models within the Databricks ecosystem, ensuring compliance with industry standards and regulations. Phase 2: Core Platform Build & Development Hands-on Implementation: Act as a lead engineer in the initial build-out of … sessions to ensure long-term sustainability of the platform. Required Skills & Experience Proven Databricks Expertise: Deep, hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform (Delta Lake, Unity Catalog, Spark, Databricks SQL Analytics). Cloud Data Architecture: Extensive experience with Azure data services (e.g., Azure Data Factory, Azure Data More ❯

for the Databricks Lakehouse Platform. Architectural Design: Lead the end-to-end design of the Databricks Lakehouse architecture (Medallion architecture), including data ingestion patterns, storage layers (Delta Lake), processing frameworks (Spark), and consumption mechanisms. Technology Selection: Evaluate and recommend optimal Databricks features and integrations (e.g., Unity Catalog, Photon, Delta Live Tables, MLflow) and complementary cloud services (e.g. … Azure Data Factory, Azure Data Lake Storage, Power BI). Security & Governance Frameworks: Design robust data governance, security, and access control models within the Databricks ecosystem, ensuring compliance with industry standards and regulations. Phase 2: Core Platform Build & Development Hands-on Implementation: Act as a lead engineer in the initial build-out of … sessions to ensure long-term sustainability of the platform. Required Skills & Experience Proven Databricks Expertise: Deep, hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform (Delta Lake, Unity Catalog, Spark, Databricks SQL Analytics). Cloud Data Architecture: Extensive experience with Azure data services (e.g., Azure Data Factory, Azure Data More ❯

for the Databricks Lakehouse Platform. Architectural Design: Lead the end-to-end design of the Databricks Lakehouse architecture (Medallion architecture), including data ingestion patterns, storage layers (Delta Lake), processing frameworks (Spark), and consumption mechanisms. Technology Selection: Evaluate and recommend optimal Databricks features and integrations (e.g., Unity Catalog, Photon, Delta Live Tables, MLflow) and complementary cloud services (e.g. … Azure Data Factory, Azure Data Lake Storage, Power BI). Security & Governance Frameworks: Design robust data governance, security, and access control models within the Databricks ecosystem, ensuring compliance with industry standards and regulations. Phase 2: Core Platform Build & Development Hands-on Implementation: Act as a lead engineer in the initial build-out of … sessions to ensure long-term sustainability of the platform. Required Skills & Experience Proven Databricks Expertise: Deep, hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform (Delta Lake, Unity Catalog, Spark, Databricks SQL Analytics). Cloud Data Architecture: Extensive experience with Azure data services (e.g., Azure Data Factory, Azure Data More ❯

plan a crucial role in architecting a scalable, reliable and efficient data solutions to support our business objectives. Responsibilities Design, develop, and maintain the data lake house architecture on AWS using best practices and industry standards. Implement data labelling strategy and principals as required by Data Governance and Information Security. Design … and implement robust ETL/ELT processes to extract, transform and load data from various sources into the data lake ensuring data quality and integrity. Ensure data quality, reliability, availability, and scalability across the data lake house. Optimize data ingestion, processing, storage, and access for various … engineering. Proficiency in Python and SQL Proficient in AWS data services such as S3, Glue, Athena, Redshift, EMR, Kinesis, Lambda, etc. Strong knowledge of data lake concepts, architectures, and design patterns. Experience in building and managing data pipelines using tools such as Airflow, Spark, Kinesis etc. Experience in working with structured, semi-structured More ❯

for the Databricks Lakehouse Platform. Architectural Design: Lead the end-to-end design of the Databricks Lakehouse architecture (Medallion architecture), including data ingestion patterns, storage layers (Delta Lake), processing frameworks (Spark), and consumption mechanisms. Technology Selection: Evaluate and recommend optimal Databricks features and integrations (e.g., Unity Catalog, Photon, Delta Live Tables, MLflow) and complementary cloud services (e.g. … Azure Data Factory, Azure Data Lake Storage, Power BI). Security & Governance Frameworks: Design robust data governance, security, and access control models within the Databricks ecosystem, ensuring compliance with industry standards and regulations. Phase 2: Core Platform Build & Development Hands-on Implementation: Act as a lead engineer in the initial build-out of … sessions to ensure long-term sustainability of the platform. Required Skills & Experience Proven Databricks Expertise: Deep, hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform (Delta Lake, Unity Catalog, Spark, Databricks SQL Analytics). Cloud Data Architecture: Extensive experience with Azure data services (e.g., Azure Data Factory, Azure Data More ❯

Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: £50,000 - £77,500 - depending on experience Department: Technology and Data We’re First Central Insurance & Technology Group (First Central for short), an innovative, market-leading insurance company. We protect the things customers love so they can get on with what matters to them in life. Data drives us. It fuels our outstanding distribution, finance, technology and legal services. Our underwriting skills are built on data expertise; it creates the insights we need to give the right cover to the right customers at the right price. But, it’s the people inside and outside our business that power us. They make us stand … Data pipelines, promoting self-testing pipelines that proactively identify processing issues or discrepancies. You’ll build solutions that pipe transform data into data lake storage areas, physical database models and reporting structures across data lake, data warehouse, business intelligence systems and analytics applications. You’ll build physical data More ❯

to technology, delivering bespoke software solutions and expert advice. Our clients are increasingly looking to us to help them make the best use of their data. In building data platforms and pipelines, our data engineers create the foundation for diverse data & analytics solutions, including data science and AI. They build data lakes and warehouses, create the processes to extract or access operational data, and transform siloed datasets into integrated data models that allow insight into business performance and problems or training of ML models. These are hands-on, client-facing roles, with openings at senior or lead level to suit your experience. You may be … or solving tough engineering challenges. You'd also be expected to spend some time on-site with clients in the London area on an ad-hoc basis. Our data engineers combine a strong software engineering approach with solid data fundamentals and experience with modern tools. We're technology agnostic, and we're open minded when it More ❯

class opportunity to help shape our organisation for the next stage of its journey. To drive towards this ambition, we are seeking a motivated individual to join our Data Platform team and support Aztec’s new technology strategy using Azure Databricks. You will lead our Data Engineering capability and collaborate with others passionate about solving business … problems. Key responsibilities: Data Platform Design and Architecture Design, develop, and maintain a high-performing, secure, and scalable data platform, leveraging Databricks Corporate Lakehouse and Medallion Architectures. Utilise our metadata-driven data platform framework combined with advanced cluster management techniques to create and optimise scalable, robust, and efficient data solutions. Implement comprehensive … Proven experience working with Azure data platform services, including Storage, ADLS Gen2, Azure Functions, Kubernetes. Background in cloud platforms and data architectures, such as Corporate DataLake, Medallion Architecture, Metadata Driven Platform, Event-driven architecture. Proven experience of ETL/ELT, including Lakehouse, Pipeline Design, Batch/Stream processing. Strong working knowledge of programming languages, including Python More ❯

that we put them first. Additional Info Hybrid Working: 3 days onsite, 2 from home Location: City of London Contract Type: Permanent Job Summary Requirement is for a Data & Analytics Engineer to work within a small Data & Analytics IT team (team size: 3-4) with strong data & analytical technology skills preferably within the financial … & MI systems built using SQL Analysis Services & SQL Reporting Services. Technology stack will include Informatica Intelligent Cloud Service, Azure Data Analytics Tools (Azure Data Lake Storage, Databricks, Synapse, Factory, PowerBI), Python & SQL databases. A strong understanding of ETL tools and Azure Data Analytics is ESSENTIAL and strong PowerBI development and administration skills … are expected. The SME will have background and experience in implementing ETL data pipelines, PowerBI platforms, and proven experience in analysing complex data sets to identify trends, patterns, and insights that will inform strategic decisions. Key Work Outputs and Accountabilities Design, develop, and maintain scalable ETL data pipelines to support data integration More ❯

and experience — talk with your recruiter to learn more. Base pay range Direct message the job poster from Anson McCade Lead Technology Headhunter | Delivering Top Talent in; Digital & Data Analytics | IT & Cyber Security | Business Change & Transformation | Role: Senior or Lead Data Architect Location: UK-wide, with a preference towards London Clearance: Must be eligible to work … in the UK We’re working with a world-leading consultancy that’s delivering cutting-edge data and cloud architecture across some of the UK’s most recognisable enterprises. As their Data & Intelligence practice continues to expand, we’re looking for experienced Senior Data Architects as well as proven Lead Data Architects … join the team. Whether your strength lies in hands-on solution architecture or enterprise-wide strategy, there’s a place for you to lead, shape and scale modern data capabilities across sectors like public services, finance, telco, manufacturing, and retail. About the Role You’ll be embedded into high-impact client projects, defining, designing, and delivering modern data More ❯

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data Mesh, lakehouse, data vault and data warehouses. Our data engineers create pipelines that support our data scientists and power our front-end applications. This means we do data-intensive work for both OLTP and OLAP use cases. Our environments are primarily cloud-native spanning AWS, Azure and GCP More ❯

provides its customers with clear advantage, and its technology-led service provides access to all major exchanges, order-flow management via screen, voice and DMA, plus award-winning data, insights and analytics. The Technology Department delivers differentiation, scalability and security for the business. Reporting to the COO, Technology provides digital tools, software services and infrastructure globally to all … design. The IT Group runs our enterprise-wide services to end users and actively manages the firm’s infrastructure and data. Within IT, Marex Technology has established a Data team that enables the firm to leverage data assets to increase productivity and improve business decisions, as well as maintain data compliance. The Data … a regulated environment and knowledge of the risk and compliance requirements associated with this. Oracle Database MongoDB Cloud Data Technologies (Mainly Azure - SQL Database, Data Lake, HD Insight, Data Factory etc.) Knowledge of advanced analytics for client segmentation, lifetime value analysis, cross-sell opportunities and campaign impact evaluation. Familiarity with CRM tools and More ❯

We have two roles available: Senior Data Engineer Data Engineer Both roles are permanent positions. Location: To be confirmed. Clearance: SC These roles require the following: Essential competencies Personal attributes: • Strong communicator, able to clearly articulate technical concepts to both technical and non-technical stakeholders. • Confident working independently or as part of a collaborative, cross-functional … and deliver innovative, practical solutions. • Team-oriented, with a positive attitude and a strong sense of ownership and accountability. Technologies, Methodologies and Frameworks: • Deep/Solid expertise in data engineering tools and practices, including Python, SQL, and modern ETL/ELT frameworks … e.g., Azure Data Factory, Talend, dbt). • Direct delivery experience using cloud-native data services, specifically in Microsoft Azure, Fabric, Dataverse, Synapse, Data Lake, Purview. • Experience designing and implementing scalable data pipelines and integration patterns across structured and unstructured data sources (e.g., Azure SQL, MySQL, MongoDB). • Familiarity with More ❯

Social network you want to login/join with: Senior Azure Data Engineer + Power BI, london (city of london) col-narrow-left Client: Location: london (city of london), United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 4 Posted: 16.06.2025 Expiry Date: 31.07.2025 col-wide Job Description: EVOCS is looking for … data marts, and data warehouse structures Integrate data from various sources including SQL databases, APIs, and cloud storage (e.g., Azure Data Lake, OneLake) Implement and maintain standards around data quality, security, and governance Participate in discussions with cross-functional teams to ensure that technical solutions align with business needs … processing using tools such as Event Hubs, Kafka, or equivalents Drive usage of Microsoft Fabric features within modern data architectures, especially OneLake, Lakehouse, and Direct Lake Power BI datasets Effectively communicate across diverse technical stakeholders and navigate challenging peer review or architectural debates Qualifications 5+ years of experience in data engineering with a More ❯

Other - EU work permit required: Yes col-narrow-right Job Views: 5 Posted: 16.06.2025 Expiry Date: 31.07.2025 col-wide Job Description: EVOCS is looking for a Senior Azure Data Engineer with Power BI + Fabric expertise to support the delivery of scalable data solutions for enterprise clients. This is a hands-on individual contributor role focused … data marts, and data warehouse structures Integrate data from various sources including SQL databases, APIs, and cloud storage (e.g., Azure Data Lake, OneLake) Implement and maintain standards around data quality, security, and governance Participate in discussions with cross-functional teams to ensure that technical solutions align with business needs … processing using tools such as Event Hubs, Kafka, or equivalents Drive usage of Microsoft Fabric features within modern data architectures, especially OneLake, Lakehouse, and Direct Lake Power BI datasets Effectively communicate across diverse technical stakeholders and navigate challenging peer review or architectural debates Qualifications 5+ years of experience in data engineering with a More ❯

being one of the UK’s leading car and home insurance providers, our client, a sophisticated and dynamic organisation, seek to add to their talented team. As a Data Architect, you will be involved in, but not limited to: Supporting the Senior Data Architect in spearheading enterprise data strategies. Continual improvement of the enterprise … data principles and blueprint designs. Participate in Solution Architecture Forum, helping to govern design or other projects and presenting own work for approval. Ensure that data management best practices are adhered to, in the areas of data security, data governance, data privacy and industry regulatory compliance Collaborate on conceptual data … Platform data pipeline patterns. Experience required: Insurance/financial services domain knowledge (preferred) Expertise in at least two of these key technologies: Azure, Azure Data Lake, Databricks, Matillion, Snowflake and Power BI (must have) Analytical system data architecture and ETL design experience Hands on experience in designing and implementing a data More ❯

continue to direct the firm's day-to-day operations. Purpose of Position Pantheon is undergoing a multi-year programme to build out a new best in class Data Platform using cloud native technologies hosted in Azure. We require an experienced and passionate hands-on Senior Data Engineer to design and implement new data … organization. Essential Technical Skills Experience in designing and developing data warehouse solutions. Building and configuring multistage Azure deployment pipelines Advanced SQL, Python, PySpark Azure Data Lake, Data Factory, Databricks and Functions Azure Data Lake Storage Gen2 Traditional ETL (Informatica, SSIS etc.) Azure Powershell/CLI Azure Key Vault ARM/… Terraform scripts to deploy infrastructure services. Essential Experience Experience in designing and building data ingestion and transformation pipelines, data lake, data warehouse/data marts etc. Experience working with Agile development methodologies such as Kanban/Scrum A natural problem-solving attitude. Conversant with current industry and technology trends. Experience with More ❯

My client, a leading multi-strategy hedge fund, is seeking a mid-level Data Engineer to join their dynamic and rapidly growing team based in London. This is an exciting opportunity to join a high-performing investment firm that places technology and innovation at the core of its operations. You’ll play a key role in empowering the … front-office investment team through the delivery of data solutions - enabling informed decision-making across the board. The successful individual will take ownership of MI & BI reporting and data analysis, partnering closely with front-office teams. Your work will support business-critical initiatives through the design, maintenance, and enhancement of ETL pipelines, while also contributing to … streamline key workflows. You’ll also be responsible for: Migrating and enhancing business reports using Power BI Managing data ingestion into the firm’s data lake and data warehouse Troubleshooting data issues, performing log analysis, and owning resolution processes Supporting both manual and automated testing for new data solutions More ❯

My client, a leading multi-strategy hedge fund, is seeking a mid-level Data Engineer to join their dynamic and rapidly growing team based in London. This is an exciting opportunity to join a high-performing investment firm that places technology and innovation at the core of its operations. You’ll play a key role in empowering the … front-office investment team through the delivery of data solutions - enabling informed decision-making across the board. The successful individual will take ownership of MI & BI reporting and data analysis, partnering closely with front-office teams. Your work will support business-critical initiatives through the design, maintenance, and enhancement of ETL pipelines, while also contributing to … streamline key workflows. You’ll also be responsible for: Migrating and enhancing business reports using Power BI Managing data ingestion into the firm’s data lake and data warehouse Troubleshooting data issues, performing log analysis, and owning resolution processes Supporting both manual and automated testing for new data solutions More ❯