76 to 100 of 265 Data Lake Jobs in London

My client is based in the London area are currently looking to recruit for an experienced AWS Databricks Data Architect to join their team. They are one of the leaders within the consulting space and are currently going through a period of growth and are looking for an experienced Data Architect to join their team. They … the market. Your role will include: Responsible for designing and implementing effective Architectural solutions around the AWS Severless Technologies (S3, Lambda, Athena, Kafka) and Databricks including Data Lake and Data Warehousing. Assess database implementation procedures to ensure they comply with GDPR and data compliance. Guide, influence and challenge the technology teams and stakeholders … Terraform, Excellent understanding of Data Modelling & Kinesis An understanding of SQL/database management. Strong hands-on experience in Data Warehouse and Data Lake technologies preferably around AWS. My client is providing access to; Hybrid 2 days (London), 28 Days Holiday, Plus Bank Holiday Private Medical Health Pension Scheme And More... This role More ❯

Job Overview We are looking for a skilled Database Designer/Senior Data Engineer to join our data engineering team. This role involves the development and maintenance of data solutions in the cloud environment. The ideal candidate will have a strong background in data processing and a solid understanding of cloud data services. Requirements Key Responsibilities Design and implement modern data platforms in cloud such as AWS or Azure. Develop data pipelines and data streaming processes. Ensure data solutions are scalable, reliable, and efficient. Collaborate with data analysts and other stakeholders to meet data requirements. Optimize data … services. Proficiency in SQL, Python, or Scala. Experience with AWS RDS, AWS Glue, AWS Kinesis, AWS S3, Redis and/or Azure SQL Database, Azure Data Lake Storage, Azure Data Factory. Knowledge of data modelling, ETL processes, and data warehousing principles. Preferred Skills Strong problem-solving and analytical skills. Experience More ❯

Job Overview We are looking for a skilled Database Designer/Senior Data Engineer to join our data engineering team. This role involves the development and maintenance of data solutions in the cloud environment. The ideal candidate will have a strong background in data processing and a solid understanding of cloud data services. Requirements Key Responsibilities Design and implement modern data platforms in cloud such as AWS or Azure. Develop data pipelines and data streaming processes. Ensure data solutions are scalable, reliable, and efficient. Collaborate with data analysts and other stakeholders to meet data requirements. Optimize data … services. Proficiency in SQL, Python, or Scala. Experience with AWS RDS, AWS Glue, AWS Kinesis, AWS S3, Redis and/or Azure SQL Database, Azure Data Lake Storage, Azure Data Factory. Knowledge of data modelling, ETL processes, and data warehousing principles. Preferred Skills Strong problem-solving and analytical skills. Experience More ❯

Senior Data Engineer - SC Cleared Remote (UK-based) £75,000 - 85,000 + Benefits Full-Time | Central Government Projects We're looking for an experienced Senior Data Engineer to lead the design and delivery of secure, scalable, cloud-native data solutions across high-impact central government programmes. This is a fully remote role , but … active SC clearance is essential . Applicants without current SC clearance cannot be considered. As a senior member of the Data Engineering team, you'll play a pivotal role in designing high-quality data pipelines, guiding delivery teams, and collaborating with cross-functional experts to solve complex data challenges. You'll champion engineering best … stakeholders to translate requirements into solutions Promoting engineering standards and contributing to internal capability building You'll Need: Active SC Clearance Proven experience with Microsoft Azure (Data Lake, Synapse, Purview) Strong understanding of secure data handling, DevOps, and compliance Experience with structured and unstructured data sources (e.g., Azure SQL, MongoDB) Familiarity with Power More ❯

thefoundation for delivering our full suite of capabilities. Job Description Employment Type: Full-Time Location: UK Job Overview We are looking for a skilled Database Designer/Senior Data Engineer to join our data engineering team. This role involves the development and maintenance of data solutions in the cloud environment. The ideal candidate will … have a strong background in data processing and a solid understanding of cloud data services. Requirements Key Responsibilities Design and implement modern data platforms in cloud such as AWS or Azure. Develop data pipelines and data streaming processes. Ensure data solutions are scalable, reliable, and efficient. Collaborate with … services. Proficiency in SQL, Python, or Scala. Experience with AWS RDS, AWS Glue, AWS Kinesis, AWS S3, Redis and/or Azure SQL Database, Azure Data Lake Storage , Azure Data Factory. Knowledge of data modelling , ETL processes, and data warehousing principles. Preferred Skills Strong problem-solving and analytical skills . More ❯

Data Engineer/Scientist Department: Consultancy Employment Type: Full Time Location: United Kingdom/Hybrid Description As a Data Engineer/Scientist at Actica, you will have the opportunity to design, build, and maintain data pipelines while developing advanced analytics solutions to unlock business problems for high-profile UK public sector organisations. Your expertise … will enable organisations to maximise the value of their data assets through robust data infrastructure and sophisticated analysis, playing a key role in nationally critical projects that make a real difference to people's everyday lives. Locations: London, Guildford, Bristol, M4 corridor Hybrid working Roles and Responsibilities Actica recognises that data engineering, analytics, and … data science are distinct but interconnected disciplines. Our data engineers focus on building and maintaining the data infrastructure that enables analytics and data science work, while our data scientists and analysts focus on deriving insights and developing models. At Actica, we're at the forefront of the UK government's More ❯

Shape the Future of Data - Azure Data Architect - Hybrid - London SE1 Job Title: Data Architect - Azure Platform Specialist Location: Hybrid - 1-2 days per week in London (SE1). 3-4 days remote Salary: £90 - £115K OTE including bonus Start: ASAP The Client: A Microsoft-focused digital consultancy at the forefront of AI, Cloud … and Data transformation. As a Microsoft Fabric Featured Partner, they design and deliver cutting-edge data solutions to clients in sectors including finance, pharma, energy, insurance, and retail. The Candidate: Were looking for a confident and technically strong Azure Data Architect with experience shaping modern, scalable data platforms. The role would suit … more of the following platforms: Azure Synapse Analytics and Azure Databricks, and supporting data technologies like Power BI, Microsoft Purview, Azure SQL Database, Azure Data Lake, etc. Excellent stakeholder engagement and communication skills. Able to consult, advise, architect and estimate project work. Strong analysis, requirements gathering and data modelling. Solid data More ❯

objections, collaborating with engineering to share insights and improve products. Maintain deep expertise in Analytics Portfolio: Microsoft Fabric (OneLake, DW, real-time intelligence, BI, Copilot), Azure Databricks, Purview Data Governance, and Azure Databases: SQL DB, Cosmos DB, PostgreSQL. Maintain and grow expertise in on-prem EDW (Teradata, Netezza, Exadata), Hadoop & BI solutions. Represent Microsoft through thought leadership in … from migration & modernization and creating new AI applications. Expertise in Azure Analytics (Fabric, Azure Databricks, Purview) and competitors (BigQuery, Redshift, Snowflake) in data warehouse, data lake, big data, analytics, real-time intelligence, and reporting with integrated Data Security & Governance. Proven ability to lead technical engagements (e.g., hackathons, PoCs, MVPs) that drive More ❯

officially designated Posts. Our UK-based staff work in King Charles Street, London, Abercrombie House in East Kilbride and in Milton Keynes. These roles contribute to FCDOs Data Strategy to improve insights to the business by provisioning it with reliable and trustworthy data sources and improving their delivery in a secure manner in compliance with Data … The team is primarily Azure focused but other Cloud technologies such as Oracle and Amazon Web Services (AWS) are used in FCDO. The successful candidates will build complex data pipelines (both Extract Transform and Load {ETL} and Extract Load and Transform {ELT}) in the Azure cloud platforms. You will work with structured and unstructured data, data lakes, data warehouses to service operational and analytical business needs. Job description The successful candidates will: Work collaboratively with Enterprise Data Colleagues to develop pipelines and data product that aligns with data architecture and engineering best practice. Develop data assets and products that assist Power Business Intelligence (BI More ❯

capability sits within CTO organization and is responsible for driving process transformation, risk reduction, and efficiencies through best-in-class technology solutions. Including, development of shared services, comprehensive data architecture, automation, end-to-end solution design for large scale transformation initiatives, and adoption of Generative AI solutions. Job Purpose : The Central Operations Enterprise Data Architect role … is a critical hire in the group to become part of the team that drives data strategy, architecture, and design across critical initiatives -payments services, global reconciliations, reg. reporting, liquidity management, and strategic general ledger. The role requires hands on experience in designing, executing, and managing enterprise-wide data architecture, including design of frameworks to standardize … data management, consumption, and data quality controls. Successful candidate will be working cross disciplines in engineering, architecture, and technology teams across Operations, Finance, CDO, and Technology functions to simplify data pipelines, codify standards, automate controls, improve analytics and reporting - with an aim to deliver value from data. The role report would report to Data More ❯

Amazon Last Mile - Routing and Planning DE Design, implement, and support data warehouse/data lake infrastructure using AWS big data stack, Python, Redshift, Quicksight, Glue/lake formation, EMR/Spark/Scala, Athena etc. • Extract huge volumes of structured and unstructured data from various sources (Relational/Non … relational/No-SQL database) and message streams and construct complex analyses. • Develop and manage ETLs to source data from various systems and create unified data model for analytics and reporting • Perform detailed source-system analysis, source-to-target data analysis, and transformation analysis • Participate in the full development cycle for ETL: design, implementation … validation, documentation, and maintenance. BASIC QUALIFICATIONS - 3+ years of data engineering experience - Experience with data modeling, warehousing and building ETL pipelines - 4+ years of SQL experience - Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS - Experience as a data engineer or related specialty (e.g., software engineer More ❯

result in low levels of production issues Awareness of industry best practice and new and upcoming technology trends Full solution architectural understanding across networking and software Experience of data programmes and data technologies, ideally gained within an enterprise scale organisation or within technology or development team or similar. Technical exposure to and/or experience in … a range of technologies and architectural best practice concepts, specifically Data … Engineering Design Snowflake expert with hands-on experience of data modelling and best practises MS Azure Data Stack - Data Factory, Data Lake Storage, Azure SQL, Azure ML, Power BI Automation of CICD pipelines across different technologies including Power BI, Snowflake and the Azure Data Stack Data Science More ❯

result in low levels of production issues Awareness of industry best practice and new and upcoming technology trends Full solution architectural understanding across networking and software Experience of data programmes and data technologies, ideally gained within an enterprise scale organisation or within technology or development team or similar. Technical exposure to and/or experience in … a range of technologies and architectural best practice concepts, specifically Data … Engineering Design Snowflake expert with hands-on experience of data modelling and best practises MS Azure Data Stack - Data Factory, Data Lake Storage, Azure SQL, Azure ML, Power BI Automation of CICD pipelines across different technologies including Power BI, Snowflake and the Azure Data Stack Data Science More ❯

Operations The Amazon ARTS (APEX-RoW Technology Solutions) team is looking for a Business Intelligence Engineer to optimize in one of the world's largest and most complex data warehouse environments. You are expected being passionate about working with huge database and someone who loves to bring datasets together to create dashboards and business reports. You are required … to have deep expertise in creation and management of datasets and optimization of database. You should be expert at implementing and operating stable, scalable data flow solutions from production systems into end-user facing applications/reports. You are required to have excellent business and communication skills to be able to work with business stakeholders to develop business … help business stakeholders to provide data pipeline solutions with Amazon/ARTS internal tech products and AWS services (e.g. Redshift, SageMaker, EMR, ETL tools, data lake); 3) develop scripts for maintenance automation; 4) execute the tech implementation with a clear milestone planning and lead it to smooth launch; 5) help business stakeholders ramp up with More ❯

Services Job ID: Amazon EU SARL (UK Branch) Locations considered: London, Paris, Madrid, Milan, Munich, Berlin Are you interested in building data warehouse and data lake solutions to shape the analytical backbone of the EU Heavy Bulky & Services team? We are hiring a Business Intelligence Engineer to scale up our data projects to … the next level. This position requires experience in building Business Intelligence solutions, paired with bringing a hands-on data engineering mentality. We seek individuals that enjoy to collaborate across stakeholders and that bring excellent statistical and analytical abilities to the team. Heavy Bulky & Services is a growing Amazon business, designed to enable Customers to enjoy website browsing, product … will have contributed to the evolution of their team's analytical practices. Experience of developing automated analytical solutions is preferred. Key job responsibilities - Design, develop and implement scalable data solutions, automate processes and accelerate analytics. - Collaborate with other Business Intelligence Engineers on a WW scope and implement data pipelines to automate manual solutions. - Develop analytics: deep More ❯

Senior Analytics Engineer We are looking for a Senior Analytics Engineer to join our Data Engineering team which sits under Tech and Data. We're looking for a Senior Analytics Engineer to join our team and help shape the data foundations for high growth areas of Funding Circle. We have opportunities across multiple teams, each tackling … unique challenges and building high impact data models to power critical reporting and self-serve analytics. This is a unique opportunity to help build a top tier Analytics Engineering function, with the full backing of the business. We're investing heavily in the people, tools, and technology needed to make it happen - and giving you the chance to … the finance they need to grow. With more than a decade of expertise under our belt, we've built a game-changer of a platform with cutting-edge data and technology that's reshaping the landscape of SME lending. Say goodbye to lengthy applications and hello to lightning-fast decisions! In just minutes, SMEs across the UK can More ❯

Our client a London based Technology and Data Engineering leader have an opportunity in a high growth AI Lab for an ‘AI Engineering Researcher' A UK based 'Enterprise' Artificial Intelligence organisation, focussing on helping accelerate their clients journey towards becoming 'AI-Optimal' - starting with significantly enhancing its abilities in leveraging AI & machine intelligence to outperform traditional competition. The … expanding research team of exceptional PhD computer scientists, software engineers, mathematicians & physicists, to use a unique multi-disciplinary approach to solving enterprise-AI problems. Principal Activities of role: Data Pipeline Development: • Design, develop, and maintain ETL processes to efficiently ingest data from various sources into data warehouses or data lakes. • Data Integration and Management: Integrate data from disparate sources, ensuring data quality, consistency, and security across systems. Implement data governance practices and manage metadata. • System Architecture: Design robust, scalable, and high-performance data architectures using cloud-based platforms (e.g., AWS, Google Cloud, Azure). • Performance Optimization: Monitor, troubleshoot, and optimize data More ❯

Our client a London based Technology and Data Engineering leader have an opportunity in a high growth AI Lab for an ‘AI Engineering Researcher' A UK based 'Enterprise' Artificial Intelligence organisation, focussing on helping accelerate their clients journey towards becoming 'AI-Optimal' - starting with significantly enhancing its abilities in leveraging AI & machine intelligence to outperform traditional competition. The … expanding research team of exceptional PhD computer scientists, software engineers, mathematicians & physicists, to use a unique multi-disciplinary approach to solving enterprise-AI problems. Principal Activities of role: Data Pipeline Development: • Design, develop, and maintain ETL processes to efficiently ingest data from various sources into data warehouses or data lakes. • Data Integration and Management: Integrate data from disparate sources, ensuring data quality, consistency, and security across systems. Implement data governance practices and manage metadata. • System Architecture: Design robust, scalable, and high-performance data architectures using cloud-based platforms (e.g., AWS, Google Cloud, Azure). • Performance Optimization: Monitor, troubleshoot, and optimize data More ❯

Our client a London based Technology and Data Engineering leader have an opportunity in a high growth AI Lab for an ‘AI Engineering Researcher' A UK based 'Enterprise' Artificial Intelligence organisation, focussing on helping accelerate their clients journey towards becoming 'AI-Optimal' - starting with significantly enhancing its abilities in leveraging AI & machine intelligence to outperform traditional competition. The … expanding research team of exceptional PhD computer scientists, software engineers, mathematicians & physicists, to use a unique multi-disciplinary approach to solving enterprise-AI problems. Principal Activities of role: Data Pipeline Development: • Design, develop, and maintain ETL processes to efficiently ingest data from various sources into data warehouses or data lakes. • Data Integration and Management: Integrate data from disparate sources, ensuring data quality, consistency, and security across systems. Implement data governance practices and manage metadata. • System Architecture: Design robust, scalable, and high-performance data architectures using cloud-based platforms (e.g., AWS, Google Cloud, Azure). • Performance Optimization: Monitor, troubleshoot, and optimize data More ❯

the role: A candidate will be a self-starter who is passionate about discovering and solving complicated problems, learning complex systems, working with numbers, and organizing and communicating data and reports. You will be detail-oriented and organized, capable of handling multiple projects at once, and capable of dealing with ambiguity and rapidly changing priorities. You will have … s degree in Computer Science, Information Technology, or a related field Proficiency in automation using Python Excellent oral and written communication skills Experience with SQL, ETL processes, or data transformation Preferred Qualifications Experience with scripting and automation tools Familiarity with Infrastructure as Code (IaC) tools such as AWS CDK Knowledge of AWS services such as SQS, SNS, CloudWatch … Intelligence Engineer in the team, you will collaborate closely with business partners, architect, design, implement, and BI projects & Automations. Responsibilities: Design, development and ongoing operations of scalable, performant data warehouse (Redshift) tables, data pipelines, reports and dashboards. Development of moderately to highly complex data processing jobs using appropriate technologies (e.g. SQL, Python, Spark, AWS More ❯

Role Title: Data Modeler Duration: contract to run until 31/12/2025 Location: London, Hybrid 2-3 days per week onsite Rate: up to £644 p/d Umbrella inside IR35 Role purpose/summary We are seeking an experienced Data Modeller with proven expertise in the London Market insurance sector. The successful candidate … will play a key role in designing and validating data models that support enterprise data initiatives. This includes working closely with data engineers, architects, and business stakeholders to ensure data structures are scalable, accurate, and aligned with business needs. Key Skills/requirements Design and maintain conceptual, logical, and physical data … to explain complex data concepts to non-technical stakeholders. Preferred Skills: Experience with insurance platforms such as Guidewire, Duck Creek, or legacy PAS systems. Knowledge of Delta Lake, Apache Spark, and data pipeline orchestration tools. Exposure to Agile delivery methodologies and tools like JIRA, Confluence, or Azure DevOps. Understanding of regulatory data requirements More ❯

Who we are looking for We are looking for a Data engineer for Cloud Data Lake activities. The candidate should have industry experience (preferably in Financial Services) in supporting enterprise applications, and exposure to Cloud based Data engineering platforms. This hands-on role will partner with senior-level development managers, architects and business … leadership to develop and execute the technology product roadmap in driving Data Governance policy implementation. The candidate should have industry experience (preferably in Financial Services) in supporting enterprise applications and exposure to Cloud based engineering platforms. What you will be responsible for Collaborate with cross-functional teams to understand project requirements, user stories and translate them into functional … in this role Strong skill-set, experience in Python, Scala/PySpark, PL/SQL, PERL/scripting . Skilled Data Engineer for Cloud Data Lake activities. The candidate should have industry experience (preferably in Financial Services) in navigating enterprise Cloud applications using distributed computing frameworks as Apache Spark, Hadoop, Hive. Working knowledgeoptimizing database performance More ❯

As a Data Engineer, You will be working on building and maintaining complex data pipelines, assemble large and complex datasets to generate business insights and to enable data driven decision making and support the rapidly growing and dynamic business demand for data. You will have an opportunity to collaborate and work with various teams … of Business analysts, Managers, Software Dev Engineers, and Data Engineers to determine how best to design, implement and support solutions. You will be challenged and provided with tremendous growth opportunity in a customer facing, fast paced, agile environment. Key job responsibilities Design, implement and support an analytical data platform solutions for data driven decisions … AWS technologies such as S3, RedShift, Lambda, Glue, etc and Explore and learn the latest AWS technologies to provide new capabilities and increase efficiency Work on data lake platform and different components in the data lake such as Hadoop, Amazon S3 etc. Work on SQL technologies on Hadoop such as Spark, Hive, Impala etc More ❯

A leading, technology-focused, multi-strategy Hedge Fund are seeking a Senior Data Engineer (Front Office) to join their innovative and highly experienced team Essential Skills: 5+ years of experience with a proven track records of building data platforms Expert-level Python and SQL skills (C++ experience is a plus, but not essential) Interest in contributing … merging technology and research to solve some of the most complex challenges and deliver high-quality returns. This flexible role (2 days WFH) involves building and supporting bespoke data solutions for Quants and Traders, and developing a scalable, global Data Lakehouse used by multiple trading … desks. You’ll design and implement data pipelines in Python within an AWS cloud environment, working on data modelling for large-scale Data Lake and OLAP systems. Key responsibilities will include optimising performance in cross-region setups, ensuring data completeness and quality, and contributing to the software and infrastructure design. This More ❯

A leading, technology-focused, multi-strategy Hedge Fund are seeking a Senior Data Engineer (Front Office) to join their innovative and highly experienced team Essential Skills: 5+ years of experience with a proven track records of building data platforms Expert-level Python and SQL skills (C++ experience is a plus, but not essential) Interest in contributing … merging technology and research to solve some of the most complex challenges and deliver high-quality returns. This flexible role (2 days WFH) involves building and supporting bespoke data solutions for Quants and Traders, and developing a scalable, global Data Lakehouse used by multiple trading … desks. You’ll design and implement data pipelines in Python within an AWS cloud environment, working on data modelling for large-scale Data Lake and OLAP systems. Key responsibilities will include optimising performance in cross-region setups, ensuring data completeness and quality, and contributing to the software and infrastructure design. This More ❯