176 to 200 of 265 Data Lake Jobs in London

look to our Business Consultants, to drive change, inspire trust with clients, and take responsibility for some of the world's most ambitious transformation projects. As a Senior Data Technology Consultant within our successful Business Consulting practice, you, alongside a collaborative team of consulting colleagues, will take a central role in providing support to our clients with a … focus on driving their business outcomes using data and technology. You will also support the growth of the wider evolving business consulting function. A key focus of the role will be to deliver practical advice to clients relating to their enterprise data technologies and strategies whilst looking out for opportunities to support them further leveraging our … quality consulting services to clients, presenting technical concepts in simple, accessible language. • Leverage your deep understanding of legacy and cloud-based data technologies, including Data Lake, Data Warehouse, and Lakehouse concepts. • Bring broad technical knowledge of Azure data products, with subject matter expertise in areas like Azure Data Factory More ❯

Overview: 3 contract data engineers to supplement existing team during implementation phase of new data platform. Main Duties and Responsibilities: Write clean and testable code using PySpark and SparkSQL scripting languages, to enable our customer data products and business applications. Build and manage data pipelines and notebooks, deploying code in a structured … trackable and safe manner. Effectively create, optimise and maintain automated systems and processes across a given project(s) or technical domain. Data Analyse, profile and plan work, aligned with project priorities. Perform reviews of code, refactoring where necessary. Deploy code in a structured, trackable and safe manner. Document your data developments and operational procedures. Ensure adherence … to data/software delivery standards and effective delivery. Help monitor, troubleshoot and resolve production data issues when they occur. Contribute to the continuous improvement of the team. Contribute to the team's ability to make and deliver on their commitments. Innovate and experiment with technology to deliver real business benefits. Regularly launch products and services More ❯

end data pipelines in Azure Databricks, handling ingestion from various data sources, performing complex transformations, and publishing data to Azure Data Lake or other storage services. Write efficient and standardized Spark SQL and PySpark code for data transformations, ensuring data integrity and accuracy across the pipeline. Automate … systems). Implement data transformation logic using Spark, ensuring data is cleaned, transformed, and enriched according to business requirements. Leverage Databricks features such as Delta Lake to manage and track changes to data, enabling better versioning and performance for incremental data loads. Data Publishing & Integration: Publish clean, transformed data to Azure Data Lake or other cloud storage solutions for consumption by analytics and reporting tools. Define and document best practices for managing and maintaining robust, scalable data pipelines. Data Governance & Security: Implement and maintain data governance policies using Unity Catalog, ensuring proper organization, access control, and metadata management More ❯

We are seeking a Platform Technical Lead to spearhead the architecture, implementation, and evolution of amodern data platformfor a leadingspecialty insurance firmin London. This role requires astrong technical backgroundinintegration and data architectures, as well asleadership experiencemanaging both technical teams and client relationships. You will be responsible for thedesign, governance, automation, and operationof theenterprise data platform, ensuring its capabilities align with business needs. The platform is built onAzure, Snowflake, Data Lakes, and Kafka, requiring expertise acrosssecurity, data governance, integrations, observability, DevOps, and automation. This is a London-based role, as regular on-site client engagement is required. The position will behired by Marionete, a leader in delivering cutting-edge … data and technology solutions. Key Responsibilities Architect and leadthe design of an enterprise-grade data platform, ensuringscalability, security, and performance. Providetechnical leadershipacrossdata integration, data modeling, security, governance, and automation, covering all layers of a data platform (ingestion, storage, processing, and consumption). Drive the implementation ofmodern platform architecturessuch asLakehouse, Kappa, and Lambda More ❯

and innovative solutions, we free up our clients' potential, thereby fulfilling our own. Take it seriously. Make it fun. Know it matters. As we develop an enterprise-wide data and digital strategy that moves us toward greater focus on the use of data and data-driven insights, we are seeking a Data & Digital Solutions Architect. The Data & Digital Solutions Architect is a hands-on, execution focused role helping translate the data and digital strategy to architecture and to execution. In this role you will have an understanding of our data and analytical systems at a granular level, and function as organizational data … do Passion for data and experience working within a data driven organization Hands-on experience with architecting, implementing, and performance tuning of: Data Lake technologies (e.g. Delta Lake, Parquet, Spark, Databricks) API & Microservices Message queues, streaming technologies, and event driven architecture NoSQL databases and query languages Data domain and event More ❯

and Global Markets to engineer scalable solutions for our global trading desks. Our team applies specialist methods from mathematics, science, and engineering to generate revenue. We focus on data, models, and algorithms for derivative valuation, risk management, automated trading, and data-driven decision-making. At Macquarie, our advantage is bringing together diverse people and empowering them … always seeking the best tools to tackle each challenge, whether that means embracing new technologies or revisiting familiar ones. You will deliver solutions for large-scale software and data engineering challenges. Working closely with business teams, you will have the autonomy to innovate and build solutions from scratch, bringing your ideas to life. With full ownership of your … and pragmatic solutions with immediate impact Ability to work independently with exceptional proficiency, collaborating with diverse stakeholders to deliver comprehensive solutions from evolving requirements Experience with cloud and data/ML technologies such as AWS, Kubernetes, data lakes, and data pipeline orchestrators is advantageous We welcome applications from anyone inspired to build a better More ❯

Principal Solution Architect Job Summary We are seeking a highly skilled Principal Solution Architect with deep expertise in IT Architecture, AWS Cloud Technologies, and data platform implementation. The ideal candidate will have strong networking knowledge and hands-on experience designing and developing scalable data platforms. This role requires leadership skills, cross-functional collaboration, and the ability … to drive the development of a new global data platform. Key Responsibilities Architectural Design in cloud based environments: Develop and implement robust IT architecture strategies for cloud and hybrid environments, leveraging AWS best practices. Design scalable, secure, and high-availability solutions tailored to business needs. Architect and optimize data platforms to enable efficient data … Route 53, and CloudFront. Configure and troubleshoot complex networking scenarios, including hybrid cloud setups. Manage network performance and security standards, including SLAs, bandwidth, latency, and IPSec VPN configurations. Data Platform Implementation: Lead the design and development of a new global data platform (PaaS) Ensure scalable storage solutions (data lakes, data warehouses) to More ❯

enterprise-wide artificial intelligence strategies. The ideal candidate will be responsible for aligning AI initiatives with business goals, developing scalable AI architectures and ensuring seamless integration across systems, data sources and workflows. This role requires a strong blend of technical acumen, strategic vision and cross-functional leadership to drive innovation and digital transformation. Your future duties and responsibilities … and frameworks across the enterprise • Develop reference architectures and best practices for AI integration, deployment and lifecycle management • Oversee the integration of AI solutions with existing enterprise systems, data platforms and cloud services • Ensure data governance, model interpretability and compliance (e.g. GDPR, ethical AI standards). • Evaluate emerging AI technologies and vendors, make recommendations on adoption … and implementation • Stay updated on industry trends and provide thought leadership in enterprise AI adoption • Mentor cross-functional teams on AI architecture, tooling and frameworks • Bridge communication between data scientists, engineers and business stakeholders • Define AI governance frameworks to manage model drift, bias and performance monitoring • Establish guardrails for ethical and responsible AI usage Required qualifications to be More ❯

for a Lead Software Development Engineer to join us as part of our team. About the role PitchBook is a platform designed for investment professionals, offering access to data and analytical tools that provide quick answers and help discover promising opportunities. It uncovers actionable insights and trends within the financial data of over three million companies. … Experience with CI/CD K8s experience Experience with unit testing English: Upper-intermediate at least (both spoken and written) Bonus points Experience building systems requiring extensive ETL, data lakes/platforms, and data sourcing Experience with Google Cloud managed services Work with projects with microservice architecture What's in it for You Reveal great tech More ❯

DATA ENGINEER LONDON BASED £55,000-65,000 PER ANNUM This UK-based Music … company are searching for a new Data Engineer, you will be joining their product engineering organisation, and will be responsible for building a new data lake in AWS. You will also be responsible for building new data pipelines in python. THE COMPANY The company is one of the largest in the UK and … implementing best coding practices within the team and will be using Python to code. Work collaboratively with the data science team, to create a data lake using AWS. Build and maintain Data Pipelines using Python. Develop real-time streaming features using big data tools such as Spark. SKILLS AND EXPERIENCE Extensive More ❯

Azure Data Architect Start: ASAP Duration: initial 6 months Location: hybrid, must be accessible to Dunstable Pay: £500-520/day (inside IR35) We are seeking a highly skilled Azure Data Platform Architect to support a key client in the delivery of advanced cloud-based data solutions. You'll work within a leading global … customer stakeholders to gather and analyse requirements - Recommend tools and design patterns based on business needs - Produce and present architecture and design documentation for governance approvals - Oversee implementation of lake house architecture using Databricks and related tools Skills & Experience Required: - 5+ years of experience in Azure data platform implementation - Deep expertise across Azure services including Data Factory, Databricks, Data Lake, Synapse, Power BI, Key Vault, and more - Strong understanding of cloud data architecture and platform design patterns - Proven ability to lead design discussions and challenge stakeholders constructively - Skilled in producing and presenting technical documentation More ❯

This fantastic institution located in SW London is seeking a dynamic and versatile Data Engineer/Analytics Engineer to contribute to the development, enhancement, and management of their data systems, pipelines, and reporting infrastructure. This role will focus on designing and maintaining efficient data pipelines while building a scalable data warehouse architecture. … A major aspect of this position will involve integrating new data sources to create a unified Single Customer View. Additionally, collaboration with the Analytics and Insights teams is essential to ensure they have seamless access to the necessary data, as well as assisting in optimizing SQL reporting processes. The ideal candidate will: Possess a strong curiosity … UAT to Production. Maintain and evolve the Single Customer View to consolidate data from various sources. Demonstrate hands-on experience with Microsoft Azure technologies (including Synapse Analytics, Datalake, Azure SQL), as well as familiarity with DevOps and GitHub practices. Contract Details: Initially a Fixed Term Contract, likely transitioning to Permanent. 1-2 days a week on-site. More ❯

POSITION OVERVIEW As a data scientist, you'll play a pivotal role in developing and enhancing our innovative supply chain solutions. You'll collaborate closely with a talented team of passionate engineers and industry experts, applying data engineering and data science techniques to optimize supply chain processes. Your expertise will directly contribute to providing … machine learning models to improve the accuracy of supply chain ETA s. Analyze large, complex datasets to extract meaningful insights and identify trends. Design, build, and maintain scalable data pipelines and data lakes to ensure efficient data flow and storage for analytics purposes. Collaborate with cross-functional teams to integrate analytics solutions into our … products. Design and build data visualization tools to communicate findings to stakeholders. Continuously research and apply the latest techniques in AI and machine learning to supply chain problems. Contribute to the development of our analytics product suite, ensuring scalability and efficiency. Communicate complex data science concepts to non-technical stakeholders. Key Responsibilities 1. Advanced Predictive Models More ❯

Delivery Consultant - Data Analytics & GenAI, AWS Professional Services Public Sector Are you a Data Analytics and … GenAI specialist? Do you have experience with real-time Data Analytics, Data Warehousing, Big Data, Modern Data Strategy, Data Lake, Data Engineering, and GenAI? Do you enjoy solving complex, high-scale data challenges involving billions of records? Are you interested in leading teams through impactful … job responsibilities Expertise: Collaborate with pre-sales and delivery teams to help partners and customers utilize AWS services like AWS Glue, Amazon S3, DynamoDB, RDS, EMR, Kinesis, Redshift, Athena, Lake Formation, DataZone, SageMaker, Quicksight, and Bedrock. Solutions: Deliver technical engagements including pre-sales visits, understanding customer requirements, creating consulting proposals, and developing packaged data analytics services. Delivery More ❯

Data Engineering Consultant - Private Equity Our client is a fast-growing, independent consultancy specialising in delivering business and technical solutions to both top-tier and boutique investment firms. Since its founding in 2011, the company has built a strong international presence, supporting a diverse client base across Western Europe and North America-offering opportunities for regular travel. They … are now seeking an experienced professional to join their team and play a key role in shaping their data engineering strategy, enhancing the technical capabilities they deliver to clients. As a Data Engineering Consultant, you will lead the design and implementation of a forward-looking … data strategy aimed at modernising data practices within the private capital markets industry. The technology stack primarily leverages Microsoft Azure services, including Data Lake, Data Factory, Synapse, and Logic/Function Apps. Skills A strong academic background, ideally in a STEM-related field Deep expertise in Microsoft Azure, SQL, and Python More ❯

Principal Data Engineer – AWS Location: London (Hybrid/Flexible) - Other UK Locations available Salary: £95,000 - £105,000 + Bonus A leading consultancy is hiring for a Principal Data Engineer to join their fast-growing data team, delivering cloud-native data platforms for high-profile clients across sectors. This role will focus … on designing and implementing end-to-end data solutions using modern AWS tooling. It suits someone who thrives in greenfield environments, enjoys client engagement, and values clean, scalable, well-documented engineering. Key Responsibilities: Design and build robust data pipelines using AWS (S3, Redshift, Glue, Lambda, Step Functions, DynamoDB). Deliver ETL/ELT solutions with Matillion … pipelines in cloud environments. Proficient in working with structured/unstructured data (JSON, XML, CSV, Parquet). Skilled in working with relational databases and data lake architectures. Experienced with Matillion and modern data visualisation tools (QuickSight, Tableau, Looker, etc.). Strong scripting and Linux/cloud environment familiarity. Desirable: Exposure to big data More ❯

A prestigious organization based in London is seeking a dynamic and versatile Data Engineer/Analytics Engineer to contribute to the development, enhancement, and management of their data systems, pipelines, and reporting infrastructure. This role will focus on designing and maintaining efficient data pipelines while building a scalable data warehouse architecture. A … major aspect of this position will involve integrating new data sources to create a unified Single Customer View. Additionally, collaboration with the Analytics and Insights teams is essential to ensure they have seamless access to the necessary data, as well as assisting in optimizing SQL reporting processes. The ideal candidate will: Possess a strong curiosity and … UAT to Production. Maintain and evolve the Single Customer View to consolidate data from various sources. Demonstrate hands-on experience with Microsoft Azure technologies (including Synapse Analytics, Datalake, Azure SQL), as well as familiarity with DevOps and GitHub practices. This role could accommodate 1 visit a month/fully remote for the right individual and is a More ❯

Overview: The Manual Data QA will be responsible for executing comprehensive manual tests to ensure the quality of data flows across various Chambers products and platforms, with a primary focus on the Databricks environment. This role plays a critical part in maintaining the integrity and reliability of our data services. Main Duties and Responsibilities … Collaborate with cross-functional teams to understand data migration requirements and ensure smooth data transition to the cloud platform. Develop and execute comprehensive test plans for data migrations, database implementations, and ETL processes. Perform data validation and ensure data integrity throughout the migration process. Implement integrated test automation to streamline … load and performance tests to assess system scalability and identify potential bottlenecks. Skills and Experience: Proven experience in testing data migrations, database implementations, and Data Lake environments. Familiarity with cloud environments, especially Azure, is highly desirable. Strong knowledge of SQL and experience in writing complex queries for data validation and testing. Proficiency in More ❯

base that offers regular travel to Western Europe and North America. They are now looking to add an experienced individual to their ranks that can help shape their data engineering strategy and improve the technical offering they provide for their clients. As a Principal Data Engineer , you'll have the opportunity to design and implement a … modernise how data is used within the private capital markets industry. The tech stack is largely centred around the services within Microsoft Azure, including Data Lake, Data Factory, Synapse, Logic Apps, and Function Apps. Minimum Requirements: An excellent educational background, preferably in a STEM-based discipline. Expert knowledge of Microsoft Azure, SQL, and … in a client-facing environment. Previous experience working within the private equity market. Additional Information: Company: Graduate Recruitment Bureau (Hiring for client) Salary Range: £85K-100K Courses Considered: Data Science, Economics, IT, Maths, and Stats Course Notes: STEM-based degree; experience outweighs academics Related Job Opportunities: One of the largest providers of consulting and technology services, offering a More ❯

Senior Data Project Manager page is loaded Senior Data Project Manager Apply locations London time type Full time posted on Posted Yesterday time left to apply End Date: July 31, 2025 (30+ days left to apply) job requisition id R Who are we? Howden is a collective - a group of talented and passionate people all around … work/life balance, career progression, sustainability, volunteering - you'll find like-minded people driving change at Howden. About the Role: We are seeking an experienced and strategic Data Project Manager to lead complex projects within our Group Data Function . You will manage high-priority initiatives across global business units, ensuring the successful delivery of … data-driven and technology projects . A strong background in data lakes, analytics, and cloud platforms is highly preferred. You will drive best practices, provide leadership, and influence stakeholders at all levels. Key Responsibilities: Strategic Project Leadership: Oversee the end-to-end execution of high-impact, multi-regional data technology projects. Stakeholder & Executive Engagement More ❯

Are you a curious and open-minded individual with an interest in state-of-the-art machine learning engineering and research? Thomson Reuters Labs is seeking a Data Engineer with a passion for solving challenging machine learning problems in a data-rich, complex and innovative environment. What does Thomson Reuters Labs do? We experiment, we build … hire worldleading specialists in SWE/Applied ML, as well as Research, to drive the company's leading internal AI model development, fueled by an unprecedented wealth of data and powered by cutting-edge technical infrastructure. You will have the opportunity to contribute to a data curation & filtering system combining the best of real-world scalable … Experience with data pipeline orchestration tools. Experience with cloud-based data platforms such as AWS, GCP, or Azure (e.g., S3, BigQuery, Azure Data Lake Storage). Comfortable working in fast-paced, agile environments, managing uncertainty and ambiguity. Preferred qualifications : Additional legal knowledge as evidenced by a degree or interest in the legal domain. More ❯

heights by spearheading the Digital Supply Chain (DSC) team. Collaborate with visionary leaders and diverse colleagues in a global environment, unlocking your full potential. If you thrive on data and digital transformation, this role is designed for you. KEY RESPONSIBILITIES: Be a part of the Digital Supply Chain (DSC) team, a new team of 40+ FTE that will … used cases for deploying digital transformation and AI in the Procurement vertical. Evaluate the feasibility of used cases from both domain (procurement) as well as Tech perspective (available data, data structure, solution integration, current project pipeline, digital & analytical capabilities, etc.). Connect with the Procurement as well as IT teams and prepare a viable business case … for all Procurement technology requests. Oversee the correct migration of existing Procurement systems and dashboards from Unilever's systems into the Ice Cream Co's own data lake/systems. Manage the transition from the existing local-dominated tooling landscape into a centralized landscape, in line with the "Global First, Local by Exception" digital agenda. Through an More ❯

Are you a talented Solutions Architect looking to make your mark in the banking sector? Our client is seeking an enthusiastic and skilled individual to join a critical data remediation programme within the Markets Platform Surveillance Team. This is a fantastic opportunity to contribute to a dynamic environment and enhance the organization's capabilities while navigating a highly … umbrella) What You'll Do: As a Solutions Architect, you will play a pivotal role in improving our CI/CD platform and ensuring the seamless integration of data into our surveillance systems. Your expertise will help shape the future of our data handling and surveillance efforts. Key responsibilities include: Data Integration : Pulling data into the surveillance system from various sources, including data lakes and data warehouses on Google Cloud Platform (GCP). Platform Enhancement : Improving the CI/CD platform using Terraform and ensuring efficient deployment processes. Pipeline Tracking : Utilizing tools like JIRA to track pipelines and manage workflows effectively. Containerization : Implementing containerization solutions to optimize application More ❯

proven track record of delivering technical solutions in Agile environments with expertise in JIRA and Confluence. Proven Solution Architect Experience: Deep technical expertise in various solution architectures, especially data platforms and complex data pipelines on public cloud platforms, with a focus on GCP. Robust Application Design Skills: Experience in designing scalable, low-latency solutions that meet … diverse functional and non-functional requirements. Data Architecture Expertise: Proven experience in designing and delivering complex batch and streaming data pipelines, understanding orchestration, and database capabilities across multiple solutions. Problem-Solving Ability: A strategic and creative thinker who can tackle complex challenges and develop innovative, robust solutions. Collaboration & Communication Skills: Strong interpersonal skills that can influence … Architect certifications. Cyber & Data Security Understanding: An appreciation for the architectural implications of application design and implementation. Modern Data Capabilities: Knowledge of Data Lake, Data Warehouse, and Machine Learning architectures. QA Best Practices: A solid understanding of data quality concepts and practices. Why Join Us? This is your chance More ❯

proven track record of delivering technical solutions in Agile environments with expertise in JIRA and Confluence. Proven Solution Architect Experience: Deep technical expertise in various solution architectures, especially data platforms and complex data pipelines on public cloud platforms, with a focus on GCP. Robust Application Design Skills: Experience in designing scalable, low-latency solutions that meet … diverse functional and non-functional requirements. Data Architecture Expertise: Proven experience in designing and delivering complex batch and streaming data pipelines, understanding orchestration, and database capabilities across multiple solutions. Problem-Solving Ability: A strategic and creative thinker who can tackle complex challenges and develop innovative, robust solutions. Collaboration & Communication Skills: Strong interpersonal skills that can influence … Architect certifications. Cyber & Data Security Understanding: An appreciation for the architectural implications of application design and implementation. Modern Data Capabilities: Knowledge of Data Lake, Data Warehouse, and Machine Learning architectures. QA Best Practices: A solid understanding of data quality concepts and practices. Why Join Us? This is your chance More ❯