1 to 25 of 127 Permanent Data Lake Jobs in London

DATA ENGINEER LONDON BASED £55,000-65,000 PER ANNUM This UK-based Music company are searching for a … new Data Engineer, you will be joining their product engineering organisation, and will be responsible for building a new data lake in AWS. You will also be responsible for building new data pipelines in python. THE COMPANY The company is one of the … the team and will be using Python to code. Work collaboratively with the data science team, to create a data lake using AWS. Build and maintain Data Pipelines using Python. Develop real-time streaming features using big data tools such as more »

DFIR, threat intelligence, DLP, deception technologies, XDR and vulnerability management In-depth knowledge of at least one SIEM platform or security data lake and related processes Knowledge of various security tools, their functions and comparisons Knowledge of network and cloud security fundamentals Ability to explain complex technical more »

Data Technical Lead Location: Hybrid - London Package: Negotiable + Benefits Brown & Brown, Inc. is one of the largest insurance brokers globally having in excess of $4bn revenue in 2023 by providing risk management solutions to help protect what our customers value most. We have experienced significant growth over … size of the business by improving customer experience, growing organically and acquiring, accelerating knowledge and capabilities of teammates and using technology with purpose. Data and Data services is a key strategic enabler in Brown and Brown Europe as we look to integrate acquired business, enable them … Technical Skills – Deep hands on experience with the following data technologies: Stack including, Azure Data factory, Azure Data Lake Storage, Azure SQL, Azure Cosmos DB, Azure Power Platform, primarily power BI Data Lake & Data Mesh implementation The rewards more »

Purpose of Job We are seeking an experienced and skilled Data Platform Delivery Lead to join our EMEA Data & Analytics Technology team. They will be responsible for leading a team of talented professionals in the successful delivery of and long-term viability of the Data Platform for SMBC EMEA. The ideal candidate should have a strong background in data platform delivery, data engineering, project management, and leadership skills to drive the team towards achieving strategic objectives. They should be curious about the technology and design of the platform as … and robust designs for delivery. Determining the scope of projects. Ensure timely and high-quality delivery of solutions. Aligning the implementation of the Data Platform to the agreed strategy of the platform. Collaborating with stakeholders in EMEA Data & Analytics Tech and outside. Especially the rest of more »

Senior Data Engineer About us HENI is a technology company pioneering art markets and information. We work with world-leading artists and estates across various sectors, including printmaking, book publishing, NFTs, digital content, and art research – all underpinned by cutting-edge research in analytics, AI, and machine learning. … Our team comprises of talented web developers, data engineers, data scientists, and infrastructure specialists who thrive on solving complex problems. We enjoy developing machine learning tools and embracing the latest in infrastructure and coding methodologies. We maintain high standards of technical quality via thorough testing and … ingestion, Prefect for pipeline orchestration, AWS Glue for managed ETL, along with Pandas and PySpark for pipeline logic implementation. We utilize Delta Lake and PostgreSQL for data storage, emphasizing the importance of data integrity and version control in our workflows. Day-to-day more »

an unparalleled educational experience. By attracting and retaining the finest educators worldwide, our client ensures an environment where excellence thrives. Job PurposeAs the Data Platform Engineer overseeing the Data Warehouse and analytics infrastructure, you will shape the backbone of our client's business analysis and reporting … capabilities, both now and into the future. They are seeking a visionary with a proven track record of architecting enterprise-level data ecosystems, someone who thrives on pushing the boundaries of what's possible in data manipulation. Your expertise will ensure data is not … management.Remarkable experience with Development and maintenance of data pipeline and ETL processes using Azure data Factory, Azure Data Lake and Azure Databricks.Exceptional experience developing codes, testing for quality assurance, administering Azure Databricks platform and exposure to Unity Catalog.Exposure of Performance Tuning, Optimization, Governance more »

AZURE DATA ARCHITECT £100K + BENEFITS & BONUS HYBRID WORKING - TRAVEL TO LONDON REQUIRED My client is a Microsoft Gold Partner who build data & AI driven services, products, platforms and experience on the Azure platform. They are leading in their space and growing due to significant client … demand. I'm looking for an Azure Data Architect to join the team at a pivotal time in the company's growth. This role will be responsible for engaging with varying customers to understand their business challenges and goals surrounding data. You will work with senior stakeholders to … in leading workstreams in these engagements Commercial experience in the Azure Data Platform (Azure Data Factory, Azure Data Lake storage, Azure Synapse, Databricks, Azure SQL etc) Experience with Data Governance and Purview A strong pedigree in SQL ideally across the full more »

Key Knowledge/Skills Detailed working knowledge of ETL/ELT, data warehousing/business intelligence methodologies and best practice including dealing with big data, cloud technology and unstructured data and the relative required approaches Knowledge of star schema structure & design, detailed understanding of … hybrid data warehouse design Detailed knowledge using cloud data products like Data Factory, Event Hubs, Data Lake, Synapse, Azure SQL server. Detailed knowledge in developing in Databricks and experience in coding with PySpark. Spark SQL ETL coding standards: ensuring that code … is standardised, self-documenting and can be reliably tested Knowledge of best practice data encryption techniques and standards Knowledge of relevant national and international legislation pertaining to the Government/Public Sector IT – e.g. Data Protection Act, EU Procurement Directives and Freedom of Information Act, including more »

DATA ARCHITECT £100,000 - £110,000 HYBRID - LONDON This scale-up telecom group is looking for someone to be hands-on when driving forward their platform's expansion to incorporate data from all business areas as they become more digitally driven... THE COMPANY This group is … to grow into other areas. They aren't just localised to this area though as they've expanded into a global presence. Their data function has undergone extreme change over the two years and they are now in a good position for you to join the team! THE … EXPERIENCE To be a successful Data Architect in this area you should be proficient in the following: Expertise in Data Lake migration Azure experience is going to be beneficial Experience with big data Previous python & engineering experience will be useful to understand the more »

Lead Data Engineer £80- £100K Hybrid, 3 days a week in Chiswick, London Our client a global brand based out of lovely offices in West London are looking for a Lead Data Engineer to work on a business-critical, brand new, innovative high availability application which … who is looking to take their next step in their career and wants to take on more responsibility. Technical Skillset: Experience as a data engineer developing and maintenance of ETL/ELT processes Good experience in Data Modelling within a cloud … based data platform Strong experience with SQL Server Azure data engineering stack, including Azure Synapse and Azure Data Lake Python, PySpark and T-SQL In return you will be offered a competitive salary and benefits package, remote working options and an opportunity to more »

is a predominantly remote role with some occasional travel into Canary Wharf. The main activities of the role will be: Understand and implement data fabric principles and methodologies to enhance data ingestion and processing. Design and implement compliance scanning solutions on on-prem/cloud along … Information Technology, or a related field. Extensive experience in Infrastructure DevOps and Automation within the financial services industry. Strong understanding of data lake principles, data fabric, and related methodologies. Deep expertise in Infrastructure As Code (IaC) principles and best practices. Proficiency with IT tooling and … workplaces Exclusive discounts with major retailers, discount gym memberships and access to our wellness centre Protecting your privacy and the security of your data is a longstanding top priority for Bell Integration. Please consult our Privacy Notice (click here) to know more about how we collect, use and more »

Cloud Architect/Data Architect Google Cloud/GCP, Machine Learning/Generative AI London Based - New Offices, Refurbed in the Heart of a buzzing area £90k-£120k basic salary + bonus Benefits: 25 days holiday + bank holidays Private Healthcare Discounted Gym Membership (Nuffield + Hussle) Life … Insurance, Income Protection, Employee Assistance Program Pension Scheme (max 10%) We are working with an AI/Data Consultancy with core clients within the Retail, Financial Services, Gaming, SaaS sectors. The successful Cloud Architect will work with an array of technologies with clients from several industries & sectors meaning … devising innovative solutions to address their needs, defining project parameters, and supervising the execution of these projects. Role of the Cloud Architect/Data Architect: Utilising advanced data processing and cloud technologies effectively to stimulate innovation. Engaging with clients to create top-notch solutions tailored to more »

My client are looking for a talented and motivated Big Data Architect (Azure, Databricks, Spark) to be based … in their London office. You'll be responsible for providing technical leadership in architecting and designing end-to-end solutions for the organisation's datalake initiatives, as they provide increasing numbers of services to consumers across multiple digital channels. You'll be responsible for managing a remote engineering team of …/experience is essential: Strong implementation experience using Spark and Databricks Strong Cloud experience (ideally Azure) Previously heavily involved in an implementation programme Data Warehouse Strong stakeholder management experience Excellent IT background, ideally within Telco/Wireless service providers but not essential Salary: Excellent + bonus + package more »

Senior Developer (Data & Analytics) £500-£575 Inside IR35 12 months Hybrid Working - 2 full days a week working in London Talent are seeking a highly skilled Senior Developer with extensive experience in ETL/ELT processes, data warehousing/business intelligence methodologies, and best practices. The … methodologies. Develop and manage cloud-based data solutions using Azure services such as Data Factory, Event Hubs, Data Lake, Synapse, and Azure SQL Server. Create and optimize data processing workflows in Databricks using PySpark and Spark SQL. Ensure ETL coding standards … hybrid data warehouse design. Extensive experience with Azure data products including Data Factory, Event Hubs, Data Lake, Synapse, and Azure SQL Server. Proficient in developing with Databricks, PySpark, and Spark SQL. Strong understanding of ETL coding standards, including standardized, self-documenting more »

Data Engineer Location: London (hybrid) Salary: Competitive, based on experience and expertise **Note: Due to the nature of client work, candidates must be sole British nationals eligible to work on secure client sites.** Are you ready to join a high-achieving team that creates innovative solutions? We are … a premier recruitment agency seeking an exceptional Data Engineer for a prestigious consulting firm renowned for its groundbreaking solutions and industry leadership. If you're passionate about cloud technologies, data engineering, and digital transformation projects, this role is an exciting opportunity for you! About the Role … As a Data Engineer, you'll join our dynamic and forward-thinking Engineering Team, participating in a broad spectrum of digital projects that span cultural awareness, business continuity, and more. You will have the autonomy to manage your time and diary to suit your work/life balance more »

Location : London, Must have SC Clearance Certificate Mode: Hybrid Salary: Competitive Overview: We are seeking a talented Data Engineer with expertise in Azure to join our team. The ideal candidate will be responsible for developing data pipelines, building and maintaining data warehouses and data lakes, implementing data integration and transformation solutions, ensuring data quality and integrity, and optimizing data performance. Responsibilities: Proven track record in the development and management of an Azure cloud-based data & analytics platform. • Comprehensive experience in the complete data … and maintaining data warehouses and data lakes on Azure, utilizing services such as Azure Synapse Analytics, Azure Data Lake Storage, etc. Expertise in implementing data integration and transformation solutions using Azure services like Azure Data Factory, Azure Logic Apps more »

An exciting opportunity for an experienced Data Engineer to join the Engineering team working within the Digital & Product Department. Location: Chiswick (West London) Work Type: 3 days in office, 2 day from home Salary: £80k - £90k dependent on experience The successful candidate will lead a team of … data engineers to design and create scalable data pipelines. This hands-on role involves discussing requirements, designing solutions, and coding to automate data ingestion, cleansing, and transformation. You'll collaborate with Products, Engineering, Data, and Business Analytics teams to understand data … with SQL Server and relational databases Good knowledge of the Azure data engineering stack, including Azure Synapse and Azure Data Lake Programming experience in Python, PySpark and T-SQL Familiarity with the wider Azure Data Solutions such as Azure ML Studio Previous experience more »

Data Engineer | ETL | Python | ML | Insuretech | £80,000 | Barnet WeDo has recently partnered with a leading Insurtech company who are currently in the market for a Data Engineer to join the business. They are looking for someone who has worked very much in a Data Engineering capacity however, they need to have showcased prior experience of working with Data Science teams on ML Pipelines and Data Platforms and have a good understanding of modelling/practices. This extends to productionising and having a good grasp on some of the … CI/CD tooling. This position will be sat within the Data Science function within the business. The Data Engineer will have the following experience and skillset. Proficient in Microsoft Azure, Python/PySpark, and SQL. Experienced in using ETL/ELT tools like SSIS and more »

to this strategy is our direct-to-consumer, e-commerce offering. To enhance our e-commerce capabilities, we are looking to recruit a Data Analyst to define and deliver a data-driven approach to our online presence. ABOUT THE ROLE As the Data Analyst … with our Chief Digital Officer, Head of E-commerce, and the wider Digital Team. Your role will involve defining, developing, and delivering our data analysis, reporting, and online proposition optimisation strategies. From analysing web traffic and customer data to assessing competitor performance and product profitability, your … essential as you will present complex analyses to stakeholders, including the CEO. Whether your background is in e-commerce, tech, finance, or another data-centric field, we seek someone with a keen eye for numbers and the ability to identify and clearly communicate trends and opportunities. WHAT YOU more »

familiarity with AWS security solutions. Understanding of fundamental security concepts and terminologies (CIA, basic cryptography knowledge etc.) Familiarity with SIEM or Data Lake tools (Splunk, Elastic Stack, OpenSearch) Experience with designing/deploying/configuring security tooling Good knowledge of identified operating system platforms; routers, network protocols more »

plan of an additional 5% of your base salary paid by us monthly to save for your future. Your Role and Responsibilities Develop data products to enable onward use of data by product teams. Support the client's architecture community in solution design Ensure solution design … adheres to best practice. Develop future roadmap of data processing capabilities: Create implantation of data processes to transform raw service outputs according to data science use case needs. Create data linking mechanisms to remove barriers to data access Develop and … working with Azure data platform - building ETL in Azure Data Factory, taking data from Azure Data Lake into star schemas in Azure Synapse. Preferred Technical and Professional Expertise Databrix and Machine Learning skills/experience About Business UnitThe Client Innovation Centre more »

CENTRAL LONDON – HYBRID (3 DAYS A WEEK) £50,000 - £70,000 A great opportunity for a BI Engineer to bridge the gap between Data Engineering and BI reporting at an asset management firm expanding their data and analytics team. THE COMPANY Join a forward-thinking asset … returns. With a focus on secured loans, high-yield bonds, and structured credit, they operate at the forefront of the financial markets, leveraging data and analytics to drive informed decision-making. THE ROLE AND RESPONSIBILITIES As a BI Engineer, you will be responsible for supporting data … fast-paced environment, working with cutting-edge technologies such as Alteryx, Tableau, SQL, and Python. Responsibilities: Contribute to the setup and maintenance of data infrastructure, including data lakes, warehouses, and SQL Server databases. Utilize your technical expertise in SQL and Python to extract, transform, and analyse more »

governance forms (both Technical Design Authority (TDA) & Architecture Review Board (ARB)). Chair the governance forums at times. Scoping and designing any relevant data migration exercise to target platform, identification of relevant tools to aid with migration. Create usable Solution Architecture documentation that can be consumed by internal … and external delivery and engineering teams Work closely with Lead Security Architect, Head of Technology & Data Architecture to ensure that: Technology & Data Architecture layers of the Target architecture are aligned to future strategic direction and taking on Security principles, capabilities, guardrails are reflected in Target Architecture … offer SaaS products Understanding of enterprise estate management in mixed SaaS, bespoke and platform environments. Technical aspects include Azure cloud, Azure Data Lake, Data Warehouse, Integration architectures (event, messaging, API), security models, threat modelling, IDP integration and security designs, MI platforms, portals, MS Power platform more »

Head of Data Engineering London Hybrid 6-12 month contract Day rate 700-800 inside IR35 Role Description: Head of Data Engineering Overview The Senior Head of Data Engineering is a strategic and leadership position responsible for overseeing the data engineering function … across the organization. This role requires a visionary leader with a deep understanding of data architecture , engineering best practices, and emerging technologies. The Head of Data Engineering will drive the development, implementation, and management of data infrastructure and ensure data accessibility, reliability … and security for the company's diverse needs. Key Responsibilities Leadership and Strategy - Develop and execute the data engineering strategy aligned with the company's business objectives. - Lead, mentor, and grow a high-performing data engineering team. - Collaborate with senior management to define data more »

F&O, Dynamics 365 Finance & Operations, D365 Finance & Operations, BI Developer, Business Intelligence Developer, ETL, SQL, BI, Business Intelligence, SSRS, PowerBI, Data Lake, Dataverse – London & Remote - £45-55k plus benefits Our client, a large end user organisation, are looking for a Dynamics F&O BI Developer … storage, processing, and analytics. Azure Synapse Serverless Leveraging Azure Synapse Serverless for scalable and cost-effective data analytics. Data Lake, Lake House and Delta Lake Experience Familiarity with Data Lake and Delta Lake technologies for storing and processing … BI projects. Experience working with Data Analysis Expressions (DAX) Extensive experience with SQL Experience with Data Verse Data Lake Experience Location: London/Remote Catch Resource Management is a leading provider of Dynamics 365, JD Edwards, NetSuite and other ERP resources to both more »