26 to 50 of 128 Data Ingestion Jobs in London

to be the cornerstones of our organization and these values are upheld only because of our people. Role Essential Responsibilities Design and implement Azure data engineering solutions (Ingestion & Curation) Create and maintain Azure data solutions including Azure SQL Database, Azure Data Lake, and Azure Blob Storage. … Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure. Utilizing Azure Data Factory or comparable technologies, create and maintain ETL (Extract, Transform, Load) operations Use Azure Data Factory and Databricks to assemble large, complex data sets Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data. Ensure data quality/security and compliance. Optimize Azure SQL databases for efficient query performance. Collaborate with data engineers, and other stakeholders to understand requirements and translate them into scalable and reliable data More ❯

build out our internal engineering capability, spearheading our tech transformation, building best in class products and tackling exciting and complex challenges along the way! Data is at the core of what we do at QiH, but our people are at the heart of our success! At QiH, we have … culture and continuously invest in each individual. The Role Our Technology team is growing! We're looking for a highly skilled and experienced Lead Data Engineer to design and build a robust, scalable, and high-performance real-time data backbone. In this role you will take an individual … contributor position with leadership responsibilities focused on architecting, developing, and delivering real-time data pipelines and infrastructure. The ideal candidate will collaborate closely with engineering, product, and analytics teams to ensure seamless data flow across the organisation while leveraging cutting-edge technologies for real-time data processing. More ❯

I'm working with a client who is looking for a Data Engineer to take ownership of their data infrastructure as they build out a brand-new insurance-focused product suite. This is a rare opportunity to work directly with experienced data leaders who've collaborated on … platform. The business has secured 60% of its funding and is about to sign its first client. They've already designed tooling for AI, data migrations, and integrations-and now need someone to help build out these tools on top of Informatica, while translating SAP requirements and setting up … robust data infrastructure in the cloud. Please apply if the below sounds exciting to you-but only if you have strong experience with SAP, Informatica, and AWS. Key Responsibilities Build and optimize scalable data pipelines on Informatica Translate SAP data requirements into Informatica-based solutions Design tooling More ❯

I'm working with a client who is looking for a Data Engineer to take ownership of their data infrastructure as they build out a brand-new insurance-focused product suite. This is a rare opportunity to work directly with experienced data leaders who've collaborated on … platform. The business has secured 60% of its funding and is about to sign its first client. They've already designed tooling for AI, data migrations, and integrations-and now need someone to help build out these tools on top of Informatica, while translating SAP requirements and setting up … robust data infrastructure in the cloud. Please apply if the below sounds exciting to you-but only if you have strong experience with SAP, Informatica, and AWS. Key Responsibilities Build and optimize scalable data pipelines on Informatica Translate SAP data requirements into Informatica-based solutions Design tooling More ❯

We are a leading multi-strategy systematic hedge fund based in London, leveraging advanced technology and data to drive our trading strategies. Our team includes top quantitative researchers, data scientists, and engineers, all collaborating to develop innovative solutions. We are seeking an AI Data Engineer to join … our team, focusing on building and automating data systems that support AI-driven initiatives. Role Overview: As an AI Data Engineer, you will be responsible for designing, building, and optimizing data infrastructure to support AI and machine learning initiatives within our trading strategies. You will develop and … maintain data pipelines, ensure seamless data integration for AI models, and automate processes to improve efficiency. Your role will be pivotal in enabling the successful deployment and operation of AI-driven trading models. Key Responsibilities: Design, develop, and maintain scalable data pipelines that process large volumes of More ❯

We are a leading multi-strategy systematic hedge fund based in London, leveraging advanced technology and data to drive our trading strategies. Our team includes top quantitative researchers, data scientists, and engineers, all collaborating to develop innovative solutions. We are seeking an AI Data Engineer to join … our team, focusing on building and automating data systems that support AI-driven initiatives. Role Overview: As an AI Data Engineer, you will be responsible for designing, building, and optimizing data infrastructure to support AI and machine learning initiatives within our trading strategies. You will develop and … maintain data pipelines, ensure seamless data integration for AI models, and automate processes to improve efficiency. Your role will be pivotal in enabling the successful deployment and operation of AI-driven trading models. Key Responsibilities: Design, develop, and maintain scalable data pipelines that process large volumes of More ❯

The role We are looking for a Data Engineer to join the Data Science & Engineering team in London. Working at WGSN Together, we create tomorrow A career with WGSN is fast-paced, exciting and full of opportunities to grow and develop. We're a team of consumer and … design trend forecasters, content creators, designers, data analysts, advisory consultants and much more, united by a common goal: to create tomorrow. WGSN's trusted consumer and design forecasts power outstanding product design, enabling our customers to create a better future. Our services cover consumer insights, beauty, consumer tech, fashion … interiors, lifestyle, food and drink forecasting, data analytics and expert advisory. If you are an expert in your field, we want to hear from you. Role overview As a Data Engineer, you will be responsible for managing and improving the company's data ingestion, transformation, and More ❯

Job ID: AMZN Dev Cntr Poland sp. z.o.o Ring AI Data Management are seeking to hire a Data Engineer (a) to work in one of the largest and complex data warehouse (DW) environments, (b) to lead the design and implementation of the architecture of DW solutions for … daily experience in the design, creation, management, and business use of large datasets. Key job responsibilities - Work with engineering and business stakeholders to understand data requirements - Lead the design, model, and implementation of large, evolving, structured and unstructured datasets - Evaluate and implement efficient distributed storage and query techniques - Interact … and integrate with internal and external teams and systems to extract, transform, and load data from a wide variety of sources - Implement robust and maintainable code with clear and maintained documentation - Implement test automation on code implemented through unit testing and integration testing A day in the life The More ❯

Sr. Delivery Consultant - Data Architect, ProServe SDT North The Amazon Web Services Professional Services (ProServe) team is seeking a skilled Delivery Consultant to join our team at Amazon Web Services (AWS). In this role, you'll work closely with customers to design, implement, and manage AWS solutions that … trends and emerging technologies Sharing knowledge within the organization through mentoring, training, and creating reusable artifacts BASIC QUALIFICATIONS - 5+ years' experience architecting and implementing data platforms (Data Lake, Data Lakehouse, Data Mesh, Data Warehouse) at enterprise scale - Experience building end-to-end data solutions … including data ingestion (batch/streaming), storage, orchestration, governance, security, analytics, and observability. - Highly technical and analytical mindset with ability to think strategically about business, product, and technical challenges in an enterprise environment - Extensive hands-on experience with data platform technologies, including at least three of: Spark More ❯

We are seeking an experienced Informatica IDMC (Intelligent Data Management Cloud) Developer to join our team on a contract basis. This role involves hands-on development and implementation of Informatica cloud assets to support our ongoing data initiatives. You'll work closely with our internal teams to help … IDMC in a cloud environment - Design, build, and maintain workflows, pipelines, and assets across IDMC modules - Collaborate with internal stakeholders to define and implement data solutions - Ensure quality, performance, and governance across all developed assets - Provide documentation, best practices, and knowledge transfer to internal teams Essential Experience: - Candidates must … have strong hands-on experience in the following IDMC modules: - Platform Setup & Configuration - Data Governance & Catalog - Data Ingestion & Replication - Data Integration - Data Quality - Data Profiling - Data Validation - Metadata Command Center Ideal Candidate Profile: - Proven experience delivering Informatica IDMC projects in a contractor capacity More ❯

We are seeking an experienced Informatica IDMC (Intelligent Data Management Cloud) Developer to join our team on a contract basis. This role involves hands-on development and implementation of Informatica cloud assets to support our ongoing data initiatives. You'll work closely with our internal teams to help … IDMC in a cloud environment - Design, build, and maintain workflows, pipelines, and assets across IDMC modules - Collaborate with internal stakeholders to define and implement data solutions - Ensure quality, performance, and governance across all developed assets - Provide documentation, best practices, and knowledge transfer to internal teams Essential Experience: - Candidates must … have strong hands-on experience in the following IDMC modules: - Platform Setup & Configuration - Data Governance & Catalog - Data Ingestion & Replication - Data Integration - Data Quality - Data Profiling - Data Validation - Metadata Command Center Ideal Candidate Profile: - Proven experience delivering Informatica IDMC projects in a contractor capacity More ❯

Databricks Data Engineer - Contract Day Rate: £450 - £550 P/day Location: Hybrid (Remote/2 Days a week in London) Start: Immediate Start Note: Capital Markets, or at least Banking experience is a MUST! Key expertise and experience we're looking for: Data Engineering in Databricks - Spark … programming with Scala, Python, SQL Ideally experience with Delta Lake Databricks workflows, jobs, etc. Familiarity with Azure Data Lake: experience with data ingestion and ETL/ELT frameworks Data Governance experience - Metadata, Data Quality, Lineage, Data Access Models Good understanding of Data Modelling … concepts, Data Products and Data Domains Unity Catalog experience is a key differentiator - if not then experience with a similar Catalog/Data Governance Management component MS Purview (Metadata and Data Quality tool) experience is a bonus - experience in similar tools is valuable (Collibra, Informatica Data More ❯

Director of Data Engineering Location: Hybrid – London, Blackfriars Join us at Merlin Entertainments as we transform the future of digital guest experiences across our iconic global attractions. We're seeking an experienced Director of Data Engineering to lead the design, development, and implementation of Merlin's enterprise data architecture and engineering strategy. You'll oversee the data engineering and platform function, with a key focus on building a high-performing team and driving excellence across Databricks-based data platforms. About the Role A key focus will be managing the development and operational management of the … data platform using Databricks. The successful candidate will also lead a high-performing team of data engineers and platform specialists, fostering collaboration across technical and business teams to deliver innovative solutions that drive value and support data-driven decision-making. Key Responsibilities as Director of Data More ❯

Director of Data Engineering Location: Hybrid – London, Blackfriars Join us at Merlin Entertainments as we transform the future of digital guest experiences across our iconic global attractions. We're seeking an experienced Director of Data Engineering to lead the design, development, and implementation of Merlin's enterprise data architecture and engineering strategy. You'll oversee the data engineering and platform function, with a key focus on building a high-performing team and driving excellence across Databricks-based data platforms. About the Role A key focus will be managing the development and operational management of the … data platform using Databricks. The successful candidate will also lead a high-performing team of data engineers and platform specialists, fostering collaboration across technical and business teams to deliver innovative solutions that drive value and support data-driven decision-making. Key Responsibilities as Director of Data More ❯

delivery, and operation of all AWS global infrastructure. In other words, we're the people who keep the cloud running. We support all AWS data centers and all of the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely … own them to completion. AWS Infrastructure Supply Chain (AIS-SC) organization works to deliver innovative solutions to source, build and maintain our socially responsible data center supply chains. We are a team of highly-motivated, engaged, and responsive professionals who enable the core sustainable infrastructure of AWS. Come join … a part of history as we deliver results for the largest cloud services company on Earth! AWS is seeking a highly motivated and passionate Data Engineer who is responsible for designing, developing, testing, and deploying Supply Chain Analytical Solutions. In this role you will collaborate with business leaders, work More ❯

part of an agile team that's constantly pushing the envelope to enhance, build, and deliver top-notch technology products. As a Senior Lead Data Lake Engineer at JPMorgan Chase within the Corporate Technology, Chief Data and Analytics Office Development Team, you will play a pivotal role in … technologies (AWS or GCP) via hands on experience or certification Solid understanding of software applications and technical processes within a related technical discipline (e.g. data ingestion, data storage, data serving, APIs, etc.). Hands-on experience in data lake or data warehouse and related More ❯

Software is an artisanal software engineering firm built on the values of trust, respect, curiosity, and craftsmanship, and delivering purpose-built solutions to drive data-led transformation for organisations. Our emphasis is on craft as we create purpose-built solution, leveraging Data Engineering, Platform Engineering and Data Science with a razor-sharp focus to solve complex business and technology challenges, and provide customers with a competitive edge. As a Senior Data Engineer, you’ll feel at home if you are hands on, grounded, opinionated and passionate about delivering comprehensive data solutions that align with … modern data architecture approaches. Your work will range from building a full data platform to building data pipelines or helping with data architecture and strategy. This role is ideal for those looking to have a large impact and huge scope for growth, while still being hands More ❯

AWS Data Engineer (Must hold current SC) The Company: At Amber Labs, we are a cutting-edge UK and European technology consultancy that prioritises empowering autonomy, promoting experimentation, and facilitating rapid learning to provide exceptional value to our clients. Our company culture is centred around collaboration, where all colleagues … delivery times. Our team consists of highly-skilled cross-functional consultants, analysts, and support staff. Overview: We are looking for a highly skilled AWS Data Engineer to join our team. The ideal candidate will have expertise in designing, building, and managing scalable data pipelines on AWS. You will … work closely with data scientists, analysts, and software engineers to implement efficient data solutions that support business intelligence and analytics needs. Responsibilities: Design, develop, and maintain scalable ETL/ELT pipelines using AWS services such as AWS Glue, Lambda, Step Functions, and Kinesis. Work with structured and unstructured More ❯

REQUIRED: DATA ENGINEER - SC CLEARED LOCATION: FULLY REMOTE WITH OCCASIONAL TRAVEL TO CENTRAL LONDON IR35 STATUS: INSIDE DURATION: 6 MONTH INITIAL CONTRACT Seeking a Data Engineer who has a strong understanding of data concepts - data types, data structures, schemas (both JSON and Spark), schema management … etc - Strong understanding of complex JSON manipulation - Experience working with Data Pipelines using a custom Python/PySpark frameworks - Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing/handling Reference Data. - Strong understanding of Data Security … principles - data owners, access controls - row and column level, GDPR etc including experience of handling sensitive datasets - Strong problem solving and analytical skills, particularly able to demonstrate these intuitively (able to work a problem out, not follow a work instruction to resolve) - Experience working in a support role would More ❯

Job Title: Azure Data Engineer Lead - Outside IR35 Location: Remote Salary/Rate: £450 per day - Outside IR35 Start Date: 28/04/2025 Duration: 3 months We have an exciting opportunity with a major retail company. They are looking for a Lead Azure Data Engineer to … working outside IR35 and fully remote. Job Responsibilities/Objectives: Lead the design and execution of the end-to-end migration from on-premise data systems to Azure, ensuring scalability, reliability, and performance. Develop and optimise data pipelines using Azure Data Factory, Synapse Analytics, and related services … to support data ingestion, transformation, and integration. Architect and implement Azure-based data solutions, including Data Lake, SQL Database, and Databricks, aligning with business and technical requirements. Establish cloud data engineering best practices, including governance, security, monitoring, and cost optimisation. Collaborate cross-functionally with cloud More ❯

REQUIRED: DATA ENGINEER - SC CLEARED Location: Fully remote with occasional travel to Central London IR35 Status: Inside Duration: 6-month initial contract Job Description: Seeking a Data Engineer with a strong understanding of data concepts, including data types, data structures, schemas (JSON and Spark), and … schema management. Key Skills and Experience: Strong understanding of complex JSON manipulation Experience with Data Pipelines using custom Python/PySpark frameworks Knowledge of the 4 core Data categories (Reference, Master, Transactional, Freeform) and handling Reference Data Understanding of Data Security principles, access controls, GDPR, and … advanced proficiency, Pydantic bonus) SQL PySpark Delta Lake Bash Git Markdown Scala (bonus) Azure SQL Server (bonus) Technologies: Azure Databricks Apache Spark Delta Tables Data processing with Python PowerBI (Data ingestion and integration) JIRA Additional Notes: Candidates with current or past high-level security clearance are encouraged More ❯

Data Engineer 4 months - with extensions Remote Active SC clearance required £640 per day inside ir35 REQUIRED Strong understanding of data concepts - data types, data structures, schemas (both JSON and Spark), schema management etc Strong understanding of complex JSON manipulation Experience working with Data Pipelines … using a custom Python/PySpark frameworks Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing/handling Reference Data. Strong understanding of Data Security principles - data owners, access controls - row and column level, GDPR etc including … both CLI usage and scripting) Git Markdown Scala DESIRABLE Azure SQL Server as a HIVE Metastore DESIRABLE TECHNOLOGIES Azure Databricks Apache Spark Delta Tables Data processing with Python PowerBI (Integration/Data Ingestion) JIRA If you meet the above requirements, please apply for the vacancy to be More ❯

concepts and advanced applied experience Extensive development experience using Python/SQL Solid understanding of software applications and technical processes within related disciplines (e.g., data ingestion, data storage, data serving, APIs) Hands-on experience with data lake or data warehouse technologies (e.g., Spark, ETL More ❯

This is an exciting contract opportunity for an SC Cleared Azure Data Engineer to join an experienced team in a new customer engagement working at the forefront of data analytics and Artificial Intelligence. Our client equips ambitious organisations with enduring control of their data and AI evolution. … This role is an opportunity to lead the build of bespoke data systems for our clients. Responsibilities: Design and implement scalable data pipelines and ETL processes using Azure and Databricks technologies including Delta Live Tables. Lead technical discussions with clients and stakeholders to gather requirements and propose solutions. … Help clients realise the potential of data science, machine learning, and scaled data processing within Azure/Databricks ecosystem. Mentor junior team members and support their personal development. Take ownership for the delivery of core solution components. Support with planning, requirements refinement and work estimation. Skills and Experiences More ❯

Senior Data Engineer | London, Hybrid | £70,000-£90,000 DOE My client is growing fast and looking for a skilled Data Engineer to help shape the backbone of their data capabilities. You’ll play a key role in refining infrastructure, ensuring seamless data access, and supporting … a modern data platform built for scale. Working closely with teams across the business, you’ll help drive smarter decisions and long-term success through well-structured, reliable, and secure data systems. Key Responsibilities: Define and implement a modular data architecture designed to scale with business growth … Develop and maintain a cloud-based data warehouse using AWS to manage high-volume data ingestion Design robust, high-throughput data pipelines that scale with increased demand. Automate ETL/ELT processes and integrate multiple data sources to ensure real-time availability Promote high data More ❯