1 to 25 of 51 Data Ingestion Jobs in the City of London

As a Senior Data Engineer, you will be responsible for the development of complex data sources and pipelines into our data platform (i.e. Snowflake) along with other data applications (i.e. Azure, Airflow, etc.) and automation. The Senior Data Engineer will work closely with the data, Architecture, Business Analyst, Data Stewards to integrate and … align requirements, specifications and constraints of each element of the requirement. They will also need to help identify gaps in resources, technology, or capabilities required, and work with the data engineering team to identify and implement solutions where appropriate. Work type: Contract Length: initial 6 months Work structure: hybrid 2 days a week in London. Primary Responsibilities: Integrate data from multiple on prem and cloud sources and systems. Handle data ingestion, transformation, and consolidation to create a unified and reliable data foundation for analysis and reporting. Develop data transformation routines to clean, normalize, and aggregate data. Apply data processing techniques to handle complex data structures, handle missing or inconsistent data, and More ❯

Snr Data Analyst, Power BI | Cross-Asset Trading – Clearing & Market Making | London, Hybrid | £110k + Bonus, Benefits A leading global financial services firm is seeking a talented Data Analyst to join their enterprise Data team, working closely with the Clearing or Market Making business stream. This role offers the chance to be at the forefront of a … data-driven transformation within a dynamic and rapidly growing organisation. The business provides market access, trading, and infrastructure services across major global markets, with a strong presence in commodities, energy, and financial products. The technology function is central to its operations, delivering innovative data solutions and infrastructure across global teams. This opportunity offers great career progression development working … in a high-growth, fast-paced environment with global reach. You will be part of a be part of a forward-thinking technology and data team to work in a collaborative culture that values innovation. 💸The salary on offer for this Data Analyst Role, is up to £110,000 + Bonus and Benefits package. 📍The company works a More ❯

Job Title: QA Tester – ETL Testing (Informatica) & Azure Data Engineering Location: Westminster, London Type: Full-time Department: Quality Assurance/Data Engineering Reporting to: Data Engineering Lead Job Summary: We are looking for a skilled and detail-oriented QA Tester with hands-on experience in ETL testing (Informatica or Microsoft Fabric) and a strong understanding of Azure … Data Engineering tools such as Azure SQL Database, Azure Blob Storage, and Azure Data Factory. The ideal candidate will possess good hands-on knowledge of writing and executing SQL queries to validate data across systems. Experience in Microsoft Fabric is considered a strong bonus. Key Responsibilities: Perform ETL/data pipeline testing using Informatica PowerCenter or … IICS. Validate data ingestion, transformation, and loading processes across Azure services. Execute source-to-target data validation, data profiling, and data reconciliation. Write and execute complex SQL queries to validate data transformations, aggregations, and business rules in Azure SQL Database and other relational platforms. Test data flows and integrations involving: Azure Data More ❯

Direct message the job poster from Selby Jennings Consultant | Data & Analytics | Selby Jennings Our client is redefining electronic trading by developing high-performance, AI-powered solutions that enable smarter, faster decision-making in dynamic markets. Our client is looking for a Senior Data Engineer to be a part of a forward-thinking team in London, working at the … intersection of AI and trading technology. You'll collaborate with leading researchers, data scientists, and engineers to build scalable data infrastructure that supports advanced analytics and machine learning. This is a greenfield opportunity with full ownership from design to deployment, allowing you to shape systems that directly impact trading and AI capabilities. In this role, you'll architect … and maintain robust data pipelines for ingesting, transforming, and storing large volumes of structured and unstructured data, including time-series datasets. You'll automate ETL processes, implement validation frameworks, and ensure high data quality and system reliability. You'll also support AI model development by optimising data retrieval, integrating scalable APIs, and working with vector databases More ❯

As the Lead Enterprise Architect for Data & AI, you will be responsible for defining and driving an enterprise-wide data and information architecture strategy. Your role will ensure data platforms, governance frameworks, and information assets are fully aligned with the needs of business domains such as Commercial, Finance, Supply Chain, Product, and Regulatory. You will lead the … design of a scalable, trusted, and connected data architecture that enables AI/ML, advanced analytics, digital transformation, and regulatory compliance. This is a critical leadership role in advancing the organization’s data-driven enterprise vision. Core Responsibilities of the Enterprise Architect Role: Bridge alignment between business and IT across a federated technology environment. Build strong stakeholder relationships … grow its organizational influence. Continuously evolve the enterprise architecture practice and service portfolio. Mentor architects, product managers, and business leaders to embed architectural thinking. Principal Accountabilities: Define the enterprise data architecture vision, target state, and guiding principles, aligned with business priorities and regulatory frameworks. Lead architecture for enterprise data platforms such as Azure Synapse, Databricks, Power BI, and More ❯

My client, a leading multi-strategy hedge fund, is seeking a mid-level Data Engineer to join their dynamic and rapidly growing team based in London. This is an exciting opportunity to join a high-performing investment firm that places technology and innovation at the core of its operations. You’ll play a key role in empowering the front … office investment team through the delivery of data solutions - enabling informed decision-making across the board. The successful individual will take ownership of MI & BI reporting and data analysis, partnering closely with front-office teams. Your work will support business-critical initiatives through the design, maintenance, and enhancement of ETL pipelines, while also contributing to the development of … automation solutions to streamline key workflows. You’ll also be responsible for: Migrating and enhancing business reports using Power BI Managing data ingestion into the firm’s data lake and data warehouse Troubleshooting data issues, performing log analysis, and owning resolution processes Supporting both manual and automated testing for new data solutions Engaging directly More ❯

to join their team. They are one of the leaders within the Insurance space. They are currently going through a period of growth and are looking for an experienced Data Engineer to join their team. They are backed by a huge Multi National equity firm who are looking to bolster my client's financial position. They are expected to … implement and utilise the most in-demand and cutting-edge technology on the market right now. Your role will include: Collaborate closely with stakeholders to understand and interpret their data and analytics needs. Design and deploy end-to-end data solutions using Microsoft Fabric, encompassing data ingestion, transformation, and visualisation workflows. Construct and refine data models, pipelines, and ETL frameworks within the Fabric ecosystem. Leverage Fabric’s suite of tools to build dynamic reports, dashboards, and analytical applications. Maintain high standards of data integrity, consistency, and system performance across the data environment. Advise on data governance, security protocols, and compliance strategies aligned with industry best practices. Diagnose and resolve technical challenges More ❯

About the Role The Data Engineer plays a key role in shaping and delivering Global Switch’s modern data platform, enabling insights and analytics across a global data centre organisation. Reporting to the Head of Data, this role is central to the development and optimisation of scalable data pipelines and solutions within Microsoft Fabric, supporting … the company’s transformation into a data-driven enterprise. This is an exciting opportunity for a technically skilled and business-savvy professional who thrives on building data products that empower decision-making. You’ll work closely with cross-functional teams to understand stakeholder needs and deliver high-quality, cloud-native data solutions that drive value across the … organisation. Key Responsibilities Design, develop, and maintain scalable data pipelines within the Modern Data Platform. Collaborate with business and technical teams to gather requirements and translate them into effective data products. Implement robust data models and ensure data integrity, performance, and reliability. Develop and optimise batch and real-time ETL processes from diverse data More ❯

Data Engineer (Snowflake) - Nationwide Charity Location: London-based (Hybrid - 1 day per week in the office) Contract: FTC - 23 months £56,500 Overview: We are currently recruiting on behalf of a well-established nationwide charity for a Snowflake Data Engineer . This is an exciting opportunity to play a key role in designing and developing data products … and services that improve how data is accessed and used across the organisation. You'll help ensure data is accurate, timely, and fit for purpose, supporting a wide range of departments and strategic goals. Role Summary: As a Data Engineer, you will work to connect systems, streamline business processes, and optimise data warehouse platforms. You'll … collaborate with stakeholders to deliver bespoke BI solutions, ranging from custom queries to dashboards and reports. Your work will transform raw data into actionable insights that drive the charity's mission forward. In addition to building and maintaining data integration pipelines, you'll help ensure the ongoing performance and reliability of the charity's core data platforms More ❯

A global, privately-owned media group with brands spanning across publishing, audio, and out-of-home advertising, is seeking an Analytics Engineer to join their growing data team. As the business matures its data capabilities, they are evolving toward a clearer separation between Data Engineering, Analytics Engineering, and Data Product disciplines. This role will sit firmly … in the Analytics Engineering function, focused on modelling and building the semantic layer that powers consistent, reliable insights across the company’s BI and data science platforms. This role will focus on the “middle layer", designing dimensional models, building reusable dbt pipelines, and ensuring the semantic layer meets the needs of Power BI, Looker, and data science teams. … There’s also the opportunity to stretch into data product work, partnering with stakeholders to understand and translate business requirements into robust data models. Key Responsibilities: Build and maintain well-structured, scalable data models using dbt and Kimball/Medallion architecture. Develop and own the semantic layer, shifting calculations and business logic out of BI tools and More ❯

Our client is redefining electronic trading by developing high-performance, AI-powered solutions that enable smarter, faster decision-making in dynamic markets. Our client is looking for a Senior Data Engineer to be a part of a forward-thinking team in London, working at the intersection of AI and trading technology. You'll collaborate with leading researchers, data scientists, and engineers to build scalable data infrastructure that supports advanced analytics and machine learning. This is a greenfield opportunity with full ownership from design to deployment, allowing you to shape systems that directly impact trading and AI capabilities. In this role, you'll architect and maintain robust data pipelines for ingesting, transforming, and storing large volumes … of structured and unstructured data, including time-series datasets. You'll automate ETL processes, implement validation frameworks, and ensure high data quality and system reliability. You'll also support AI model development by optimising data retrieval, integrating scalable APIs, and working with vector databases such as FAISS and Pinecone. Familiarity with MLOps, LLM workflows, and RAG frameworks More ❯

Responsibilities Develop and maintain infrastructure for our SaaS platform that delivers AI-driven computer vision tools to researchers and scientists. Collaborate with a multidisciplinary team of machine learning engineers, data scientists, software developers, and biologists. Build and support GPU-accelerated environments for training and real-time inference of deep learning models. Deploy and manage ML pipelines using tools like … Docker, Kubernetes, and frameworks such as Kubeflow or Ray. Create and document APIs that enable internal and external users to access data and model outputs. Implement secure authentication and authorization systems for platform users. Maintain and improve our cloud platform’s reliability, security, and compliance (e.g., GDPR, HIPAA readiness). Automate testing, training, and deployment of models through robust … CI/CD pipelines. Monitor and troubleshoot performance issues across data and inference workflows in production. What We’re Looking For 5+ years of experience in DevOps, MLOps, SRE, or Data Engineering roles. Strong proficiency with public cloud platforms (e.g., GCP, AWS, or Azure), with preference for GCP. Expertise in Terraform and infrastructure-as-code practices. Solid experience More ❯

Senior Data Engineer MUST HAVE SNOWFLAKE, AWS, SNOWPRO CORE Salary - £70-80k with 15% bonus Hybrid working - couple of days in the office City of London We are looking for: Good understanding of data engineering principles A good technical grasp of Snowflake and automating it, and transforming complex datasets SnowPro core certification. AWS Skillset Delivery experience Building … solutions in snowflake Insurance experience - advantageous but not necessary MUST HAVE SNOWFLAKE, AWS, SNOWPRO CORE Key Responsibilities: Lead the design and implementation of Snowflake [and Redshift] based data warehousing solutions within an AWS environment Mentoring of team members through code reviews and pair programming Build and support new AWS native cloud data warehouse solutions Develop and optimize ETL … processes using AWS services (e.g. AWS Glue, Lambda) to ensure efficient data ingestion, transformation, storage, and cost optimization. Deliver data to presentation layers to power reporting and analytics Collaborate with stakeholders, product owners, and developers Create and maintain data models that serve as a foundation for reporting and analytics Monitor performance metrics and troubleshoot issues within More ❯

At Insurwave, we are looking for remarkable people who thrive on making an exceptional contribution. We now have an exciting opportunity for a Senior Data Scientist to play a key role in our Data and AI team. If making a difference gets you out of bed in the morning, then this could be the perfect opportunity and the … start of something incredible! What will you be doing? As a Senior Data Scientist, you will be a technical expert in a multidisciplinary team developing advanced AI and analytics solutions for the Insurwave platform. You’ll take full ownership of complex machine learning initiatives, from problem framing and algorithm selection to scalable deployment and monitoring. You will work across … processing, risk modelling, and asset behaviour analytics Conduct advanced experimentation and model development, applying deep learning, LLMs, or probabilistic methods as appropriate Build high-performance, production-grade pipelines for data ingestion, feature engineering, model training, and real-time inference Develop and maintain robust APIs and infrastructure for scalable model deployment, monitoring, and versioning Apply rigorous model evaluation, performance More ❯

development of scalable trading platforms and internal developer tools. Collaborate with quant researchers, traders, and infrastructure teams to deliver performant solutions. Maintain and optimize trading infrastructure (low-latency systems, data pipelines, monitoring, etc.). Manage a team of software engineers, mentor junior developers, and drive best practices across the stack. Ensure high code quality through code reviews, CI/… workflows. Own deployment pipelines across cloud providers (AWS, GCP, Azure) using containerization (Docker, Kubernetes). Design and implement microservices and APIs for internal tools and external integrations. Work with data at scale (market data ingestion, real-time analytics, historical storage). Required Skills & Experience: Core Engineering: 5+ years of experience in software development, with exposure to both … and storage configurations. Familiarity with monitoring/logging tools (e.g., Prometheus, Grafana, ELK stack). Trading Systems & Finance: Solid understanding of trading infrastructure, latency optimization, execution systems, and market data feeds. Experience working in or with quantitative research, HFT, or hedge fund teams is highly desirable. Leadership: Proven experience managing and mentoring a team of 3–10 engineers. Ability More ❯

transforms, and handling seasonality. Proven ability to tune the performance of existing deployed forecasting models. Must have experience with Azure Machine Learning Python SDK v1/v2 to: Manage data, models, and environments Build/debug AML pipelines to stitch together multiple tasks (feature engineering, training, registering models, etc.) and production workflows using Azure ML pipelines Schedule Azure ML … jobs Deploy registered models to create endpoints. Good to have experience with K-Means clustering. Must utilized Azure services such as Azure Data Factory, Azure Databricks, Azure Data Lake, and Azure Key Vault to architect and maintain scalable data solutions. Design, develop, and deploy new Azure Data Factory (ADF) pipelines for data ingestion, transformation … and logging, ensuring robustness and reliability. Proficiently transform and manipulate data using PySpark and Python, leveraging their capabilities to derive actionable insights from complex datasets. Collaborate with cross-functional teams to understand data requirements and translate them into effective technical solutions. Lead the implementation and optimization of CI/CD pipelines using Azure DevOps, ensuring a seamless build More ❯

My client, a prestigious multi-strat hedge fund, is seeking an exceptional Data Engineer/Developer to join their Reporting & Analytics team within their London office. In this role, you will play a key role in designing and developing next-generation reporting and data analytics solutions, with a strong focus on P&L reporting and operational decision-making. … As part of a rapidly growing firm, you will work on revamping existing implementations, optimising data pipelines, and implementing rigorous data standards to support key business functions across the firm. This role will offer continuous exposure to Portfolio Managers, Risk & Portfolio Research teams, Product Control, and Compliance, providing an opportunity to influence critical workflows and enhance data-driven decision-making across the organization. Responsibilities Design & implement robust data pipelines and ETL processes, ensuring high standards in data modeling, documentation, and testing. Develop and optimize SQL queries for data extraction, aggregation, and reporting. Collaborate with key business teams (Risk, Portfolio Research, Compliance) to build efficient and scalable reporting solutions. Support critical production processes, including More ❯

risk management for years to come. As a Quant Developer, you will work at the intersection of quantitative research, machine learning, and software engineering. You'll collaborate with quants, data scientists, and portfolio managers to design and implement scalable systems for data ingestion, model training, and real-time signal deployment. Key Responsibilities Design and develop robust, high … performance systems for AI/ML model development and deployment. Collaborate with quantitative researchers to translate trading strategies into production-ready code. Build and maintain data pipelines for structured and unstructured financial data. Implement backtesting frameworks and simulation environments. Optimise model inference and execution latency for real-time trading. Contribute to architectural decisions and technology stack selection for the … and deployment. Experience with quantitative finance, including time series analysis, alpha modelling, or risk analytics. Familiarity with cloud infrastructure (e.g., AWS, GCP) and containerisation (Docker, Kubernetes). Proficiency with data engineering tools Experience working in fast-paced, collaborative environments with agile methodologies. Nice to Have Prior experience in a hedge fund, prop trading firm, or investment bank. Exposure to More ❯

Data Engineer - BigQuery Salary: £75,000-£80,000 Role type: Permanent Location: London/Hybrid Our client is seeking a skilled Data Engineer to take the lead in developing and optimising their data systems. This role will focus on SQL database application development, utilising Google BigQuery as the new platform for data management, and supporting the … development of data analytics and visualisation tools such as PowerBI. The Data Engineer will be responsible for ensuring seamless data ingestion, transformation, and reporting, as well as working with key stakeholders to provide timely and actionable insights. Main Responsibilities Engineering : Developing and maintaining business-critical data ingestion, analysis, and management systems. Utilizing API and … ETL methods to integrate with internal systems. Building tools to minimize errors and improve customer experience. Automating data development and governance to improve data health. Designing procedures for system troubleshooting, capacity management, and scaling infrastructure. Analytics : Developing scripts and data outputs to automate reporting and visualizations. Ensuring timely and easily translatable data delivery for business stakeholders. More ❯

Job Title: Test Data Engineer Location: London, UK (3 days in office) SC Cleared: Required Job Type: Full-Time Experience: 5+ years Job Summary: We are seeking a highly skilled and detail-oriented Test Data Engineer to join our growing team. As a Test Data Engineer, you will be responsible for designing, implementing, and executing test cases … applications in the banking domain. You will work closely with the development team to identify and resolve defects, ensure product quality, and deliver reliable and efficient software solutions. Key Data-Testing Responsibilities: Test Internal data ingestion and reference data ingestion Data Ingress – Source to Landing Zone Landing Zone to Bronze Bronze to Silver – Data … tests Output Dataset labelling algorithm tests Vintaging labelling algorithm tests Key Core-Testing Responsibilities: Plan, Schedule, Coordinate, prep, run and report manual tests when required, defect lifecycle, prep test data, mainly for System Test, SIT and migrations testing. Analyse and prioritise test cases to ensure effective test coverage, identify defects, and track resolution to closure; participate in Defect Triage More ❯

Join us as we embark on a ground breaking journey at the forefront of technology innovation. As our Data Engineering Manager , you'll play a pivotal role in driving our technological evolution and shaping the future of our industry. Our client is an API first Technology Investment Platform who are making incredible waves in the industry. The role is … paying up to £95,000 + benefits We are looking for: Deep understanding of Financial Services (FS) Deep expertise in the full data engineering lifecycle—from data ingestion through to end-user consumption Practical experience with modern data tools and platforms, including Redshift, Airflow, Python, DBT, MongoDB, AWS, Looker, and Docker Strong grasp of best practices … in data modelling, transformation, and orchestration Proven ability to build and support both internal analytics solutions and external-facing data products Solid understanding of Agile methodologies (Scrum, Kanban) and modern software development workflows Skilled in using CI/CD pipelines, infrastructure-as-code, and scalable cloud-based data infrastructure Knowledgeable in data governance, privacy regulations, etc More ❯

waves in the industry. The role is paying up to £95,000 + benefits We are looking for: Deep understanding of Financial Services (FS) Deep expertise in the full data engineering lifecycle—from data ingestion through to end-user consumption Practical experience with modern data tools and platforms, including Redshift, Airflow, Python, DBT, MongoDB, AWS, Looker … and Docker Strong grasp of best practices in data modelling, transformation, and orchestration Proven ability to build and support both internal analytics solutions and external-facing data products Solid understanding of Agile methodologies (Scrum, Kanban) and modern software development workflows Skilled in using CI/CD pipelines, infrastructure-as-code, and scalable cloud-based data infrastructure Knowledgeable … in data governance, privacy regulations, etc Experienced in leveraging metrics and observability tools to ensure platform reliability and team performance In this role, your focus on technology innovation and excellence will be paramount to our success. Are you ready to join us on this exciting journey and make your mark on the future of our industry More ❯

col-wide Job Description: Salary - £70-80k with 15% bonus Hybrid working – couple of days in the office City of London We are looking for: Good understanding of data engineering principles A good technical grasp of Snowflake and automating it, and transforming complex datasets AWS Skillset Delivery experience Building solutions in snowflake Insurance experience – advantageous but not necessary … Key Responsibilities: Lead the design and implementation of Snowflake [and Redshift] based data warehousing solutions within an AWS environment Mentoring of team members through code reviews and pair programming Build and support new AWS native cloud data warehouse solutions Develop and optimize ETL processes using AWS services (e.g. AWS Glue, Lambda) to ensure efficient data ingestion, transformation, storage, and cost optimization. Deliver data to presentation layers to power reporting and analytics Collaborate with stakeholders, product owners, and developers Create and maintain data models that serve as a foundation for reporting and analytics Monitor performance metrics and troubleshoot issues within the data architecture, ensuring optimal health and efficiency. Skills and Experience: Bachelor’s More ❯

Job Title: KDB Developer Location: London - Hybrid Type: Permanent Position We are actively recruiting multiple KDB Developers at various levels for a fast-paced, data-driven organisation at the forefront of real-time analytics and high-performance computing. This organisation are a consultancy working with some of the leading names in the financial services sector. The Role: As a … KDB Developer, you will be responsible for designing, developing, and maintaining high-performance applications and data analytics solutions using kdb+/q. You’ll work closely with quants, traders, and data scientists to deliver scalable systems and actionable insights from large volumes of time-series data. Key Responsibilities: Design, implement, and optimise kdb+/q-based applications and … data pipelines Work on real-time data ingestion, transformation, and analysis Collaborate with stakeholders to gather requirements and translate them into technical solutions Maintain and enhance existing codebases, ensuring high availability and performance Contribute to architectural decisions and best practices for kdb+ systems Troubleshoot and resolve production issues quickly and effectively Required Skills & Experience: Strong hands-on More ❯

Test Data Engineer, london (city of london) Client: Mastek Location: london (city of london), United Kingdom Job Category: Other - EU work permit required: Yes Job Views: 7 Posted: 26.06.2025 Expiry Date: 10.08.2025 Job Description: Job Title: Test Data Engineer Location: London, UK (3 days in office) SC Cleared: Required Job Type: Full-Time Experience: 5+ years Job Summary … We are seeking a highly skilled and detail-oriented Test Data Engineer to join our growing team. As a Test Data Engineer, you will be responsible for designing, implementing, and executing test cases and scripts for software applications in the banking domain. You will work closely with the development team to identify and resolve defects, ensure product quality … and deliver reliable and efficient software solutions. Key Data-Testing Responsibilities: Test Internal data ingestion and reference data ingestion Data Ingress – Source to Landing Zone Landing Zone to Bronze Silver to Gold – Curation Registration in Unity Catalogue/Purview Macrobond PCX use case tests Scripts PCX use case tests Output Dataset labelling algorithm tests More ❯