1 to 25 of 78 Data Ingestion Jobs in London

Senior Data Engineer We are seeking a highly skilled Data Engineer to focus on maintaining data streams and ETL pipelines within a cloud-based environment. The ideal candidate will have experience in building, monitoring, and optimizing data pipelines, ensuring data consistency, and proactively collaborating with upstream and downstream teams to enable seamless data flow … across the organization. In this role, you will not only troubleshoot and resolve pipeline issues but also contribute to enhancing data architecture, implementing best practices in data governance and security, and ensuring the scalability and performance of data solutions. You will play a critical role in understanding the business context of data, supporting analytics and decision … making by collaborating with data scientists, analysts, and other key stakeholders This position requires client presence between 25%-50% of the time per month at the client's office, which is located in London. Key Responsibilities: Data Pipeline Development & Maintenance Build, maintain, and optimize scalable ETL/ELT pipelines using tools such as Dagster, or similar. Ensure high More ❯

health by developing the most advanced biosensing technology science has to offer, capable of accessing bodily information in a painless, real-time and affordable way. Description Join Sava's Data Engineering team at a pivotal stage of growth, contributing to the development and scaling of both, internal and customer-facing data infrastructures. As a Data Engineer II … you will help implement reliable data pipelines, storage solutions, and reporting systems that power analytical and operational workflows across the company. This is a unique opportunity to work in a multidisciplinary environment, collaborating with a top-tier team in engineering, science, and product to shape the foundation of Sava's data architecture. Responsibilities Build and maintain scalable, reliable … data pipelines, ensuring efficient and accurate data ingestion, transformation, and delivery. Design, implement, and optimize data storage solutions using modern databases and cloud-native tools. Develop and maintain reporting systems and dashboards to support data-driven decision-making. Collaborate with backend and infrastructure teams to integrate data services with applications and customer-facing tools. More ❯

Who We Are BCG Expand is a specialist firm providing unique data-driven business intelligence as a trusted advisor to senior executives across the world's leading financial services firms to help them to operate more effectively. Expand's activities are focused on data benchmarking in technology, operations, FinTech and market data, with expertise around strategic cost … maintain our professional, supportive and entrepreneurial working culture that we all enjoy. What You'll Do We're looking for a highly skilled Principal Engineer with a passion for data, cloud technologies, and building robust, scalable solutions - who also brings strategic product thinking and cross-functional leadership . In this role, you'll not only lead the design and … implementation of the data infrastructure that powers advanced analytics and data science across our platforms, but also contribute to the strategic direction, roadmap, and execution of data products in collaboration with senior stakeholders. Architect & Build Scalable Data Solutions Collaborate closely with senior product stakeholders to understand data needs and architect end-to-end ingestion More ❯

Data Engineer - STR - London, UK Job Description OVERVIEW CoStar Group (NASDAQ: CSGP) is a leading global provider of commercial and residential real estate information, analytics, and online marketplaces. Included in the S&P 500 Index and the NASDAQ 100, CoStar Group is on a mission to digitize the world's real estate, empowering all people to discover properties, insights … our employees, and investors. By equipping the brightest minds with the best resources available, we provide an invaluable edge in real estate. STR is the global leader in hospitality data benchmarking, analytics, and marketplace insights for the global hospitality industry. STR delivers data that is confidential, accurate, and actionable to empower our clients to strategize and compete in … their markets. Learn more about STR . ROLE DESCRIPTION The role of Data Engineer will be a key role within the STR Data Operations Team, reporting to the Director, Technical Data Operations. The Data Operations team is responsible for data acquisition, data loading, data quality, data processing, data delivery, and data More ❯

technology enables customers to easily personalise their creations to keep for themselves or share as the most thoughtful of gifts. About The Role: We are looking for a Senior Data Platform Engineerto join our London-based data team and help us build a platform that accelerates a decentralised-by-design data adoption model within the business. We … care about trustable, usable data and we want you to bring your technical expertise to make that happen. You're a software engineer at heart, you thrive in the data domain and care deeply about customer outcome-focused engineering excellence. Day to day, you will build observable, scalable and resilient data services that ingest data from … source systems into consuming applications such as our data warehouse to meet the needs of analytics engineers and stakeholders. You will make significant contributions to solution designs, provide technical guidance to other team members and champion iterative approaches to software delivery. You will work within a cross-functional mission-led team, led by a product manager, an engineering manager More ❯

My client is based in the London area and is currently looking to recruit an experienced Fabric Data Engineer to join their team. They are one of the leaders within the Insurance space. They are currently going through a period of growth and are looking for an experienced Data Engineer to join their team. They are backed by … implement and utilise the most in-demand and cutting-edge technology on the market right now. Your role will include: Collaborate closely with stakeholders to understand and interpret their data and analytics needs. Design and deploy end-to-end data solutions using Microsoft Fabric, encompassing data ingestion, transformation, and visualisation workflows. Construct and refine data models, pipelines, and ETL frameworks within the Fabric ecosystem. Leverage Fabric's suite of tools to build dynamic reports, dashboards, and analytical applications. Maintain high standards of data integrity, consistency, and system performance across the data environment. Advise on data governance, security protocols, and compliance strategies aligned with industry best practices. Diagnose and resolve technical challenges More ❯

280+ Employees - We have over 280 passionate employees transforming the industry across the Group, one home at a time The Role We're seeking a motivated and detail orientated Data Engineer with experience in building and managing scalable data pipelines and infrastructure. You'll be responsible for developing robust ETL/ELT processes, optimising and maintaining data warehouse, and ensuring data quality and standards through comprehensive monitoring and governance. Collaborating closely with our Data, Tech and Product teams, you'll help shape and support our data strategy. The ideal candidate should have familiarity in SQL and cloud-based data solutions, be driven towards automated solutions and thrives in transforming complex data infrastructure into streamlined, reliable, and actionable assets. Responsibilities: Own Data Infrastructure : Design, develop, and maintain scalable data pipelines and infrastructure to enable reliable analytics and data-driven decisions. ETL/ELT Development : Build, optimise, and manage ETL processes in BigQuery and associated tools to ensure timely and accurate data ingestion. Data Warehousing : Maintain and More ❯

Senior Data Engineer - London/Hertfordshire - Hybrid Are you a technically strong Azure Data Engineer looking to join a forward-thinking, innovative company? Here's your chance to work with a leading construction and infrastructure company who foster a culture of collaboration, support, and strong communication. You'll be responsible for ensuring seamless integration and optimization of diverse … business data streams - from CRM, finance, procurement to fleet and commercial data - enabling reliable, high-quality data for analytics and business insights. While this role won't be client-facing, strong communication and collaboration skills within their agile team are essential. Requirements: Experience orchestrating data pipelines with Azure Data Factory (ADF) Proficiency managing API data ingestion and orchestration using Azure Databricks Working with Azure Data Lake (ADL) Storage, on-prem SQL, and Python Vast Azure data engineering experience Proficient in ADF, Databricks, ADL, SQL, Python Desirable: Exposure to D365 or similar CRM environments Azure Synapse experience Power BI knowledge Benefits: Salary: £70,000 - £80,000 depending on experience Hybrid working More ❯

Senior Data Engineer - London/Hertfordshire - Hybrid Are you a technically strong Azure Data Engineer looking to join a forward-thinking, innovative company? Here's your chance to work with a leading construction and infrastructure company who foster a culture of collaboration, support, and strong communication. You'll be responsible for ensuring seamless integration and optimization of diverse … business data streams - from CRM, finance, procurement to fleet and commercial data - enabling reliable, high-quality data for analytics and business insights. While this role won't be client-facing, strong communication and collaboration skills within their agile team are essential. Requirements: Experience orchestrating data pipelines with Azure Data Factory (ADF) Proficiency managing API data ingestion and orchestration using Azure Databricks Working with Azure Data Lake (ADL) Storage, on-prem SQL, and Python Vast Azure data engineering experience Proficient in ADF, Databricks, ADL, SQL, Python Desirable: Exposure to D365 or similar CRM environments Azure Synapse experience Power BI knowledge Benefits: Salary: £70,000 - £80,000 depending on experience Hybrid working More ❯

Company Details Required Skills & Knowledge: Solid hands-on experience with Microsoft Technology Stack (2022) Extensive experience of SQL/data warehousing Proven experience of data engineering activities such as release management, environment controls, CICD pipeline orchestration. Experience working with Python and Power BI Proven ability of Microsoft BI software development including database/reporting development, full development cycle … Strong data modelling, problem solving, information analysis, attention to detail, flexibility of approach. The ability to deal directly with business functions and interpret their thoughts into successful data & analytics solutions. The ability to design and develop complex BI/MI solutions in line with agreed architectural principles and toolsets. Capable of working closely with the business and delivering … complex requirements to tight timescales. Strong interest in latest data engineering technologies, best practises, techniques and trends in the data industry Required Toolsets: Kimball data modelling Microsoft SQL Server Analysis Services (Multidimensional/Tabular), Power BI Python, MDX, DAX Responsibilities The Data Manager will be responsible for supporting Head of Data and Analytics and wider More ❯

LBANK, Gett, Kaizen Gaming, and TransferGo. Our products are recognised by industry leaders like Gartner's Magic Quadrant, Forrester Wave and Frost Radar. Our tech stack: Superset and similar data visualisation tools. ETL tools: Airflow, DBT, Airbyte, Flink, etc. Data warehousing and storage solutions: ClickHouse, Trino, S3. AWS Cloud, Kubernetes, Helm. Relevant programming languages for data engineering … tasks: SQL, Python, Java, etc. What you will be doing: Designing and developing scalable and efficient data pipelines, ETL processes, and data integration solutions to support data ingestion, processing, and storage needs. Ensuring data quality and reliability by implementing data validation, data cleansing, and data quality monitoring processes. Optimising database performance by … tuning queries, implementing indexing strategies, and monitoring and analysing system performance metrics. Collaborating with cross-functional teams to gather requirements, understand data needs, and develop data solutions that align with business objectives. Staying up-to-date with emerging technologies and industry trends in data engineering, and identifying opportunities for process improvements and tool enhancements. Establishing and enforcing More ❯

a part of a team that continually strives to do right by our community and users. We’re constantly innovating to raise the bar of engine and game development. DATA ENGINEERING What We Do Our mission is to provide a world-class platform that empowers the business to leverage data that will enhance, monitor, and support our products. … We are responsible for data ingestion systems, processing pipelines, and various data stores all operating in the cloud. We operate at a petabyte scale, and support near real-time use cases as well as more traditional batch approaches. What You'll Do You will be responsible for designing, building, and maintaining our data infrastructure to ensure … the reliability and efficiency of our data and systems used by our Machine Learning team. Your role will include creating and maintaining data pipelines that transform and load data from various products and managing the AWS infrastructure for our machine learning platform. Additionally, you will work with engineers, product managers, and data scientists to design and More ❯

Data Engineer (Enterprise DataWarehouse Developer) Description: As a Data Engineer, you'll design and maintain data scrapers and data pipelines, design & optimize analytics & relational databases, and build analytics models using DBT and bespoke aggregation engines. You'll work closely with business stakeholders, other BI Developers and DataOps as well as System engineers to support both data … bespoke tools written in Python/Java, as well as tools such as Meltano, Airflow, Mulesoft/Snaplogic, Apache NIFI, and Kafka, ensuring a robust, well-modelled, and scalable data analytics infrastructure running on MySQL and Postgres style databases primarily. Requirements: Advanced SQL development and deep understanding of RDBMS concepts and engines Practical knowledge of Data Warehouse infrastructure … architecture and best practices Practical database design, development, administration knowledge (any RDBMS) Strong understanding of data governance, quality, and privacy (e.g. GDPR compliance) Proficiency in ELT/ETL processes Strong experience in data ingestion, transformation & orchestration technology (ETL tools such as Informatica, Datastage, SSIS, etc ) or open source Meltano, Airbyte, and Airflow Proven experience with DBT (data More ❯

solutions through an agile framework. Our Technology team is responsible for: Software Development Infrastructure DevOps and Automation Service Delivery Project Management Brief Role Objective: As a member of the Data Development team, you will be responsible for developing ways to consolidate, analyse, and present our data to support the operations and future growth of our business. In this … role you will be involved in a wide variety of high-profile business facing initiatives from developing data extracts and PowerBI reports to optimising the data warehouse to deliver data more quickly to the business each day. You will work with the Lead Data Engineer and other members of the Data Engineering team to deliver … our new strategic enterprise data platform based on Snowflake and DBT, while also maintaining our legacy data platform. Key Responsibilities: Data warehouse design and implementation working towards the creation of a single source of truth. Development of data ingestion/transformation pipelines using Fivetran, DBT and Gitlab. Creation of management information dashboards. Work with business More ❯

technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by diversity and inclusion, talented peers, comprehensive benefits and career development opportunities. Come make … to propel healthcare forward in a way that better serves everyone. With our hands at work across all aspects of health, we use the most advanced development tools, AI, data science and innovative approaches to make the healthcare system work better for everyone. As a Principal Data Engineer at Optum, you'll be responsible for working with key … developers to ensure a shared understanding of business and technical requirements Design and build data pipelines to process terabytes of data Develop and recommend best practice re dataingestion, processing,cleaningand standardizingof data (typically on Azure) Create Docker images for various applications and deploy Design and build tests Troubleshoot production issues Analyze existing Data solutions and recommend More ❯

an office in Central London and work as a mix from office and home. The role We are looking for a senior Machine Learning Engineer with a spike in data engineering and maintaining real-time data pipelines. You will work with our Product & Engineering team along the end-to-end algorithm lifecycle to advance the Trudenty Trust Network. … A bit more on what you’ll do: Data Engineering Develop and maintain real-time data pipelines for processing large-scale data Ensure data quality and integrity in all stages of the data lifecycle Develop and maintain ETL processes for data ingestion and processing Algorithm Development, Model Training and Optimisation Design, develop, and … algorithms for fraud prevention and user personalization Train and fine-tune machine learning models using relevant datasets to achieve optimal performance Implement strategies for continuous model improvement and optimization Data Mining & Analysis Apply data mining techniques such as clustering, classification, regression, and anomaly detection to discover patterns and trends in large datasets. Analyze and preprocess large datasets to More ❯

or any other characteristic protected by law. Accommodation is available upon request for candidates taking part in the selection process. How does this role contribute to our collective success? Data and analytics are of critical importance for AXIS. Simply put, we want to turn data into information, that can be used to: Enable decisions to be made with … Innovation & Analytics team is a core component of AXIS's strategy, with the potential to transform the way in which we do business and enable us to effectively leverage data and technology to our competitive advantage. The central mandate is to horizon-scan, prototype and implement innovative solutions to business problems. What will you do in this role? The … developing and implementing artificial intelligence solutions to solve complex problems and enhance business operations. The successful candidate will work cross-functionally; operating between the business stakeholders, technology teams, and data scientists, to ensure that final solutions meet requirements. The Lead AI Solutions Engineer is an expert in large language models, Gen AI application implementation and machine learning, and has More ❯

Senior Data Engineer MUST HAVE SNOWFLAKE, AWS, SNOWPRO CORE Salary - £70-80k with 15% bonus Hybrid working - couple of days in the office City of London We are looking for: Good understanding of data engineering principles A good technical grasp of Snowflake and automating it, and transforming complex datasets SnowPro core certification. AWS Skillset Delivery experience Building … solutions in snowflake Insurance experience - advantageous but not necessary MUST HAVE SNOWFLAKE, AWS, SNOWPRO CORE Key Responsibilities: Lead the design and implementation of Snowflake [and Redshift] based data warehousing solutions within an AWS environment Mentoring of team members through code reviews and pair programming Build and support new AWS native cloud data warehouse solutions Develop and optimize ETL … processes using AWS services (e.g. AWS Glue, Lambda) to ensure efficient data ingestion, transformation, storage, and cost optimization. Deliver data to presentation layers to power reporting and analytics Collaborate with stakeholders, product owners, and developers Create and maintain data models that serve as a foundation for reporting and analytics Monitor performance metrics and troubleshoot issues within More ❯

The banks' expertise and services span across Business Services, Corporate banking, Wealth Management, Group Functions, Retail and Investment Banking. On behalf of this organisation, AMS are looking for a Data Engineer for a 12 month contract based in London (hybrid -2 days in the office per week). Purpose of the role: The Data Engineer will help drive … the build of effortless, digital first customer experiences, simplifying the organisation by developing innovative data driven solutions through data pipelines, modelling and ETL design, inspiring to be commercially successful through insight and keep customers and the organisation's data safe and secure. What you'll do: Build advanced automation of data engineering pipelines through removal of … manual stages. Embed new data techniques through role modelling, training and experiment design oversight. Deliver a clear understanding of data platform costs to meet departments cost saving and income targets. Source new data using the most appropriate tooling for the situation. Develop solutions for streaming data ingestion and transformations in line with the client's More ❯

We are looking for a highly talented and driven Data Engineer who takes pride in their work, to expand our Engineering team in London. Successful candidates will join a cross functional team including product managers and designers working closely with the rest of our business to deliver working code that solves real problems for both internal and external customers. … services managed by your team, ensuring that their development aligns with the higher level AlphaSights Engineering strategy, while mentoring more junior Engineers. If you're passionate about solving complex data challenges with code, and enjoy collaborating with talented colleagues in a high-performance environment, this role is a perfect fit for you. What you'll do: Design solutions: Design … develop, deploy and support data infrastructure, pipelines and architectures, contributing to an architectural vision that will scale up to be the world's leading research platform. Ship working code: Write clean, efficient, and maintainable code that powers data pipelines, workflows, and data operations in a production environment. Implement reliable, scalable, and well-tested solutions to automate data More ❯

About the Role: Grade Level (for internal use): 11 About the Role: The Team: The Enterprise Marketing Analytics team empowers data-informed decision-making across S&P Global by delivering actionable insights on marketing performance. We focus on enterprise-level reporting across web, paid media, events, and campaign conversions-driving consistency, engagement, and ROI through advanced analytics. Our team … Analytics, Power BI, Marketo, Adobe Target, Drift, Conversica, ON24, Adobe CDP, and Salesforce. Impact: Reporting to the Director of Enterprise Marketing Analytics, this role leads marketing performance reporting and data visualization across the enterprise. You will define and manage scalable reporting processes, own the marketing data visualization strategy, ensure insights are accurate, accessible, and aligned to strategic priorities. … This role is essential in translating data into action, influencing decision-making, and fueling marketing effectiveness at scale. Responsibilities Enterprise Marketing Reporting & Visualization Own the marketing analytics dashboard strategy using Power BI or other tools ; oversee data ingestion, transformation, visualization, and automation to support self-service access. Lead Adobe Analytics Workspace reporting for engagement and conversion metrics More ❯

Support end-to-end deployment of ML models (batch and real-time) from code validation through to production rollout under guidance from senior team members. Work with Data Science teams to facilitate smooth model handover and ensure deployment readiness aligned with implementation standards. Build and maintain CI/CD pipelines for model deployment, scoring, and operational monitoring. Debug and … fix pipeline issues including data ingestion problems, model scoring failures, and deployment errors. Write comprehensive tests for ML pipelines (unit, integration, validation) and implement data quality checks and operational monitoring. Ensure deployed models meet audit, reconciliation, and governance requirements. Monitor production models for operational health, troubleshoot failures, and track data/variable drift over time. Work … with Platform Engineers within the team to create reusable MLOps templates and support Data Scientists in using them effectively. Support model migrations across data sources, tools, systems, and platforms. Participate in code reviews, knowledge sharing, and pod activities (standups, grooming, delivery check-ins). Learn from senior team members and contribute to continuous improvement of model delivery practices. More ❯

ll build analytics pipelines that help deliver actionable, automated user insights from product adoption data. You'll collaborate directly with product analysts, engineers, and business stakeholders to build robust data integrations, including Salesforce, Gainsight, JIRA and Github. Your work will be pivotal in empowering data-driven decisions across our organization, significantly impacting product strategy and customer engagement. What … to expect? Analytics Pipelines & Data Modeling: Convert existing product analyst queries and reports into efficient and maintainable analytics pipelines primarily using dbt, and optimize data for self-service analytics in Tableau. Owning Analytics Projects: Take ownership of medium-to-large analytics projects, from scoping through implementation. Product Event Data: Transform and maintain scalable product adoption datasets, ensuring … ease of downstream integration and rapid onboarding of new events or features. Data Egestion: Develop and manage data pipelines for exporting curated datasets from Redshift to platforms like Salesforce and Gainsight using reverse ETL tools (e.g., Hightouch). Data Ingestion: Own end-to-end responsibility for ingesting key productivity data from platforms such as GitHub More ❯

Senior Data Engineer - £82,000 - Hybrid & Remote Flexibility No Sponsorship Available Company We are a forward-thinking technology company dedicated to simplifying complex processes through innovative solutions. With a focus on enhancing efficiency and transparency, we strive to deliver outstanding experiences for our customers. Our team operates with a flexible hybrid working model, fostering collaboration and innovation in a … supportive environment. Responsibilities As a Senior Data Engineer, you'll play a pivotal role in advancing our data infrastructure to support our expanding scope and responsibilities. Key responsibilities include: Enhancing the reliability and stability of our data infrastructure. Managing data ingestion pipelines using tools such as Azure Data Factory (ADF) and Python. Ensuring the … quality of raw datasets to empower Data Analysts in creating robust data models. Deploying and managing data tools on Kubernetes (Airflow, Superset, RStudio Connect). Supporting Data Analytics through the management of DBT, DevOps, and deployment rules. You will have the opportunity to work end-to-end, making meaningful contributions within a small, agile team. Experience More ❯

Job Title: Data Engineer (NPPV3 + SC Clearance Required)Location: Hybrid (UK-based)Contract: 3 Months (with potential extension/move to other projects)Rate: £550 per day (Outside IR35)A bout the Role:We are seeking an experienced Data Engineer with NPPV3 + SC clearance to join our dynamic team on a 3-month contract basis. This … role offering the flexibility of working both remotely and onsite. The contract has potential for extension or transition onto other exciting projects. Key Responsibilities:* Design, build, and maintain scalable data pipelines using Azure Data Factory, Databricks, and Azure Synapse.* Collaborate with cross-functional teams to ensure data quality and integration.* Support data ingestion, transformation, and … storage in cloud environments.* Troubleshoot and optimise data workflows for performance and reliability.* Maintain compliance with security protocols and clearance requirements. Essential Skills & Experience:* Must hold NPPV3 + SC clearance (this is a mandatory requirement).* Proven expertise in Azure Data Factory, Databricks, and Azure Synapse Analytics.* Strong experience in building and managing cloud-based data solutions. More ❯