26 to 50 of 459 Data Pipeline Jobs in London

London, United Kingdom Posted on 16/04/2025 We are looking for a Data Architect to design, develop, and manage data solutions that align with business needs and industry standards. The ideal candidate will have expertise in Java, SQL, Python, and Spark (PySpark & SparkSQL … a plus. The role requires strong communication skills to collaborate effectively with both technical and non-technical stakeholders. You will be responsible for data governance, data modelling, metadata management, and ensuring compliance with data standards. Key Responsibilities: 1. Data Architecture & Engineering Design … and implement scalable data architectures that align with business objectives. Work with Java, SQL, Python, PySpark, and SparkSQL to build robust data pipelines. Develop and maintain data models tailored to organizational needs. Reverse-engineer data models from existing live systems. Utilize Microsoft More ❯

automation. This team will be required to build high quality data models and products that monitor and reports on data pipeline health and data quality. Lead project work efforts internally and externally setting project deliverables, review design documents, perform code reviews and help … junior members of the team. Collaborate with engineering teams to improve, maintain, performance tune, and respond to incidents on our big data pipeline infrastructure. Own building out key components for observability and intelligent monitoring of data pipelines and infrastructure to achieve early and automated anomaly … SQL and a proven ability to learn new programming languages Experience with workflow orchestration tools such as Airflow Deep understanding of end-to-end pipeline design and implementation. Attention to detail and quality with excellent problem solving and interpersonal skills Preferred Qualifications Advanced degrees are a plus. Strong data More ❯

DATA ARCHITECT Location: London, UK Reporting To: SVP, Technology Contract Term: Full time, permanent About LIV Golf The LIV Golf League is the world's only global golf league featuring 13 teams, a 14-tournament schedule, and many of the world's best golfers. Launched in 2022, the … level and create new value within the golfing ecosystem while enhancing the game's societal impact. Role Objective We are seeking a visionary Data Architect to lead the design, implementation, and optimization of LIV Golf's enterprise data ecosystem. This role will focus on building scalable … data infrastructure that supports real-time analytics, player performance tracking, fan engagement, and business intelligence initiatives. The ideal candidate will have a passion for sports, a deep understanding of data architecture, and a knack for solving complex data challenges in a dynamic environment. Responsibilities More ❯

Role : Data Engineer Work Mode : Hybrid Contract Role Location : London OR Edinburgh Experience : 10 -12 Years Job Description: Design, develop, and maintain ETL processes to move data from source systems to Snowflake, Teradata and SQL Server environments. Build scalable data pipelines for processing and … storing large datasets. Collaborate with analysts, and business stakeholders to define and implement data strategies. Optimize database performance, including query optimization, indexing, and partitioning across all enterprise data platforms. Work with cross-functional teams to gather requirements and develop effective data solutions. Ensure data … and workflows to meet business requirements. Create and maintain documentation for data systems, pipelines, and processes. Monitor and troubleshoot data pipeline performance issues and resolve them promptly. Assist in the migration and integration of data from legacy systems into Snowflake, and other strategic More ❯

Data Engineering Lead (Snowflake & AWS Environment) Hybrid working: 3 days in TW6, Middlesex offices & 2 days home/remote Salary: Negotiable to £70,000 DOE plus 40 % bonus potential Job Ref: J12869 Please note we can only accept applications from those with current UK working rights for this … role, this client cannot offer visa sponsorship. An exciting opportunity has arisen within a FTSE 100 company for a Data Engineering Lead to play a pivotal role in operating and delivering the organisation’s data products. This position holds significant responsibility within the data leadership team, ensuring the data solutions and business processes are fully aligned and contribute to the vision and strategic direction of the organisation. This is an exciting to time to join the organisation as they are in the early stages of a major programme of work More ❯

Data Engineering Lead (Snowflake & AWS Environment) Hybrid working: 3 days in TW6, Middlesex offices & 2 days home/remote Salary: Negotiable to £70,000 DOE plus 40 % bonus potential Job Ref: J12869 Please note we can only accept applications from those with current UK working rights for this … role, this client cannot offer visa sponsorship. An exciting opportunity has arisen within a FTSE 100 company for a Data Engineering Lead to play a pivotal role in operating and delivering the organisation’s data products. This position holds significant responsibility within the data leadership team, ensuring the data solutions and business processes are fully aligned and contribute to the vision and strategic direction of the organisation. This is an exciting to time to join the organisation as they are in the early stages of a major programme of work More ❯

out our internal engineering capability, spearheading our tech transformation, building best in class products and tackling exciting and complex challenges along the way! Data is at the core of what we do at QiH, but our people are at the heart of our success! At QiH, we have … flow across the organisation. Build efficient, fault-tolerant, and highly available data ingestion processes. Monitor and improve data pipeline performance and scalability for low-latency and high-throughput. Work closely with cross-functional teams, including software engineers, data scientists, and business … or a similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team. Comfortable in a fast-paced environment. Have excellent More ❯

We are hiring 2x Data Engineers (Bioinformatics) to join our Data Engineering Team! As a Data Engineer (Bioinformatics) at Our Future Health, you will have a solid understanding and experience of bioinformatics in particular, tools and methods associated with genomic data. You can design … products. You're a strong communicator where you're comfortable working with a multidisciplinary team of Scientists, Software Engineers, Product Managers and other Data Engineers. What You'll Be Doing: Support the build of data pipelines from data providers to our primary data … engineering across industry, research and Government and facilitating the adoption of standards. Providing technical input into the upstream parts of the data pipeline, including the specification and transfer of data from data providers. Routine ad-hoc data curation activities requiring hands More ❯

Data Engineering Lead (Snowflake & AWS Environment) Hybrid working: 3 days in TW6, Middlesex offices & 2 days home/remote Salary: Negotiable to £70,000 DOE plus 40 % bonus potential Job Ref: J12869 Please note we can only accept applications from those with current UK working rights for this … role, this client cannot offer visa sponsorship. An exciting opportunity has arisen within a FTSE 100 company for a Data Engineering Lead to play a pivotal role in operating and delivering the organisation's data products. This position holds significant responsibility within the data leadership team, ensuring the data solutions and business processes are fully aligned and contribute to the vision and strategic direction of the organisation. This is an exciting to time to join the organisation as they are in the early stages of a major programme of work More ❯

out our internal engineering capability, spearheading our tech transformation, building best in class products and tackling exciting and complex challenges along the way! Data is at the core of what we do at QiH, but our people are at the heart of our success! At QiH, we have … time querying and analytics use cases. Build efficient, fault-tolerant, and highly available data ingestion processes. Monitor and improve data pipeline performance and scalability for low-latency and high-throughput. Ensure data integrity, quality, and security across the entire pipeline. Implement monitoring, logging … or a similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent More ❯

team and be a part of our mission to create a healthier future for all. We have an exciting opportunity for a Senior Data Scientist with a background in machine learning, statistics, data science to join our Pharmacology team in either London UK or West Point … PA. In this role, you will be working collaboratively with our local discovery teams and other AIML & data science functionsto advance therapeutic programs in Immunology, Neuroscience and Infectious Diseases/Vaccines, and to build computational pipelines that make better use of the data we generate. If … biotech setting. An understanding of pharmacology experiments for the purposes of drug discovery, and how to design statistically sound experiments that will help drive pipeline decisions. A background in statistics and machine learning. Experience with databases, database queries, scripting, programming and/or the use of statistics or analysis More ❯

Principal Data Engineer - Core Systems Team Hybrid in London/Remote within UK The company Fast, flexible finance empowers small businesses to manage their cash flow better and seize opportunities - making their business and the economy stronger as a whole. At iwoca, we do just that. We help … We'll get there by continuing to make our finance ever more relevant and accessible to more businesses by combining cutting-edge technology, data science, and a 5-star customer service. The team You'll join the Core Systems team, who are responsible for driving innovation across the … business by optimising development, building data systems, and continuously improving iwoca products. We follow Agile-inspired processes, using continuous integration and delivery, so that features go live in days or weeks, not months or years. The role As the Principal Data Engineer, you'll be responsible More ❯

Data Engineer Junior to Mid-Level London - 4 Days on-site £40,000 - £65,000 DOE + Equity + Unlimited Annual Leave This is an excellent opportunity for a Junior Data Engineer to join a rapidly growing start-up offering great progression and the chance to … short-term staffing needs, this innovative solution optimises workforce efficiency. In this varied role you will be responsible for building and maintaining scalable data pipelines for data integration into customer-facing mobile and web apps, as well as internal dashboards. Responsibilities include designing and implementing data architecture to optimise data storage, retrieval, and processing, alongside developing ETL processes to ingest, transform, and load data from various sources, particularly APIs. The ideal candidate will possess strong foundations in data architecture with a degree in a relative subject or industry More ❯

AWS Data Engineer Salary: Negotiable to £80,000 Dependent on Experience London: Hybrid working 3 days per week in the office 2 days home-based Job Ref: J12931 A leader in consumer behaviour analytics, seeks a driven AWS Data Engineer to guide data infrastructure … architecture, working alongside a small talented team of engineers, analysts, and data scientists. In this role, you'll enhance the data platform, develop advanced data pipelines, and integrate cutting-edge technologies like DataOps and Generative AI, including Large Language Models (LLMs). You'll … skills for data transformation, cleaning, and loading. ·Strong coding experience with Python and Pandas. ·Experience with any flavour of data pipeline and workflow management tools: Dagster, Celery, Airflow, etc. ·Build processes supporting data transformation, data structures, metadata, dependency and workload management. More ❯

Job Title: Senior Generative AI Data Engineer ? Location: London, United Kingdom ? Salary: Negotiable ? Recruitment Agency: 167 Solutions ? Job Description: 167 Solutions is seeking an experienced Senior Generative AI Data Engineer to join our client's innovative team in London. This pivotal role offers … the opportunity to lead the development and deployment of generative AI models, driving transformative solutions across various business domains. ? Key Responsibilities: Data Pipeline Architecture: Design, develop, and maintain robust, scalable data pipelines to support generative AI models, ensuring efficient data flow and processing. … or similar. ? Data Processing: Expertise in big data technologies such as Apache Spark, Hadoop, and experience with data pipeline tools like Apache Airflow. ? Cloud Platforms: Strong experience with cloud services, particularly AWS, Azure, or Google Cloud Platform, including services like S3, EC2, Lambda More ❯

ASCS. We are establishing a new centralized Business Intelligence team to build self-service analytical products for ASCS that provide relevant insights and data deep dives across the business. By leveraging advanced analytics and AI/ML, we will transform catalog data into predictive insights, helping … prevent customer issues before they arise. Real-time intelligence will support proactive decision-making, enabling faster, data-driven decisions across the organization and driving long-term growth and an enhanced customer experience. We are looking for a creative and goal-oriented Business Intelligence Engineer (BIE) to join our … Spark, Hive, Hadoop, PyTorch, PySpark) PREFERRED QUALIFICATIONS - Master's degree, or Advanced technical degree - Knowledge of data modeling and data pipeline design - Experience with statistical analysis, co-relation analysis - Experience in the data/BI space - Experience in designing and implementing custom reporting More ❯

Exciting Opportunity in Health Tech! Position: Data Engineer (Python/Databricks) Location: Remote Salary: up to £80,000 + Bens Reporting To: Vice President of Software Development Are you passionate about health tech and innovation? Do you want to be at the forefront of transforming clinical research with … cutting-edge technology? If so, we have an exciting new role for you! Join our dynamic and forward-thinking team as a Data Engineer and help us build secure, scalable microservices that operationalize clinical research applications. This is your chance to make a meaningful impact on healthcare while … working with some of the most advanced technologies in data engineering. About Us We are a pioneering health tech company dedicated to revolutionizing clinical research through innovative data solutions. Our cross-functional team, including Frontend Developers, QA Engineers, and DevOps Engineers, collaborates to create high-performance More ❯

digital business model and diversify into other digital media products across multiple Clarion portfolios. To support this growth, Clarion requires an outstanding Azure Data Engineer to join the Data and Analytics team. You will be responsible for designing, developing, and implementing data migration solutions. … Your expertise in Azure services, data integration, and Terraform will be essential as we build innovative solutions for our transformation programmes. Key Responsibilities Data Migration: Migrate data from existing Big Data platforms into Dynamics 365, Data Verse and other Azure … data environments. Migrating from hand-rolled Code-based ETL processes to modern Azure Cloud Computing solutions Data Aggregation: Aggregate from smaller data sources and SaaS platforms into our Microsoft data environments Azure Development: Utilize Azure Data Factory, Azure Functions, and More ❯

Senior Data Engineer Hybrid working (3 days per week onsite in London) 6 Month contract initially, good scope for extension Market rates (Umbrella-PAYE) One of our blue chip clients … are looking for a number Senior Data Engineer to join the team on a long term programme of work. Key Responsibilities: * ETL Pipeline Development: Develop, optimize, and maintain ETL pipelines to efficiently extract, transform, and load data from various sources, ensuring high data quality. * Monitor and troubleshoot production data pipelines, ensuring their performance and reliability. * Mentor junior engineers and lead technical discussions to drive best practices and innovation within the team. * Stay up to date with the latest trends and technologies in data engineering and recommend solutions More ❯

Date: March 31, 2025 (30+ days left to apply) Job requisition id R26624 At FactSet, we're working to be the best financial data provider. We need highly motivated, talented individuals empowered to find answers through creative technology to get there. As a Software Engineer in Data Solutions Engineering, you will be part of our Digital Transformation, a mission to automate our data acquisition, quality assurance, content creation, and analytics in a scalable cloud environment. With the guidance of financial experts, you will leverage these large data sets to improve the … mapping, and company valuation. There is huge potential for machine learning, analytics, and NLP. Your responsibilities: Build and scale an automatic data pipeline Ingest and analyze various data sources to drive innovation in content creation. Automate the acquisition, relevance scoring, and storage of incoming sources. More ❯

join us at FPSM and be part of something bigger… What we are looking for: We are seeking a highly motivated and experienced Data Product Manager to own the strategy, roadmap, and execution of our Customer Data Platform (CDP) and Single Customer View (SCV). This … is a critical role within our rapidly growing data organization, and you will be instrumental in enabling data-driven decision-making across the business. You'll work closely with engineering, marketing, product, and other internal teams to ensure our CDP is a powerful tool for understanding … with overall business objectives and stakeholder needs. Requirements Gathering: Proactively gather, analyze, and prioritize requirements from internal stakeholders (marketing, product, analytics, etc.) for data integration, access, and utilization. Data Source Integration: Oversee the integration of various data sources into the CDP, ensuring data More ❯

join us at FPSM and be part of something bigger… What we are looking for: We are seeking a highly motivated and experienced Data Product Manager to own the strategy, roadmap, and execution of our Customer Data Platform (CDP) and Single Customer View (SCV). This … is a critical role within our rapidly growing data organization, and you will be instrumental in enabling data-driven decision-making across the business. You'll work closely with engineering, marketing, product, and other internal teams to ensure our CDP is a powerful tool for understanding … with overall business objectives and stakeholder needs. Requirements Gathering: Proactively gather, analyze, and prioritize requirements from internal stakeholders (marketing, product, analytics, etc.) for data integration, access, and utilization. Data Source Integration: Oversee the integration of various data sources into the CDP, ensuring data More ❯

as we possibly can. If this sounds good to you, then we're thrilled you're considering joining our team! Role Overview The Data Engineer will play a critical role in implementing and maintaining Lumon's technical data infrastructure. Reporting to the Head of Data … who thrives on designing scalable data solutions and enjoys solving complex data challenges. Key Responsibilities Data Infrastructure and Pipeline Development Design, develop, and maintain ETL/ELT pipelines using Azure Synapse Analytics, Data Factory and other Azure related cloud warehouse tools. … years in data engineering or similar technical roles, preferably in financial services or a related field. Hands-on experience with Azure Synapse, Pipeline creations, Azure Data Factory, and Azure Data Lake Storage. Strong proficiency in SQL, including database management, query optimisation, and troubleshooting. More ❯

as we possibly can. If this sounds good to you, then we're thrilled you're considering joining our team! Role Overview The Data Engineer will play a critical role in implementing and maintaining Lumon's technical data infrastructure. Reporting to the Head of Data … who thrives on designing scalable data solutions and enjoys solving complex data challenges. Key Responsibilities Data Infrastructure and Pipeline Development Design, develop, and maintain ETL/ELT pipelines using Azure Synapse Analytics, Data Factory and other Azure related cloud warehouse tools. … years in data engineering or similar technical roles, preferably in financial services or a related field. Hands-on experience with Azure Synapse, Pipeline creations, Azure Data Factory, and Azure Data Lake Storage. Strong proficiency in SQL, including database management, query optimisation, and troubleshooting. More ❯

Job Title: Senior Data Architect Location: Hybrid (London, UK) Salary: £75,000–£135,000 per annum, plus 20% Bonus and Benefits About the Role: Our client is a leading professional services firm with a strong UK presence and global reach, delivering expert audit, tax, advisory, and risk solutions. … As part of their ongoing investment in digital transformation, they are scaling their data capabilities to better serve clients in regulated and data-intensive sectors. This newly created Senior Data Architect role will focus on designing and implementing data architecture solutions for … architecture within the insurance and asset management industries, including regulatory and compliance requirements. Proficiency in SQL , Python , and modern data pipeline design (ETL/ELT). Experience with both batch and streaming data processing architectures. Strong communication and stakeholder engagement skills, particularly with More ❯