126 to 150 of 460 Data Pipeline Jobs in London

Data Solutions Architect Hybrid - Inside IR35 We're looking for a seasoned Data Solutions Architect with deep expertise in the Lloyd's of London Market and a strong background in Snowflake and ETL/data transformation tools. As a Data Architect, you … ll lead the design, development, and deployment of enterprise-grade data architecture using Snowflake and a modern suite of data tools. You'll work at the intersection of IT and business, collaborating closely with stakeholders to create solutions that unlock the full value of data-driving decision-making and business growth across the organisation. Key Responsibilities Architecture Design: Design and maintain enterprise data architecture models, data flows, and integrations using Snowflake. Data Governance: Manage data collection, storage, transformation, and protection across systems. Database Development: Create More ❯

Principal Data Engineer £120,000-£140,000 ️ Snowflake, SQL, dbt London, hybrid working Finance empowering small business We have joined forces with a leading UK fintech who help small businesses get the funds they need, when they need it. They are looking for a Principal Data Engineer who will be responsible for the data platform. In this role you'll define data strategy, drive the evolution of data infrastructure, and ensure data systems enable impactful decision-making across the business. What you'll be doing You … Snowflake platform to ensure it is a highly efficient and accessible resource for the business. You will lead the development and optimisation of data pipelines using dbt, enabling faster and more reliable data flows. You will collaborate with analysts and data scientists to design More ❯

AWS Data Engineer (Must hold current SC) The Company: At Amber Labs, we are a cutting-edge UK and European technology consultancy that prioritises empowering autonomy, promoting experimentation, and facilitating rapid learning to provide exceptional value to our clients. Our company culture is centred around collaboration, where all … times. Our team consists of highly-skilled cross-functional consultants, analysts, and support staff. Overview: We are looking for a highly skilled AWS Data Engineer to join our team. The ideal candidate will have expertise in designing, building, and managing scalable data pipelines on AWS. You … will work closely with data scientists, analysts, and software engineers to implement efficient data solutions that support business intelligence and analytics needs. Responsibilities: Design, develop, and maintain scalable ETL/ELT pipelines using AWS services such as AWS Glue, Lambda, Step Functions, and Kinesis. Work with More ❯

for our IT & Digital department. This is a hybrid role based in London. As a Lead Machine Learning Engineer on the National Grid Data Science team, you will develop data pipelines, take data science prototype models to production, fix production bugs, monitor operations, and … Machine Learning projects end-to-end. Develop platform tooling (e.g., internal conda library, CLI tool for project setup, and provisioning infrastructure) for the Data Science team. Work with data scientists to understand their data needs and put together data pipelines to ingest … data. Work with data scientists to take data science model prototypes to production. Mentor and train junior team members. Work with internal IT teams (security, Cloud, Global Active Directory, Architecture, Networking, etc.) to advance the team's projects. Enhance code deployment lifecycle. Improve model monitoring frameworks. More ❯

highly skilled Lead AI Solutions Engineer to design, build, and optimize a Retrieval-Augmented Generation (RAG) system that underpins our self-serve analytics data applications. In this pivotal position, you will develop a scalable RAG platform, integrate multiple data sources, and create intuitive data … our Group's RAG Initiative, you will collaborate with a cross-functional team to implement self-service AI-driven solutions - ranging from NLP Data Analysis and Data Discovery to other analytics assistants we will be deploying to the business to streamline data access, insights … process automation. About A&I The Automation and Insights (A&I) department is at the forefront of transforming Kindred into a highly automated, data-driven powerhouse. In this dynamic environment, you'll work alongside experts in data, insights, and cutting-edge AI to build an AI More ❯

This role reports to the Analytics Director, Adobe.pub EMEA. The ideal candidate will possess strong analytical and communication skills and have experience analysing data to answer critical business questions. They will design and implement effective data visualizations and reports, while collaborating closely with cross-functional teams. … to create insightful solutions for business questions. Plan and execute analyses to guide campaign strategies, and produce comprehensive reports for presentations and publications. Data Transformation: Develop and maintain data pipelines for efficient data integration. Conduct data wrangling and cleaning to ensure quality … and usability. Collaborate with engineers to optimise data infrastructure, implementing checks to maintain data accuracy and integrity. Communication: Articulate complex data analysis results to diverse stakeholders and keep projects progressing by liaising directly with key stakeholders. Lead meetings and present findings clearly to facilitate More ❯

landscape and make a meaningful difference in people's lives. Together, we can solve the world's toughest hiring problems. Within ITA, the Data team delivers high-quality recruiting data, reusable tools, and analytics reporting to Amazon Talent Acquisition customers. The Data team is … decisions a million times over. Focused on building scalable long term solutions, the team empowers engineers and consumers to develop, access, and analyze data independently with a high degree of data privacy and integrity. As a member of the Data team, you will deliver … robust data solutions that drive fair and efficient hiring and rapidly evolve with the needs of the business. Key job responsibilities Develop and maintain end to end scalable data infrastructure and data pipelines. Work closely with business owners, developers, Business Intelligence Engineers to explore More ❯

REQUIRED: DATA ENGINEER - SC CLEARED LOCATION: FULLY REMOTE WITH OCCASIONAL TRAVEL TO CENTRAL LONDON IR35 STATUS: INSIDE DURATION: 6 MONTH INITIAL CONTRACT Seeking a Data Engineer who has a strong understanding of data concepts - data types, data structures, schemas (both … JSON and Spark), schema management etc - Strong understanding of complex JSON manipulation - Experience working with Data Pipelines using a custom Python/PySpark frameworks - Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing/handling Reference … Data. - Strong understanding of Data Security principles - data owners, access controls - row and column level, GDPR etc including experience of handling sensitive datasets - Strong problem solving and analytical skills, particularly able to demonstrate these intuitively (able to work a problem out, not follow a work instruction More ❯

Senior Data Engineer (Ingestion) Hybrid Working (6 days per month onsite in our Farringdon office) Who are we? Toyota Connected Europe wants to create a better world through connected mobility for all. We are a new company created to bring big data and a customer focus … an opportunity to have an immediate impact and voice: what you create today, you will see being used tomorrow. About the role: The Data Engineering team enables and manages the ingestion of low latency, high volume car telemetry data that powers our engineering and data science teams to build smart and insightful products. We are looking for an experienced Data Engineer to join the team who will have a key role in the design, development, implementation and documentation of large-scale, distributed software data applications, systems and services. You More ❯

REQUIRED: DATA ENGINEER - SC CLEARED Location: Fully remote with occasional travel to Central London IR35 Status: Inside Duration: 6-month initial contract Job Description: Seeking a Data Engineer with a strong understanding of data concepts, including data types, data structures … schemas (JSON and Spark), and schema management. Key Skills and Experience: Strong understanding of complex JSON manipulation Experience with Data Pipelines using custom Python/PySpark frameworks Knowledge of the 4 core Data categories (Reference, Master, Transactional, Freeform) and handling Reference Data Understanding of … Data Security principles, access controls, GDPR, and handling sensitive datasets Strong problem-solving and analytical skills Experience in support roles, incident triage, and handling (SLAs) Linux system administration basics, Bash scripting, environment variables Experience with browser-based IDEs like Jupyter Notebooks Familiarity with Agile methodologies (SAFE, Scrum, JIRA More ❯

Job Description: Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in … a dynamic, fast-growing category? If so, you might be the ideal candidate for the role in the Data and Analytics function for Global Pet Nutrition (PN) at Mars. Pet Nutrition (PN) is the most vibrant category in the FMCG sector. As we work to transform this exciting … decision making, deep technical know-how, and an ability to navigate complex business challenges while building and leading a team of world class data and analytics leaders. With Digital First, PN is moving to a Product based model to create business facing digital capabilities. Develop and maintain robust More ❯

Job Title: Azure Data Engineer Lead - Outside IR35 Location: Remote Salary/Rate: £450 per day - Outside IR35 Start Date: 28/04/2025 Duration: 3 months We have an exciting opportunity with a major retail company. They are looking for a Lead Azure Data … outside IR35 and fully remote. Job Responsibilities/Objectives: Lead the design and execution of the end-to-end migration from on-premise data systems to Azure, ensuring scalability, reliability, and performance. Develop and optimise data pipelines using Azure Data Factory, Synapse Analytics, and … related services to support data ingestion, transformation, and integration. Architect and implement Azure-based data solutions, including Data Lake, SQL Database, and Databricks, aligning with business and technical requirements. Establish cloud data engineering best practices, including governance, security, monitoring, and cost optimisation. More ❯

Business Intelligence Engineer, Amazon Last Mile Come help Amazon create cutting-edge data and science-driven technologies for delivering packages to the doorstep of our customers! The Last Mile Routing & Planning organization builds the software, algorithms, and tools that make the "magic" of home delivery happen: our flow … Amazon Fresh, Lockers, and other new initiatives. As part of the Last Mile Science & Technology organization, you'll partner closely with Product Managers, Data Scientists, and Software Engineers to drive improvements in Amazon's Last Mile delivery network. You will leverage data and analytics to generate … scripting languages (e.g., Python, Java, or R) Master's degree or advanced technical degree Knowledge of data modeling and data pipeline design Experience with statistical analysis and correlation analysis Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have More ❯

Data Lead - Strategy & Integration - £500 (outside IR35) - London/Hybrid Investigo are supporting a growing IT Consultancy, currently going through an exciting period of change due to acquisitions, and now require a strategic, hands-on Data Strategy & Integration Lead. This hire will help architect and execute … the company-wide data infrastructure from the ground up. Building the data warehouse, connecting disparate systems, and driving the overall data vision. £500 p/day (Outside IR35) 6 months (with view to extend) Hybrid - London (city)/remote. The team currently work … systems and environments is essential Deep experience integrating CRM and financial platforms Solid understanding of ETL/ELT processes and modern data pipeline tools Hands-on experience managing data migrations from diverse legacy systems into new consolidated environments (e.g., post-M&A or system upgrades More ❯

Data Engineer 4 months - with extensions Remote Active SC clearance required £640 per day inside ir35 REQUIRED Strong understanding of data concepts - data types, data structures, schemas (both JSON and Spark), schema management etc Strong understanding of complex JSON manipulation Experience working … with Data Pipelines using a custom Python/PySpark frameworks Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing/handling Reference Data. Strong understanding of Data Security principles - data owners, access … CLI usage and scripting) Git Markdown Scala DESIRABLE Azure SQL Server as a HIVE Metastore DESIRABLE TECHNOLOGIES Azure Databricks Apache Spark Delta Tables Data processing with Python PowerBI (Integration/Data Ingestion) JIRA If you meet the above requirements, please apply for the vacancy to be More ❯

10959HS £550 - 620 per day Data Solutions Architect Hybrid - Inside IR35 We're looking for a seasoned Data Solutions Architect with deep expertise in the Lloyd's of London Market and a strong background in Snowflake and ETL/data transformation tools . As … a Data Architect , you'll lead the design, development, and deployment of enterprise-grade data architecture using Snowflake and a modern suite of data tools. You'll work at the intersection of IT and business, collaborating closely with stakeholders to create solutions that unlock … the full value of data-driving decision-making and business growth across the organisation. Key Responsibilities Architecture Design: Design and maintain enterprise data architecture models, data flows, and integrations using Snowflake. Data Governance: Manage data collection, storage, transformation, and protection More ❯

promise of AI. GX has built specialised knowledge AI assistants for the banking and insurance industry. Our assistants are fed by sector-specific data and knowledge and easily adaptable through ontology layers to reflect institution-specific rules. GX AI assistants are designed for Individual Investors, Credit and Claims … professionals and delivering 10x improvements by supporting them in their day-to-day tasks. Responsibilities: Helping to architect, design, implement, and optimise our data ingestion, transformation, and spreading pipelines and processes. Developing data models, processing pipelines, and back-end services supporting the data science … least one of the big 3 cloud ML stacks (AWS, Azure, GCP). Hands-on experience with open-source ETL, and data pipeline orchestration tools such as Apache Airflow and Nifi. Experience with large scale/Big Data technologies, such as Hadoop, Spark, Hive, Impala More ❯

and open-minded individual with an interest in state-of-the-art machine learning engineering and research? Thomson Reuters Labs is seeking a Data Engineer with a passion for solving challenging machine learning problems in a data-rich, complex and innovative environment. What does Thomson Reuters …/Applied ML, as well as Research, to drive the company's leading internal AI model development, fueled by an unprecedented wealth of data and powered by cutting-edge technical infrastructure. You will have the opportunity to contribute to a data curation & filtering system combining the … and ability to work with limited supervision. Experience with relational and NoSQL databases (e.g., PostgreSQL, MySQL, MongoDB, Cassandra). Experience with data pipeline orchestration tools. Experience with cloud-based data platforms such as AWS, GCP, or Azure (e.g., S3, BigQuery, Azure Data Lake More ❯

This is an exciting contract opportunity for an SC Cleared Azure Data Engineer to join an experienced team in a new customer engagement working at the forefront of data analytics and Artificial Intelligence. Our client equips ambitious organisations with enduring control of their data and AI evolution. This role is an opportunity to lead the build of bespoke data systems for our clients. Responsibilities: Design and implement scalable data pipelines and ETL processes using Azure and Databricks technologies including Delta Live Tables. Lead technical discussions with clients and … stakeholders to gather requirements and propose solutions. Help clients realise the potential of data science, machine learning, and scaled data processing within Azure/Databricks ecosystem. Mentor junior team members and support their personal development. Take ownership for the delivery of core solution components. Support with More ❯

to solve complex business problems. He/She will also be responsible for creating robust/automated reporting frameworks to increase visibility into data and enable data-driven decision making. Another key aspect of the job is to unearth insights from data to help … in driving process excellence. This position requires excellent statistical knowledge, superior analytical abilities, good knowledge of business intelligence solutions, and exposure to efficient data engineering practices. The BIE will also be a good stakeholder manager as he/she will have to work closely with Ops stakeholders. Candidates … Python, Java, or R). Minimum Qualifications: Master's degree or advanced technical degree. Knowledge of data modeling and data pipeline design. Experience with statistical analysis and correlation analysis. Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have More ❯

Fully remote - UK £70,000 (DOE) + Excellent benefits Data ignites passion, community fuels collaboration, and growth knows no bounds. Recognized for innovation and rapid growth, this esteemed consultancy has been named Databricks EMEA Emerging Business Partner of the Year and has achieved a second consecutive appearance on … the Financial Times FT1000 list. We are looking for Senior Data Engineers to join remotely in the UK. Be the first to have an influential impact on our UK hub. "As a Data Engineer, you will have the unique opportunity to push the boundaries of technology … record working with Databricks (PySpark, SQL, Delta Lake, Unity Catalog). You have extensive experience in ETL/ELT development and data pipeline orchestration (Databricks Workflows, DLT, Airflow, ADF, Glue, Step Functions). You're proficient in SQL and Python, using them to transform and optimize data. More ❯

My client, a prestigious multi-strat hedge fund, is seeking an exceptional Data Developer to join their Reporting & Analytics team within their London office. In this role, you will play a key role in designing and developing next-generation reporting and data analytics solutions, with a … on P&L reporting and operational decision-making. As part of a rapidly growing firm, you will work on revamping existing implementations, optimising data pipelines, and implementing rigorous data standards to support key business functions across the firm. This is a business-critical role, working closely … financial analytics, or similar data-driven workflows in a financial institution. Experience with database performance tuning, query optimization, and data pipeline automation. A deep understanding of financial markets, trading workflows, and risk analytics is highly beneficial. Ability to work independently and within a small, high More ❯

My client, a prestigious multi-strat hedge fund, is seeking an exceptional Data Developer to join their Reporting & Analytics team within their London office. In this role, you will play a key role in designing and developing next-generation reporting and data analytics solutions, with a … on P&L reporting and operational decision-making. As part of a rapidly growing firm, you will work on revamping existing implementations, optimising data pipelines, and implementing rigorous data standards to support key business functions across the firm. This is a business-critical role, working closely … financial analytics, or similar data-driven workflows in a financial institution. Experience with database performance tuning, query optimization, and data pipeline automation. A deep understanding of financial markets, trading workflows, and risk analytics is highly beneficial. Ability to work independently and within a small, high More ❯

as core functions. In this pivotal role, you'll be supporting these functions with business intelligence you derive from our vast array of data and will play a role in the long-term growth and success of Amazon in the APAC region. You will be working with stakeholders … team to build their strategies by using extremely large volumes of complex data. You will be exploring datasets, writing complex SQL queries, building data pipelines, and data visualization solutions with AWS Quicksight. You will also be building new Machine Learning models to predict the outcomes of … scripting language (e.g., Python, Java, or R) Master's degree or Advanced technical degree Knowledge of data modeling and data pipeline design Experience with statistical analysis, correlation analysis Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a More ❯

Are you looking for a role where you can design and implement cutting-edge data systems in a company making a real difference in healthcare? Do you want to be part of a mission-driven team revolutionising how healthcare teams communicate, while working in a hybrid setup that … values collaboration and flexibility? If you’re a hands-on Data Architect with a passion for building scalable systems and solving complex data challenges, this could be the perfect role for you. What’s in it for you? You’ll receive a salary of up to … parental leave, and even a policy that allows you to work abroad for short periods. What you’ll be doing As the Senior Data Architect, you’ll be the first hire in this role, reporting directly to the Head of Data. This is a hands-on position where More ❯