51 to 75 of 123 Data Pipeline Jobs in the South East

Senior Data Engineer UK Remote, £500/day, Outside IR35 A Senior Data Engineer is required by fast-growing HealthTech/Tech for Good start-up. This is a 3-month contract initially with scope for extension. In this role, you will design, build, and maintain robust data pipelines using streaming and batch processing … technologies and help to lead the modernization of their data lakehouse. You will implement and manage secure cloud infrastructure using AWS, championing a culture of data quality, security, and technical excellence. Key Responsibilities: Architect and maintain data pipelines (streaming & batch) Build and manage scalable data storage solutions (data lakehouse) Design … and maintain secure AWS cloud infrastructure Implement and manage CI/CD and DevOps pipelines Champion data quality, security, and best practices Collaborate with cross-functional teams Implement and manage MLOps capabilities Essential Skills: Advanced Python programming skills Expertise in data engineering tools and frameworks (Apache Flink) Hands-on AWS experience (Serverless, CloudFormation, CDK) Strong understanding More ❯

THIS ROLE IS OFFICE BASED - CITY OF LONDON We’re looking for a curious, driven Data Engineer to help modernize legacy platforms, build cutting-edge data pipelines, and shape the future of our data strategy. You’ll design ETL processes, develop APIs, and turn messy data into powerful insights that fuel every … corner of our business. What you’ll do: Build and own robust, scalable data pipelines & APIs. Design smart cloud solutions with SQL, dbt, and Azure. Champion data quality — because data is a product, not just a process. Collaborate on architectural decisions and mission-critical projects. Solve complex data challenges with creativity and … precision. What you bring: BS in Computer Science, Engineering, or equivalent. 7+ years plus experience working with Azure/Data Strong SQL, Python, and data warehousing/ETL expertise. Hands-on Azure experience (certification = a big plus). Experience with IaC, CI/CD, and modern DevOps tools. Excellent communication skills — you can talk tech with More ❯

Data Consultant - ETL Projects UK - Remote | Permanent Up to £55,000 + Benefits Are you a Data Consultant with a strong background in hands-on ETL development and data transformation? Are you looking to lead technically complex data migration projects that genuinely impact digital services across Local Government? We're hiring for … an experienced Data Consultant to join a growing team delivering mission-critical data migration projects. You'll be part of an expert unit supporting onboarding for a new cloud platform - working directly with raw datasets from legacy systems and transforming them into future-ready digital infrastructure. What You'll Be Doing: Leading data migration … and ETL projects for UK public sector software implementations Analysing and mapping legacy database schemas (Oracle, SQL Server, Postgres) Extracting raw data from legacy systems, often with limited documentation Writing and executing SQL scripts and/or using ETL tools to build data pipelines Creating transformation logic and performing reconciliation and validation Working closely with customers More ❯

Senior Data Engineer (Kafka Python AWS) London/WFH to £80k Are you a data technologist with fluent Python coding skills who enjoys working with and continually learning new technologies? You could be progressing your career in a senior, hands-on role at a "Tech for Good" company that is enabling life changing education to be … accessed by millions of students across the globe (currently 22 countries and growing). As a Data Engineer you'll help to design, build and operate the data pipelines and data services that power learning experiences. Collaborating with software engineers, data scientists and product managers, you will grow your expertise while contributing … as well as reporting and analytics requirements across the business as it scales to 10's of millions of students. Typically you'll be developing and maintaining reliable data pipelines for both real-time streaming and batch workloads, assisting with the evolution of the Lakehouse, collaborating with analytics, product and machine learning teams to make data More ❯

to join our Analytics Engineering team, with a focus on supporting our commercial team and data. This is an exciting opportunity to hone your skills with the modern data stack while working closely with sales, finance and data teams to deliver robust reporting and insights. You will play a key role in building scalable data pipelines, maintaining reliable reporting frameworks, and ensuring the accuracy of commercial data that powers decision-making across the business. This role would suit someone who is starting their career in Analytics Engineering and is keen to learn quickly from an experienced team and is not afraid to get their hands dirty. Key Responsibilities Be the first … point of contact for all Commercial data requests. Investigate these requests and either solve them or triage them. Contribute to building and maintaining a best-in-class Looker setup, supporting dashboards and Explores that drive adoption and value across the business. Partner with commercial stakeholders to understand reporting needs, translating them into effective Looker solutions that support sales More ❯

Marketing AdTech Data Analyst Location: London (hybrid 3 days per week onsite) Contract: 12 months, up to £425pd inside IR35 Our client is looking for a highly skilled Data Analyst to join a fast-paced ad-tech environment supporting data-driven marketing and advertising initiatives. This role is ideal for someone who thrives at … marketing strategy and technical execution, with strong hands-on experience in Tableau, SQL and Python. You'll work closely with global stakeholders across Marketing, Sales, Engineering and external data partners to deliver insights, optimise campaign performance and shape go-to-market strategies for digital advertising products. What You'll Do: Deliver custom insights and analytics to support marketing … KPIs, campaign performance and user engagement. Develop and maintain ETL pipelines using SQL and Python. Translate business needs into analytical requirements, communicating findings with clarity and impact. Conduct data analysis and A/B testing to measure the effectiveness of marketing initiatives. Contribute to scalable reporting solutions, data transformation and automation strategies. What You'll Bring More ❯

LondonCompetitive Salary + Benefits + Bonus Deerfoot is partnering with a leading international banking group to hire a Market Risk BI Developer who will design and implement advanced data solutions using Snowflake and Power BI . This role works closely with risk managers and business stakeholders to build scalable analytics for market risk (including VaR, sensitivities, and stress … architecture. Collaborate with risk managers and business users to translate requirements into technical solutions. Troubleshoot data issues, optimise performance, and ensure system reliability. Support CI/CD pipeline deployment and automated testing for data solutions. Mentor junior developers and foster knowledge sharing within the team. Skills & Experience Required: Proven experience as a BI Developer or … and Power BI. Strong expertise in data modelling, query optimisation, and Power BI development (DAX, dashboards). Hands-on with dbt, SQL, Python, and ELT/ETL pipeline development. Experience with Azure Data Factory (ADF), and migrating legacy data platforms to cloud solutions. Financial services exposure, ideally with market or credit risk analytics. More ❯

Data Integration Engineer - Investment Banking My client, a top-tier investment bank, is seeking a Data Integration Engineer to join their growing data team. This is a hands-on engineering role focused on PL/SQL , Databricks , and Azure-based data lakehouse environments . You'll play a key role in building … and maintaining robust ETL pipelines , supporting data migration efforts, and collaborating closely with DBAs and Technical Leads . While prior experience in banking or regulatory reporting is a plus, it's not a requirement. Key Skills & Experience: Essential: Strong hands-on experience with PL/SQL and T-SQL Proven expertise in Databricks (on-prem and cloud) Solid … background in ETL development , data pipelines , and data migration Familiarity with Azure Storage and Data Lakehouse architecture Experience with SQL Server and Oracle databases Proficiency with ETL tools such as SSIS , DataStage , or Alteryx Strong collaboration skills with DBAs and Tech Leads Excellent problem-solving and communication abilities Desirable: Exposure to AQR Reporting for More ❯

compromised. Architect, develop, and maintain core SaaS platform components across the stack: Backend: Python, PostgreSQL Frontend: TypeScript, NextJS/NodeJS Ensure smooth integration of AI tools, APIs, and data pipelines to enhance platform intelligence. Champion engineering best practices, including testing, CI/CD, and secure coding standards. Conduct regular code reviews and provide technical mentorship. Reporting to the More ❯

IR35) We're working with a fully remote, boutique AI consultancy and Palantir services partner. They help organisations in energy, healthcare, financial services, media, and recruitment use modern data and AI platforms to solve complex business challenges. They are building a Palantir Foundry practice led by ex-Palantir leadership. Thanks to these relationships, they're trusted with high … to see their work make an immediate impact. What You'll Do Work closely with clients to solve their toughest challenges using Palantir Foundry and modern AI/data tools. Design and deploy scalable data pipelines, workflows, and applications. Model complex datasets and create APIs that deliver real business value. Collaborate with technical and non-technical … data platforms such as: Snowflake, Databricks, AWS Glue/Redshift, Google BigQuery Software engineering skills in Python, Java, or TypeScript/React. Strong data modelling, pipeline development, and API design experience. Excellent problem-solving and communication skills. Why This Role Stands Out Impactful work at the intersection of AI, data, and business transformation. More ❯

Data Engineer (Snowflake & Asset Management) 6 Month Contract London (Hybrid) Via Umbrella Our UK leading Life Insurance client are looking for a Data Engineer to join their team on an initial 6-month contract. Key skills/knowledge required: Proficiency in Python and SQL for data engineering and application development Experience building data pipelines and ETL processes for capital markets data Hands-on experience with AWS cloud services and Snowflake data platform Experience with cloud-native development patterns and AWS services (Lambda, S3, RDS, etc.) Hands-on experience integrating with OMS platforms (Aladdin, Murex, Charles River, etc.) including data extraction, transformation, and real-time feeds … into technical solutions Strong collaboration and communication skills, with ability to work effectively in cross-functional teams Understanding of trading workflows, trade lifecycle, and settlement processes Knowledge of data engineering workflows, pipelines, and integration patterns Awareness of regulatory and compliance considerations in capital markets delivery Familiarity with agile principles and team-based delivery Desirable: Familiarity with agile delivery More ❯

We are looking for a talented Data Engineer , specialising in designing and optimising scalable data pipelines, and cloud-based data solutions. This position requires expertise in Python, SQL, dpt and experience with Snowflake. If you have industry experience working with Retail and CPG this could be the role for you! Our client is the … leading collaborative commerce platform that connects CPG brands to real-time sales and inventory data from 40+ retailers and distributors. We serve 7,000+ brands representing over $2.5T in retail sales across 250K+ stores, working with industry leaders like Nestlé, and KraftHeinz. This business believes that teamwork drives innovation. By uniting talented, motivated creators from diverse backgrounds, they … and efficient. If youre passionate about solving meaningful problems while building a company youll be proud of, wed love to hear from you. About the position As a Data Engineer, youll play a vital role in turning complex, messy datasets into reliable, actionable insights that drive smarter decisions for our customers. Youll design and optimise scalable data More ❯

grown into a dynamic team with a strong reputation for digital innovation and customer-centricity - and they're just getting started. With an award-winning platform and a data-driven approach to decision-making, they combine cutting-edge technology with a commitment to transparency and service excellence. With a strong focus on digital transformation, automation, and AI, they … and redesign manual processes. Supporting integration of APIs, SaaS platforms, and internal systems (e.g. SQL Server, Azure). Experimenting with generative AI and agentic workflows (e.g. intelligent routing, data extraction). Working with the Data Engineering team to enable automations with trusted data pipelines. Documenting workflows, decision rules, and automation logic for transparency and … audit. Participating in Agile sprints & code reviews. Job requirements - Automation Developer Graduate in Computer Science, Software Engineering, Data Science, AI/ML, Information Systems, or a related STEM field. Curiosity and passion for AI, automation, and problem-solving. Evidence of working on AI/Automation/Workflow projects - this can be from university, personal projects, internships, or work More ❯

Palantir Foundry Consultant – London – Up to £100,000 per annum Are you a data-driven professional with a passion for delivering impactful solutions using Palantir Foundry? Do you excel in translating complex data challenges into actionable insights? If so, Morela has an exciting opportunity for you. Morela is proud to partner with one of the UK … s leading Palantir consultancies, renowned for their innovative approach to data integration and analytics. As they continue to expand, they're seeking a skilled Palantir Foundry Consultant to join their dynamic team. This role is perfect for someone who combines technical expertise with a consultative mindset, aiming to drive value across diverse projects. You'll be part of … operates at the intersection of advanced technology and real-world impact. Serving both public and private sectors, their services encompass strategic advisory, digital transformation, and end-to-end data engineering. As a Palantir Foundry Consultant, you'll play a crucial role in designing and implementing data solutions that empower clients to make informed decisions. Core Responsibilities More ❯

Palantir Foundry Consultant – London – Up to £100,000 per annum Are you a data-driven professional with a passion for delivering impactful solutions using Palantir Foundry? Do you excel in translating complex data challenges into actionable insights? If so, Morela has an exciting opportunity for you. Morela is proud to partner with one of the UK … s leading Palantir consultancies, renowned for their innovative approach to data integration and analytics. As they continue to expand, they're seeking a skilled Palantir Foundry Consultant to join their dynamic team. This role is perfect for someone who combines technical expertise with a consultative mindset, aiming to drive value across diverse projects. You'll be part of … operates at the intersection of advanced technology and real-world impact. Serving both public and private sectors, their services encompass strategic advisory, digital transformation, and end-to-end data engineering. As a Palantir Foundry Consultant, you'll play a crucial role in designing and implementing data solutions that empower clients to make informed decisions. Core Responsibilities More ❯

MI Developer has become available within Saga's finance department. Supporting both the Finance team and the wider Insurance business, this role sits within a team of 4 Data and Analytics professionals and provides the opportunity to work across multiple data sources, platforms and analytics tools. The role is all about providing technical data … Academy, our bespoke learning platform Main Responsibilities Following our Business Code of Conduct and always acting with integrity and due diligence Supporting our Ops MI team with their data pipelines, troubleshooting issues as they occur and amending processes as business requirements develop Specifying, designing and automating of business processes using BI tools Creating and maintaining technical documentation and … best practices for all aspect of data extraction and transformation Assisting the team with identifying gaps in data sets and ensuring we have optimal data quality for accurate and timely reporting and MI Working with the business and technical stakeholders, identify and develop data sets to assist in efficient reporting and MI More ❯

Integration Engineer Mainly Remote (monthly meet ups with team) Outside IR35 Experienced Integration Engineer needed to help design and develop data integration solutions using Informatica IPaaS and Snowflake data Lakehouse You will Design, develop and implement ELT & ETL pipelines and workflows using Informatica PowerCenter and Informatica Cloud. Build connectors from multiple sources including Cloud platforms, bespoke … and out of the box APIs, databases, and flat files. Develop and maintain data mappings, transformations, data pipelines and integration workflows. Implement real-time and … batch data integration solutions to support business requirements Ensure any integration pipelines or ELT/ETL workflows are built with error and exception handling built into the pipeline/workflow. Your experience Proven experience in a similar role so you can provide advice and guidance to the team Excellent experience in Informatica ELT & ETL development and integration More ❯

Integration Engineer Mainly Remote (monthly meet ups with team) Outside IR35 Experienced Integration Engineer needed to help design and develop data integration solutions using Informatica IPaaS and Snowflake data Lakehouse You will Design, develop and implement ELT & ETL pipelines and workflows using Informatica PowerCenter and Informatica Cloud. Build connectors from multiple sources including Cloud platforms, bespoke … and out of the box APIs, databases, and flat files. Develop and maintain data mappings, transformations, data pipelines and integration workflows. Implement real-time and … batch data integration solutions to support business requirements Ensure any integration pipelines or ELT/ETL workflows are built with error and exception handling built into the pipeline/workflow. Your experience Proven experience in a similar role so you can provide advice and guidance to the team Excellent experience in Informatica ELT & ETL development and integration More ❯

into the clear and concise technical development specifications Managing and updating the project management board for responsible projects aligned with the I&D Analytics project roadmap Working with data engineers and business owners to create data pipelines for reporting and analytics with consideration of the planned system integration, transformation and data migration Delivering Power … BI data models and reports with the Power BI developers' best practise and compliance with the SNG data policies and procedures Supporting the maintenance, management and enhancement of I&D systems and ensuring they are fit for purpose and continuously improved, and supporting the routine review of the quality and completeness of data What … a similar role as an Insight Analyst, ideally in the Real Estate, or broader Investment and Development sectors. You will need strong Power BI reporting experience with advanced data analytics and visualisation skills. You'll also have: Strong financial acumen and commercial knowledge to interpret complex financial and investment analysis Excellent stakeholder management skills with clear communication and More ❯

into the clear and concise technical development specifications Managing and updating the project management board for responsible projects aligned with the I&D Analytics project roadmap Working with data engineers and business owners to create data pipelines for reporting and analytics with consideration of the planned system integration, transformation and data migration Delivering Power … BI data models and reports with the Power BI developers' best practise and compliance with the SNG data policies and procedures Supporting the maintenance, management and enhancement of I&D systems and ensuring they are fit for purpose and continuously improved, and supporting the routine review of the quality and completeness of data What … a similar role as an Insight Analyst, ideally in the Real Estate, or broader Investment and Development sectors. You will need strong Power BI reporting experience with advanced data analytics and visualisation skills. You'll also have: Strong financial acumen and commercial knowledge to interpret complex financial and investment analysis Excellent stakeholder management skills with clear communication and More ❯

Make your mark in a collaborative and purpose-driven team.** We are seeking a Senior AWS Data Engineer to join a leading organisation's AWS - Data team. This permanent, hybrid position requires you to work in the office two-three days a week in London. This is a great opportunity for candidates who are passionate about … data engineering and want to contribute to impactful projects in a supportive environment. **Key Responsibilities:**- Develop and maintain AWS data pipelines and infrastructure.- Collaborate with cross-functional teams to design data solutions.- Optimise existing data processes for efficiency and performance.- Ensure data quality and security standards are met.- Stay up … to-date with AWS developments and best practices.**Key Requirements:**- Proven experience with AWS services and tools.- Strong knowledge of data modeling and ETL processes.- Proficiency in programming languages such as Python or SQL.- Excellent problem-solving skills with a proactive approach.- Ability to communicate effectively within a team.If you are a skilled and driven AWS Data More ❯

JoinusasaDataPlatformEngineerforourclient. AtPeregrine,werealwaysseekingSpecialistTalentthathavetheidealmixofskills,experience,and attitude,toplacewithourvastarrayofclients.FromBusinessAnalystsinlargegovernment organisationstoSoftwareDevelopersintheprivatesectorwearealwaysinsearchofthebest talenttoplace,now. HowSpecialistTalentWorks AtPeregrine,wefindthebesttalentforourclients.AsapermanentemployeeofPeregrine,with accesstoallourstandardbenefits,youwillbedeployedacrossourportfolioofclientsasaspecialistconsultant,workingonawidearrayofcomplexprojectsacrossmultipleindustries. TheRole: AsaDataPlatformEngineerinahighlyregulatedenvironment,youwillberesponsiblefordesigning,building,andmaintainingsecureandscalabledatainfrastructurethatsupportsbothcloudandonpremisesplatforms.Youwillplayakeyroleinensuringthatalldatasystemscomplywithindustryregulationsandsecuritystandardswhileenablingefficientaccessforanalyticsandoperationalteams. AstrongcommandofApacheNiFiisessentialforthisrole.Youwillbeexpectedtodesign, implement,andmaintaindataflowsusingNiFi,ensuringaccurate,efficient,andsecuredata ingestion,transformation,anddelivery.Youshouldbeadeptatidentifyingandresolvingissues withinNiFiflows,managingperformancebottlenecks,andimplementingrobusterrorhandling strategies. Youllworkcloselywithcross-functionalteamsincludingdataarchitects,complianceofficers,and cybersecurityspecialiststointegratedatafromvarioussystemssuchasdatabases More ❯

JoinusasaDataPlatformEngineerforourclient. AtPeregrine,werealwaysseekingSpecialistTalentthathavetheidealmixofskills,experience,and attitude,toplacewithourvastarrayofclients.FromBusinessAnalystsinlargegovernment organisationstoSoftwareDevelopersintheprivatesectorwearealwaysinsearchofthebest talenttoplace,now. HowSpecialistTalentWorks AtPeregrine,wefindthebesttalentforourclients.AsapermanentemployeeofPeregrine,with accesstoallourstandardbenefits,youwillbedeployedacrossourportfolioofclientsasaspecialistconsultant,workingonawidearrayofcomplexprojectsacrossmultipleindustries. TheRole: AsaDataPlatformEngineerinahighlyregulatedenvironment,youwillberesponsiblefordesigning,building,andmaintainingsecureandscalabledatainfrastructurethatsupportsbothcloudandonpremisesplatforms.Youwillplayakeyroleinensuringthatalldatasystemscomplywithindustryregulationsandsecuritystandardswhileenablingefficientaccessforanalyticsandoperationalteams. AstrongcommandofApacheNiFiisessentialforthisrole.Youwillbeexpectedtodesign, implement,andmaintaindataflowsusingNiFi,ensuringaccurate,efficient,andsecuredata ingestion,transformation,anddelivery.Youshouldbeadeptatidentifyingandresolvingissues withinNiFiflows,managingperformancebottlenecks,andimplementingrobusterrorhandling strategies. Youllworkcloselywithcross-functionalteamsincludingdataarchitects,complianceofficers,and cybersecurityspecialiststointegratedatafromvarioussystemssuchasdatabases More ❯

/ML model development and deployment Strong skills with Python, TensorFlow/PyTorch, and cloud platforms (AWS/Azure/GCP) MLOps assurance and risk experinece Knowledge of data pipelines, model optimisation, and real-world deployments Details: Contract: 6 months (with view to extend) Location: London (Hybrid) Start: ASAP If you’re interested, please apply or reach out More ❯

team on an initial 6-month contract. Key skills/knowledge required: Strong proficiency in Python-based testing frameworks (e.g., Behave, PyTest, unittest) Experience in automated testing of data pipelines and ETL processes Experience with data validation frameworks and testing data transformations across Snowflake and AWS environments Knowledge of API testing tools and frameworks … Postman, REST Assured, or similar) Hands-on experience testing OMS platform integrations (Aladdin, Murex, Charles River) and validating trade lifecycle data flows Experience integrating automated tests into CI/CD pipelines for continuous delivery Hands-on experience validating data accuracy and integrity across multiple systems Experience working embedded within cross-functional agile teams rather than separate … QA departments Strong problem-solving skills with ability to design effective test strategies for complex data workflows Ability to work collaboratively with engineers, delivery roles, and business stakeholders Knowledge of testing methodologies for data pipelines, APIs, and integrations Knowledge of data quality testing patterns and anomaly detection in capital markets data Understanding More ❯