76 to 100 of 123 Data Pipeline Jobs in the South East

Data Engineer - AI/ML | Leading Pharma | London (Hybrid) My client a leading pharmaceutical company is looking for a Data Engineer who thrives on solving complex challenges and building scalable data solutions that power scientific innovation. Key Responsibilities Build and maintain data pipelines using Python, Spark, SQL, BigQuery , and Google Cloud Collaborate … with scientists to meet research data needs Ensure high-quality, well-documented, and tested code Improve performance metrics and integrate data across teams Requirements Essential: Experience in data engineering Strong Python & SQL skills Experience with Google Cloud or similar Familiarity with unit testing , Git , and Agile Desirable: Experience with bioinformatics or scientific datasets Knowledge More ❯

CONTRACT PALANTIR DATA ENGINEER - DV CLEARED NEW OUTSIDE IR35 CONTRACT OPPORTUNITY AVAILABLE WITHIN A HIGHLY SECURE DELIVERY PROGRAMME FOR A DV CLEARED PALANTIR FOUNDRY DATA ENGINEER. Contract opportunity for a Palantir Data Engineer to support cutting-edge National Security projects Up to £650 per day (Outside IR35) DV clearance is essential Based full-time … using Foundry tools like Workshop Strong communication skills to collaborate with technical and military/non-technical stakeholders Experience integrating data from multiple sources into Foundry via Pipeline Builder and custom code Comfortable working on-site within a secure environment and at pace with end-users TO BE CONSIDERED... . Please either apply by clicking online or … consent for us to process and submit your application to our client in conjunction with this vacancy only. KEY SKILLS: PALANTIR/FOUNDARY/DATA ENGINEER/PIPELINE BUILDER/ONTOLOGY/PYTHON/TYPESCRIPT/FULL STACK/DV CLEARED/LONDON/CONTRACT More ❯

Technical Business Analyst, SC Cleared We are seeking a highly skilled Technical Business Analyst with over 8 years of experience to support the evolution of enterprise modern cloud data platform used for ingesting and processing large volumes of streaming and batch data sets to support analytically and AI use-case. As a key interface between the … technical and business stakeholders, you will be responsible for translating data engineering, architectural and analytically requirements into detailed, actionable user stories with clear audit tracking and version control. Key Responsibilities Collaborate with Data Architects, Engineering teams, Security, Product teams and business stakeholders to gather, analyse, and translate their needs into clear and technically sound requirements. Facilitate … with stakeholders to mitigate issues. Champion agile practices, facilitating sprint planning and retrospectives where needed. Key Skills 8+ years of experience as a Technical Business Analyst in complex, data-centric environments. Proven ability to write technical user stories for data pipelines, ETL processes, and analytics platforms. Strong understanding of data platforms, data More ❯

A small and highly successful organisation in the sustainability space are seeking a Data Engineer with some Power Platform experience to join their team. They have an office space in London, though this role is remote and is therefore open to candidates across the UK. They are on a mission to help businesses to reduce their carbon emissions … ecosystem - including Azure, Power Apps, Power Pages and Power BI, and allows for real-time engagement and AI-guided action plans for carbon reduction. Responsibilities include: Use Azure Data Factory to ingest, transform, and expose data across the platform Help centralise data into a Data Lake, ensuring it's clean, structured, and … accessible Build data pipelines for LLMs and integrate tools like GPT to drive intelligent outputs Develop user-facing applications using Power Apps and Power Pages Design and manage APIs to connect front-end applications and integrate external services (e.g. procurement systems, AI tools) Collaborate directly with users to turn complex needs into streamlined solutions It's a broad More ❯

A small and highly successful organisation in the sustainability space are seeking a Data Engineer with some Power Platform experience to join their team. They have an office space in London, though this role is remote and is therefore open to candidates across the UK. They are on a mission to help businesses to reduce their carbon emissions … ecosystem - including Azure, Power Apps, Power Pages and Power BI, and allows for real-time engagement and AI-guided action plans for carbon reduction. Responsibilities include: Use Azure Data Factory to ingest, transform, and expose data across the platform Help centralise data into a Data Lake, ensuring it's clean, structured, and … accessible Build data pipelines for LLMs and integrate tools like GPT to drive intelligent outputs Develop user-facing applications using Power Apps and Power Pages Design and manage APIs to connect front-end applications and integrate external services (e.g. procurement systems, AI tools) Collaborate directly with users to turn complex needs into streamlined solutions It's a broad More ❯

A small and highly successful organisation in the sustainability space are seeking a Data Engineer with some Power Platform experience to join their team. They have an office space in London, though this role is remote and is therefore open to candidates across the UK. They are on a mission to help businesses to reduce their carbon emissions … ecosystem - including Azure, Power Apps, Power Pages and Power BI, and allows for real-time engagement and AI-guided action plans for carbon reduction. Responsibilities include: Use Azure Data Factory to ingest, transform, and expose data across the platform Help centralise data into a Data Lake, ensuring it's clean, structured, and … accessible Build data pipelines for LLMs and integrate tools like GPT to drive intelligent outputs Develop user-facing applications using Power Apps and Power Pages Design and manage APIs to connect front-end applications and integrate external services (e.g. procurement systems, AI tools) Collaborate directly with users to turn complex needs into streamlined solutions It's a broad More ❯

A small and highly successful organisation in the sustainability space are seeking a Data Engineer with some Power Platform experience to join their team. They have an office space in London, though this role is remote and is therefore open to candidates across the UK. They are on a mission to help businesses to reduce their carbon emissions … ecosystem - including Azure, Power Apps, Power Pages and Power BI, and allows for real-time engagement and AI-guided action plans for carbon reduction. Responsibilities include: Use Azure Data Factory to ingest, transform, and expose data across the platform Help centralise data into a Data Lake, ensuring it's clean, structured, and … accessible Build data pipelines for LLMs and integrate tools like GPT to drive intelligent outputs Develop user-facing applications using Power Apps and Power Pages Design and manage APIs to connect front-end applications and integrate external services (e.g. procurement systems, AI tools) Collaborate directly with users to turn complex needs into streamlined solutions It's a broad More ❯

Data Engineer (Defence) £45,000 - 55,000 GBP Onsite WORKING Location: Stevenage, Cambridge, East of England - United Kingdom Type: Permanent Data Engineer - Generative AI | Defence Technology A leading international defence organisation is seeking an experienced Data Engineer with a focus on Generative AI to join their IM GenAI Delivery Office in the UK. You … support advanced defence systems and critical internal projects. What you'll do: Develop and maintain robust, secure, and high-integrity datasets for internal defence applications. Design and optimise data pipelines to ensure reliable, compliant, and scalable solutions. Apply your expertise in Generative AI, NLP, and OCR to enhance and automate data processes. Build ETL workflows, APIs … and other data integration solutions tailored for mission-critical environments . Provide guidance on emerging technologies and support the roadmap for next-generation defence data solutions . Skills and experience: Strong knowledge of SQL (MS SQL, Oracle) and noSQL databases (e.g., MongoDB, Neo4J, InfluxDB). Hands-on experience with data processing and integration tools More ❯

Lead Data Engineer- £70,000 Industry: Banking and Financial Services Location : Flexible- Basingstoke (3 Days a Week) Overview of Role: This role will lead the design and optimisation of our Azure Databricks data platform , ensuring it is secure, scalable, and well governed. You'll take ownership of change controls, lineage, and documentation while driving adoption of … best practice. Alongside this, you'll support AI/ML initiatives and provide technical guidance to colleagues, driving value from data. Key Responsibilities Lead the roadmap for data engineering, ensuring alignment with business needs and regulatory requirements. Own the design and optimisation of the Azure Databricks platform, focusing on performance, resilience, and compliance. Implement change controls, lineage, and … observability to detect and resolve issues. Maintain clear documentation of data pipelines, models, and governance processes. Implement tools to improve data quality, efficiency, and adoption. Support AI/ML delivery by building/optimising, ensuring platform readiness. Manage SQL workloads within Databricks, balancing cost and performance Ensure access controls, GDPR compliance, and security standards Work with More ❯

Data Scientist (LLMs & ML) - UK Remote A fast-growing healthcare organization is seeking a Data Scientist with strong experience in machine learning, deep learning, and Large Language Models (LLMs) to help drive innovation and automation in clinical services. This is a full-time, remote UK-based role. ***Unfortunately we can't consider candidates that require sponsorship … solutions (e.g., LangChain, Mastra.ai, Pydantic) for document processing, summarization, and clinical Q&A systems. Develop and optimize predictive models using scikit-learn, PyTorch, TensorFlow, and XGBoost. Design robust data pipelines using tools like Spark and Kafka for real-time and batch processing. Manage ML lifecycle with tools such as Databricks , MLflow , and cloud-native platforms (Azure preferred). … Collaborate with engineering teams to ensure scalable, secure ML infrastructure aligned with compliance standards (e.g., ISO27001). Ensure data governance, particularly around sensitive healthcare data. Share best practices and stay current with developments in AI, ML, and LLMs. Proven experience with LLM frameworks and transformer-based architectures. Strong Python skills and familiarity with key ML/DL libraries. More ❯

As an FPGA/Digital Design Engineer, you will: Architect, design, implement, verify, and integrate digital logic (RTL) for FPGA/SoC platforms Work on signal processing, communications, data pipelines, digital beamforming, or related SDR blocks Develop testbench environments, simulations, and formal/verifiable test strategies Perform timing closure, clock domain crossing, resource balancing, power optimisation Define interfaces More ❯

Lead Data Engineer - £65-70k + bonus + benefits - Basingstoke 3 days a week The Opportunity: We are looking for a Lead Data Engineer to join a Basingstoke based organisation who are investing heavily in their Digital Transformation Programme. The Role: There is currently 1 Data Engineer in the team and you'll … join to play a proactive role in the delivery of their next-generation data platform, manage/mentor the existing person and drive the design, development and governance of the data pipelines. You'll be working really closely with stakeholders across the technology function and within the business and will the availability, integrity and compliance of … background/attributes include: - In depth experience of modern data solution architecture design and delivery in a hybrid cloud environment but predominantly Azure/Databricks - Experience of pipeline orchestration management - Strong exprerience with Databricks - Experience of implementing machine learning and AI - Tooling such as Purview and Unity Catalog as well as the use of observability tools such More ❯

Genome Campus near Cambridge, UK. The Chemical Biology Resources team provides world-leading chemogenomics resources to the scientific community. ChEMBL is a database of quantitative small-molecule bioactivity data curated primarily from the scientific literature widely used to support drug discovery projects in industry and academia. SureChEMBL is a complementary patent resource containing chemical structures extracted from patents … of biological interest. OPSIN converts systematic chemical names into chemical structures. In this role, you will develop and maintain software infrastructure to ensure robust, reliable, and automated ChEMBL data releases through streamlined build, packaging, testing, and deployment processes, while also assisting in the development and maintenance of other services supported by the team. You will be working closely … mix of professional scientists and software engineers. The EBI is part of the European Molecular Biology Laboratory (EMBL) and it is a world-leading bioinformatics centre providing biological data to the scientific community with expertise in data storage, analysis and representation. EMBL-EBI provides freely available data from life science experiments, performs basic research More ❯

Data Engineer - AI & Machine Learning Focus Location: Kings Cross, London Hybrid Working: 2-3 days in the office, remainder from home Duration: 6 Months Contract Type: Inside IR35 (via Umbrella) Are you passionate about building data solutions that drive cutting-edge AI and Machine Learning applications? We're looking for a talented Data Engineer … to join a forward-thinking team developing transformative therapies for both existing and emerging diseases. This is your opportunity to work on impactful projects at the intersection of data engineering and life sciences, using modern cloud technologies to deliver scalable, reliable data pipelines that support scientific innovation. What You'll Be Doing: Designing and building data pipelines using tools like Python, Spark, SQL, BigQuery, and Google Cloud Storage Ensuring pipelines meet the specific needs of data-driven scientific applications Writing high-quality, well-documented code with automated testing Monitoring and improving key metrics across tools and services Collaborating with cross-functional teams to build end-to-end data solutions Participating More ❯

Retention Data | BI Analyst 12-month contract day rate of £230-£255 (umbrella) Are you passionate about turning data into insights that drive customer loyalty? We are looking for a skilled BI Analyst on a 12-month contract to work on high-impact projects focused on customer retention . This role offers a You will transform … you will uncover actionable insights and drive retention-focused initiatives. Key Responsibilities Build automated, scalable reporting solutions focused on customer retention . Create dashboards combining internal and external data sources for senior leadership presentations. Collaborate with teams to identify gaps in data capture strategies and implement enhancements. Provide actionable insights on key retention metrics. Maintain relationships … with stakeholders to ensure retention-related data needs are met. Qualifications Proven experience as a BI Analyst or similar role, ideally focused on customer retention . Strong ability to analyze and combine large datasets using advanced SQL . Experience with BI/reporting tools such as Tableau or Qlik . Experience building and maintaining data pipelines More ❯

Data Analytics Engineer My client in the Insurance industry are searching for a Data Analytics Engineer. We are looking for someone who has: A strong engineering background and possibly BI experience. Solid proficiency in SQL and Python as core programming skills. Proven experience in building data pipelines, including API integrations. Knowledge of Power BI … and Databricks. Key responsibilities will include: Supporting BAU (Business-as-Usual) tasks. Leading a major project to rebuild two core APIs and their associated pipelines. Assisting in optimising several pipeline SQl based. Assisting in optimising a few Power BI semantic models. Please apply! Carbon60, Lorien & SRG - The Impellam Group STEM Portfolio are acting as an Employment Business in relation More ❯

Overview Role Description In this role you will collaborate with data scientists, ML engineers, and analysts to design and implement solutions that extract insights from unstructured text. Projects may include topic modelling and entity recognition, text generation, ontology creation, and conversational AI. This role demands strong technical expertise in NLP and a passion for innovation and problem-solving. … level ML/GenAI strategies aligned with business objectives. Designing and developing NLP solutions using techniques such as text classification, NER, topic modeling, and text generation. Collaborating with data engineers and MLOps teams to ensure clean data pipelines and seamless model integration. Applying LLMs (e.g., GPT, BERT, LLaMA2) and knowledge graphs to enhance natural language understanding … and reasoning. Conducting exploratory data analysis and developing robust pre-processing strategies for unstructured text. Evaluating NLP models, algorithms, and architectures to recommend optimal solutions. Communicating insights through reports, visualizations, and presentations for both technical and non-technical audiences. Mentoring junior team members and staying current with advances in NLP, LLMs, and GenAI. Essential Skills A Bachelor's More ❯

Job Title: Lead Data Engineer Industry: Banking and Financial Services Location : Flexible- Basingstoke (3 Days a Week) Overview of Role: This role will lead the design and optimisation of our Azure Databricks data platform , ensuring it is secure, scalable, and well governed. You'll take ownership of change controls, lineage, and documentation while driving adoption of … best practice. Alongside this, you'll support AI/ML initiatives and provide technical guidance to colleagues, driving value from data. Key Responsibilities Lead the roadmap for data engineering, ensuring alignment with business needs and regulatory requirements. Own the design and optimisation of the Azure Databricks platform, focusing on performance, resilience, and compliance. Implement change controls, lineage, and … observability to detect and resolve issues. Maintain clear documentation of data pipelines, models, and governance processes. Implement tools to improve data quality, efficiency, and adoption. Support AI/ML delivery by building/optimising, ensuring platform readiness. Manage SQL workloads within Databricks, balancing cost and performance Ensure access controls, GDPR compliance, and security standards Work with More ❯

the development and implementation of robust Lab/digital infrastructure within our R&D laboratories for the UK and US locations. This is pivotal in enabling efficient, automated data exchange between laboratory instruments and enterprise informatics systems such as Electronic Lab notebooks (ELN)s, Laboratory Information Management Systems (LIMS) and other relevant data setups. The ideal … implement, and maintain secure, scalable IT/OT interfaces between lab instruments (e.g., bioreactors, plate readers, liquid handlers) and informatics systems (ELN, LIMS, SDMS, etc.) Collaborate with IT, Data science and scientific teams in R&D to identify integration needs and define architecture for real-time or scheduled automated data transfer. Develop or configure middleware, APIs … and custom connectorseither in-house or in collaboration with external vendorsto enable automated data capture, contextualization, and storage. Manage instrument network connectivity (e.g., OPC-UA, MQTT, REST APIs, TCP/IP) and data pipelines from edge devices to cloud/on-premise environments. Support system validation efforts (e.g., CSV, 21 CFR Part 11 compliance) in partnership More ❯

depending on experience) Employment Type: Full-Time, Permanent Overview A dynamic consultancy is seeking a Senior Analytics Engineer to lead the development of scalable data solutions across a range of high-impact projects. This role offers the opportunity to work at the intersection of data engineering, analytics, and software development, contributing to innovative products and services … that drive real business value. Key Responsibilities Design and implement robust data pipelines using tools such as dbt and Airflow . Collaborate with clients to understand business challenges and deliver tailored data solutions. Transform raw data into structured, actionable datasets for use in AI , predictive modelling , and platform development . Promote engineering best practices … including automation, testing, and code quality. Mentor junior team members and lead technical workstreams to elevate team capability. Essential Skills & Experience Proven experience in analytics engineering , data engineering , or model development roles. Advanced proficiency in Python (including libraries such as NumPy and Pandas) and SQL . Demonstrated ability to deliver production-ready solutions in a commercial environment. Strong More ❯

Job Title: Machine Learning Engineer (Data/ML) Location: London/Hybrid - Once a week (Thursday) in London Duration: 6 Months Day Rate: £500 per day overall assignment rate to umbrella. This is a greenfield based project with crossover from another tech team conducting a similar model. You will collaborate with the Data Scientist, Data … cross-functional Tech Teams , supporting the build phase while contributing hands-on technical expertise. Key Responsibilities: Work with the Senior Engineer/Tech Lead to implement and optimise data pipelines and ML models. Develop high-quality, maintainable code in Python and PySpark. Use SQL and Jupyter to analyse, transform and validate data for ML use cases. … technical teams to deliver solutions that support business growth. Skills & Experience Required: Strong hands-on experience in Python, PySpark, SQL, Jupyter . Experience in Machine Learning engineering or data-focused development. Exposure to working in cloud platforms (AWS/Azure) . Ability to collaborate effectively with senior engineers, data scientists, and architects. Proactive, detail-oriented, and More ❯

making process, with the founders having had firsthand experience of the lengthiness and challenges of deal processes in the past. Their stack spans across back end, front end, data and crucially AI.This is a great opportunity for an entrepreneurial software engineer who wants to play a part in shaping the technical vision of this business and work on … their product from an early stage. What you'll work on: Backend APIs (Python/FastAPI): Build and maintain secure, high-performance services that drive AI features and data access at scale. RAG & vector search: Design and improve retrieval pipelines (embeddings, chunking, hybrid search, ranking, feedback loops), owning schema design, latency, and relevance across vector databases. LLM integration … Connect and orchestrate large language models (OpenAI, Bedrock, etc.), manage prompts, tools, safeguards, and evaluation. Data pipelines: Ingest, clean, and transform structured and unstructured data; design efficient schemas (Postgres/NoSQL) for search and analytics. Frontend (React/Next.js): Deliver user-friendly, performant UIs that make AI-powered features (search, filters, explanations, citations) clear and accessible. More ❯

solutions, collaborating with multidisciplinary teams to deliver secure, scalable, and innovative technology. Key Responsibilities Design, develop, and deploy AI/ML models for real-world applications. Collaborate with data scientists, engineers, and stakeholders to deliver robust solutions. Ensure all work complies with security and governance standards. Contribute to the development of AI best practices and frameworks. Essential Skills … Active SC Clearance (must be current) Strong experience in AI/ML engineering (Python, TensorFlow, PyTorch, etc.) Experience with cloud platforms (AWS, Azure, or GCP) Solid understanding of data pipelines, APIs, and MLOps Excellent problem-solving and communication skills More ❯

most dynamic and complex markets in the world. This is a chance to build high-performance, low-latency, and cloud-native systems while tackling challenges in fraud detection, data engineering, and market integrity . What you'll do Design, implement, and optimise algorithms for detecting manipulation, fraud, and compliance breaches in real-time and batch data. Build and … scale high-volume data pipelines handling billions of daily events across distributed systems. Work with cutting-edge tech including Java, Kafka, Spark, Kubernetes, Clickhouse, Snowflake, Redis . Collaborate with quants, data scientists, and compliance experts to improve surveillance strategies. Contribute to system architecture for low-latency monitoring and high-throughput processing . Continuously improve data quality, reliability, and query optimisation at scale. What we're looking for 5+ years' experience in software engineering , ideally in financial markets, market surveillance, or large-scale data systems . Strong skills in Java (or similar: Kotlin, C#, C++) ; Python is a plus. Proven experience with real-time, low-latency systems and distributed data pipelines More ❯

scalable APIs (Flask/FastAPI) for internal and external tools. Partner with business users to gather requirements and iterate quickly on solutions. Implement testing frameworks, version control, and data validation processes. Integrate with market data, storage systems, and reporting platforms. Required Skills 5+ years' Python development experience, ideally within trading, risk, or financial services. Strong background … with Pandas, NumPy, and data science libraries . Hands-on experience with Flask or FastAPI for API development. Proficiency in time-series data manipulation and back-testing methods. Experience working with large datasets and real-time data pipelines . Strong performance optimisation and debugging skills. Confident communicator with the ability to liaise directly with … business stakeholders. Desirable Skills Experience in commodities indices, structured products, or index engineering . Knowledge of market data feeds (Bloomberg, Refinitiv, etc). Familiarity with containerisation (Docker) and cloud platforms (AWS/GCP). Database experience (SQL/NoSQL) and use of modern data formats (Parquet, HDF5). Contract: Initial term with possible extension Day Rate More ❯