101 to 125 of 166 Data Pipeline Jobs in the South East

directly impact customers, teams, and commercial outcomes. The RoleYou will join a cross-functional AI function working at the intersection of software engineering, data, and machine learning. Your focus will be on turning business problems into production-ready AI systems, from early discovery through to deployment, monitoring … technical owner of AI-powered products and features from concept through to production* Design and deploy end-to-end AI solutions spanning data preparation, modelling, APIs, and production integration* Build and optimise ML and LLM-driven services, including prediction models, personalisation, and retrieval-augmented systems* Collaborate with business ...

Umbrella. Working in a Hybrid Model of 2 days a week on site. What You'll Do Contribute to the development of data pipelines, APIs, and services that power the AI Control Tower. Implement components supporting AI observability, guardrails, performance monitoring, and lifecycle controls. Develop integrations with model … registries, feature stores, lineage tools, and governance systems. Write clean, well-tested, scalable code in Python, Java, SQL, and modern data/stream processing frameworks. Build high-throughput pipelines to capture metrics such as: Model performance, drift, and degradation Operational and service health Security posture and policy adherence ...

machine learning systems, contributing directly to intelligent, scalable software used by customers every day. You'll work closely with experienced engineers, data scientists, and product teams to design, build, and deploy AI-driven features across our platform — from experimentation through to production. Key Responsibilities Design, develop, and deploy … machine learning and AI models within a SaaS environment Work with structured and unstructured data to build predictive and intelligent features Collaborate with software engineers to integrate models into production systems Evaluate, test, and improve model performance over time Contribute to data pipelines, model training workflows ...

models, and scalable API platforms. Key Responsibilities Own the technical strategy and execution across the company's core products, including real-time APIs, data pipelines, and LLM-powered analytics. Collaborate closely with Product and Data Science to deliver a forward-thinking technology roadmap. Lead, mentor … high-performing engineering team (currently ~10), nurturing a culture of ownership, speed, and technical excellence. Make key architectural decisions, ensuring system scalability and data infrastructure performance. Maintain high engineering standards while remaining close to critical technical choices. Recruit top talent and build a strong, inclusive team culture. Oversee ...

teams. What You’ll Be Doing AI Cloud Infrastructure Design, build, and operate highly available, secure infrastructure supporting AI inference, fine-tuning, and data processing workloads Manage multi-region Kubernetes clusters , including GPU-heavy environments Implement autoscaling strategies across heterogeneous compute fleets Infrastructure as Code & Automation … observable, automated systems Nice to Have Experience supporting GPU-based workloads or ML infrastructure Exposure to AI/ML platforms, inference systems, or data pipelines Familiarity with modern CI/CD tooling and GitOps approaches Experience with observability tooling (metrics, logs, tracing) Background in cloud platforms, AI infrastructure ...

Senior Data Engineer - Azure Databricks (Contract) Fully Remote | 3 Months (Likely to Extend) | Outside IR35 I'm supporting a client who is looking for an experienced Senior Data Engineer with strong Azure and Databricks skills to join them on an initial 3-month contract , with … high likelihood of extension. What you'll work on Building and optimising scalable data pipelines in Azure and Databricks Supporting data engineering best practices across cloud environments Collaborating with cross-functional teams to deliver high-quality data solutions What we're looking for Strong ...

Senior Data Engineer Location: London (2 Days in Office) Salary: £80,000 Join a growing, people-first organisation where you'll shape a modern Databricks Lakehouse platform, lead impactful data initiatives across the business, and enjoy true flexibility, autonomy and career growth. The Opportunity This … growing brand with clear purpose, strong leadership and a genuinely collaborative culture. The Role and Responsibility Design, build and maintain scalable data pipelines and products within a modern Databricks Lakehouse environment using Delta Lake, Spark, Python (PySpark) and SQL. Manage and optimise the cloud-based lakehouse, leveraging Unity ...

responsible for: Delivering strategic advisory, engineering delivery, and enablement services to Government and commercial clients. Solve user problems by drawing source and captured data into a single foundational data model and building operational workflows that materially change the decisions made Work closely with clients, to tailor … products and services Design solutions for client problem sets based on their descriptions and by exploration of the data landscape Assist in the design, development, and maintenance of data pipelines and ETL processes to build data and action models to address the workflow needs ...

Were seeking a Data Platform Engineer to design and build a modern enterprise data platform using Microsoft Fabric. Youll play a key role in architecting a secure, scalable and high-performance cloud environment that enables advanced analytics and data-driven decision making. This … hybrid role with 1 day a week in the Milton Keynes office. You will - Design and deliver a cloud-native data platform in Microsoft Fabric. Build and optimise data pipelines, lakehouse/warehouse solutions and semantic models. Implement Platform as Code (terraform) and CI/ ...

challenges.We are looking for a Senior DevSecOps Engineer to join the Pathogen Programme at EIT. In this role, you will help ensure our data platform is built to the highest standards, with a strong emphasis on automation across the development lifecycle. You will work closely with engineers … deploy data pipelines and platform features, and support bioinformaticians in building and deploying their workflows. You will be responsible for maintaining infrastructure, designing secure automation pipelines, managing cloud environments, and ensuring security and compliance. You will collaborate with cross-functional teams, data engineers, backend, and full ...

functionality, including automated responses, predictive analytics, and risk scoring • Integrating with OpenAI APIs and exploring in-house model training and fine-tuning • Creating data pipelines to support ML workflows, from preparation to deployment and monitoring • Writing and maintaining APIs for our React/TypeScript frontend • Contributing to frontend … work: • User stories are managed on Monday.com • Code quality is enforced through automated testing, PHPStan static analysis, and Laravel Pint formatting • CI/CD pipeline with automated deployments • CloudWatch for monitoring What we offer: • 25 days holiday + your birthday off (if it falls during the working week) • Flexible ...

Lead Data Engineer - Databricks - £75-80k + bonus + benefits - Basingstoke 3 days a week The Opportunity: We are looking for a Lead Data Engineer with good Databricks expeirence to join a Basingstoke based organisation who are investing heavily in their Digital Transformation Programme. … lead a small team whilst contributing personally - Be able to drive agile, modern engineering practises - Champion quality, observability and good engineering discipline, ensuring pipeline and models run cleanly and predictably ...

Google Cloud, or Azure. Knowledge of containerization and orchestration tools like Docker and Kubernetes is a plus. Experience designing end to end ML data pipelines. Experience with database technologies such as PostgreSQL, MySQL, or MongoDB, Spark, Databricks and stream data processing such as Kafka, RedPanda … stay up-to-date with the state-of-the-art open source infrastructure solutions applicable to designing and improving large scale recommender systems, data engineering, and machine learning You have a Master's or PhD in Computer Science, statistics, or an engineering field with 5+ years of experience ...

Summary: We are partnering with a leading global consultancy who are looking to hire a highly skilled Databricks Engineer to deliver modern, scalable data platforms for enterprise clients. This is a hands-on engineering role, focused on building, optimising, and operating Databricks-based Lakehouse solutions across cloud environments. … Responsibilities: Design, build, and maintain data pipelines using Databricks (Spark, Delta Lake). Develop batch and streaming workloads using PySpark/Spark SQL. Implement data ingestion, transformation, and orchestration pipelines. Optimise Databricks workloads for performance, reliability, and cost. Integrate Databricks with cloud-native services across Azure ...

frontend. We heavily leverage Postgres, Timescale and Redis for storage, and run across AWS, Azure and Terraform for infrastructure. Real-time data pipelines and edge computing are key pillars of the Ferry platform, where we augment Apache Flink and Kafka. Your Day to Day Build and scale … backend systems that power Ferry's core platform Design and implement performant APIs, data pipelines and integrations Collaborate with frontend, platform and data engineers to deliver end-to-end features Lead projects from design through deployment, embedding engineering best practices along the way Work directly with ...

Augmented Generation (RAG) AI agents and orchestration workflows Designing and maintaining backend services and APIs (primarily Python, FastAPI/Flask) Building and scaling data pipelines and search systems using: ElasticSearch, Weaviate, Pinecone, and other vector databases Engineering secure, containerised architectures using Docker and Kubernetes Ensuring high availability, fault … Experience in defence, government, healthcare, or other regulated domains Familiarity with DevOps practices, CI/CD pipelines, and infrastructure as code Exposure to data engineering, ETL pipelines, or high-volume data processing Experience working in SME or start-up environments , comfortable wearing multiple hats Salary ...

Product Engineer (Data/Platform/Applied AI) £65,000 - £80,000 Location: London (Hybrid – 3 days onsite) Level: Mid–Senior (roughly 2–6 years' experience) The Opportunity We're working with a fast-growing, venture-backed technology company building an enterprise data and AI platform … Product Engineer — someone who enjoys building real products, not just pipelines or one-off solutions. This role sits at the intersection of data engineering, backend engineering, and product thinking, with exposure to modern AI-enabled workflows. You'll work on problems where customer needs are complex and messy ...

Google Cloud, or Azure. Knowledge of containerization and orchestration tools like Docker and Kubernetes is a plus. Experience designing end to end ML data pipelines. Experience with database technologies such as PostgreSQL, MySQL, or MongoDB, Spark, Databricks and stream data processing such as Kafka, RedPanda … stay up-to-date with the state-of-the-art open source infrastructure solutions applicable to designing and improving large scale recommender systems, data engineering, and machine learning You have a Master's or PhD in Computer Science, statistics, or an engineering field with 5+ years of experience ...

native services Translate business problems into deployable ML solutions Develop and maintain end-to-end ML pipelines (training, testing, deployment, monitoring) Work with data scientists and engineers to operationalise models at scale Implement best practices for model performance, versioning, and lifecycle management Ensure solutions are secure, scalable … Platform Experience with services such as Vertex AI, BigQuery, Cloud Storage, and Cloud Functions/Cloud Run Solid Python experience for ML and data engineering workloads Experience productionising models (not just experimentation or notebooks) Understanding of MLOps concepts: CI/CD, monitoring, retraining, and model governance Ability ...

Timeline Month 1: Onboard to existing ML codebase and infrastructure; identify technical debt and reliability gaps; ship incremental improvements to model serving latency or pipeline robustness. Month 3: Own and deliver a major infrastructure component (e.g., feature store, training orchestration, or model registry); improve system observability with logging, metrics … fundamentals (model training, evaluation, common architectures) Desired Skills (Bonus Points) Experience with MLOps tooling (MLflow, Kubeflow, Weights & Biases, or similar) Building data pipelines (real-time or batch) using tools like Apache Spark, Kafka, Airflow, or dbt Familiarity with recommender systems, transformers, or graph neural networks Exposure to model ...

into real-world outcomes. The Opportunity Own the technical delivery and architecture of Onsera's core product platforms, from wearables integration and health data pipelines to member payments and AI workflows. As a player-coach, you'll design and implement full-stack solutions while setting engineering standards … quality-driven development Experience acting as a technical leader or player-coach with demonstrated ability to mentor engineers Comfortable collaborating with product, data, clinical, and non-technical stakeholders Preferred Qualifications Strong written and verbal communication skills for technical and cross-functional collaboration Experience working with Python Experience with ...

Senior .NET Engineer, Low Latency Systems and Cloud Platform (Crypto Data Infrastructure) Introduction As a Senior .NET Engineer, you will join a technical environment focused on high-performance systems and low-latency data pipelines. You will be part of a lean, engineering-driven team building … maintaining the core backend services that power advanced trading infrastructures. This work is part of a larger trading execution platform that includes market data ingestion, order routing, and portfolio-level services. You will operate in a horizontal team structure, where you are fully responsible for your area. From ...

pair on, and review the development work of other lead software developers. You'll work on the entire company's backend code base: data models, algorithms, data pipelines, infrastructure, APIs, complex database queries, ML models. Learn, use, and improve best practices and processes for high-quality ...

Media Solutions Engineer - £60K-£75K DOE The Business An established, independent digital consultancy working at the intersection of media, data, and technology. The company partners with major brands and public-sector organisations, helping them solve complex problems through bespoke technical solutions. The Role This is a hands … engineering role focused on building and maintaining tools used by both internal teams and external clients. You'll work across data, automation, and emerging AI-driven workflows, contributing directly to the design, delivery, and evolution of technical solutions. Responsibilities include: Designing, building, and maintaining internal and client-facing ...

firms and systematic hedge funds. These organizations operate at the absolute bleeding edge of technology, treating the global financial markets as a complex data science problem. They are deploying massive compute clusters and proprietary machine learning models to trade billions of dollars daily. If you're an engineer … compute layers, squeezing maximum efficiency out of hardware. Build World-Class Observability: Create robust monitoring and telemetry systems to provide real-time insights into pipeline health, trading activity, and model behavior. Work with the Best: Work side-by-side with world-class researchers and mathematicians to translate theoretical strategies ...