251 to 275 of 315 Data Pipeline Jobs in the UK excluding London

Responsibilities: Design, implement, and maintain data pipelines to ingest and process OpenShift telemetry (metrics, logs, traces) at scale. Stream OpenShift telemetry via Kafka (producers, topics, schemas) and build resilient consumer services for transformation and enrichment. Engineer data models and routing for multi-tenant observability; ensure lineage … proactive insights). Implement schema management (Avro/Protobuf), governance, and versioning for telemetry events. Build automated validation, replay, and backfill mechanisms for data reliability and recovery. Instrument services with OpenTelemetry; standardize tracing, metrics, and structured logging across platforms. Use LLMs to enhance observability capabilities (e.g., query assistance ...

have the opportunity to work on strategy. You'll be: Designing and building internal and client-facing tools used across marketing teams Developing data-driven applications on top of cloud data platforms Working closely with analytics teams and senior stakeholders to prioritise what to build next … Creating automation and AI-powered workflows to support campaign management and optimisation Improving data pipelines, models, and performance monitoring used by PPC, SEO, and content teams You'll be part of a small, highly impactful tech group where your work directly shapes how the business operates. Tech Environment ...

Umbrella. Working in a Hybrid Model of 2 days a week on site. What You'll Do Contribute to the development of data pipelines, APIs, and services that power the AI Control Tower. Implement components supporting AI observability, guardrails, performance monitoring, and lifecycle controls. Develop integrations with model … registries, feature stores, lineage tools, and governance systems. Write clean, well-tested, scalable code in Python, Java, SQL, and modern data/stream processing frameworks. Build high-throughput pipelines to capture metrics such as: Model performance, drift, and degradation Operational and service health Security posture and policy adherence ...

Months About the Role: Design, develop, and maintain scalable microservices using Java and Spring Boot. Build and optimize real-time data pipelines leveraging Apache Kafka, Flink, and Spark/Databricks. Develop robust data distribution and streaming solutions for high-throughput systems. Deploy, manage, and monitor services … Spring Boot. Hands-on experience with Microservices architecture and RESTful APIs. Proficiency with Kafka and distributed streaming systems. Solid understanding of SQL and data modeling. Experience with containerization (Docker) and orchestration (Kubernetes). Working knowledge of Flink, Spark, or Databricks for data processing. Familiarity with ...

Bournemouth, UK Job Description:- About the Role: Design, develop, and maintain scalable microservices using Java and Spring Boot. Build and optimize real-time data pipelines leveraging Apache Kafka, Flink, and Spark/Databricks. Develop robust data distribution and streaming solutions for high-throughput systems. Deploy, manage … Spring Boot. Hands-on experience with Microservices architecture and RESTful APIs. Proficiency with Kafka and distributed streaming systems. Solid understanding of SQL and data modeling. Experience with containerization (Docker) and orchestration (Kubernetes). Working knowledge of Flink, Spark, or Databricks for data processing. Familiarity with ...

platform. Work closely with product managers to prioritise features, shape the roadmap, and ensure delivery against ambitious timeline Develop scalable, high-throughput data pipelines and monitoring systems to track: AI model performance and drift Operational resilience and service health Security posture and policy adherence Guardrail compliance … secure-by-design principles, strong IAM practices, and regulatory compliance into the platform architecture. Ensure all AI systems are Partner with SMEs across data engineering, model risk, security, IMM (Independent Model Monitoring), and platform engineering. Partner with SMEs across data engineering, platform engineering, security, risk ...

Senior Data Engineer Location: London (2 Days in Office) Salary: £80,000 Join a growing, people-first organisation where you'll shape a modern Databricks Lakehouse platform, lead impactful data initiatives across the business, and enjoy true flexibility, autonomy and career growth. The Opportunity This … with clear purpose, strong leadership and a genuinely collaborative culture. The Role and Responsibility The role focuses on designing, building and maintaining scalable data pipelines and products on a modern Databricks Lakehouse environment, using Delta Lake, Spark, Python (PySpark) and SQL to deliver high-quality, well-governed data ...

Data Architect - 6-Month Rolling Contract Onsite: Warwick, 2 days per week Rate: (Inside IR35)This role requires strong hands-on capability across modern cloud data tooling, with a particular focus on Azure and Snowflake.Key Skills Required* Azure data ecosystem* Snowflake* Prometheus* Azure Data …/ELT engineeringWhat They're Looking ForA contractor who can operate autonomously, embed quickly into an enterprise environment, and deliver robust, production-ready data pipelines. Someone who can work across cloud, orchestration, monitoring, and transformation layers with confidence.Engagement Details* Inside IR35* 6-month rolling contract (expected long-term ...

Location: Remote Reports to: CTO A fast-growing technology company is seeking a Head of Engineering with deep experience building and scaling complex, data-driven platforms. This role requires someone who can lead multi-disciplinary teams, improve engineering execution, and drive technical excellence across high-scale distributed systems … machine-learning environments. Required Experience Leadership of engineering organisations of 20 or more across multiple disciplines, including backend, web, data, ML, QA, and DevOps. Proven track record designing, scaling, and maintaining distributed systems and high-volume data pipelines. Strong background in cloud-native architectures and modern ...

United Kingdom (Hybrid working) Why Apply? This is an opportunity to shape how enterprise machine learning is delivered at scale within a modern data environment. Working on a strategic Lakehouse platform built on Databricks, you will influence how production ML models are designed, governed and optimised, helping … business turn complex data into reliable, decision-driving insight. The role combines hands-on engineering with MLOps leadership, strong stakeholder engagement and long-term platform thinking. Responsibilities * Lead the design, build, deployment and monitoring of production machine learning models using Databricks, ensuring performance, reliability and continuous improvement * Define ...

Contract Opportunity for a multiple Security Cleared Data Scientists with an Industry Leading National Security Consultancy in Cheltenham. Active eDV (West) Required Up to £850 p/d Inside IR35 6-12 month engagement Urgent requirement - ASAP start 3-4 days on site per week in Cheltenham … Role As a Data Scientist, you’ll work within a multidisciplinary team to extract insight from complex, high-volume datasets. You’ll design and implement advanced analytical and machine learning models, develop robust data pipelines, and deliver actionable intelligence to inform decision-making at the highest ...

Data Engineer - AWS/Glue - £425 outside IR35 - Edinburgh Exalto consulting are currently recruiting for a contract Data Engineer, this is mostly remote, 1 day a month in Edinburgh, this is outside IR35 paying £425 per day. Key responsibilities Design, build and maintain data pipelines on AWS Develop and manage ETL processes using AWS Glue Work with structured and semi-structured data to support analytics and reporting Collaborate with engineers, analysts and stakeholders across the business Skills & experience Strong experience as a Data Engineer Hands-on AWS experience ...

Data Engineer Edinburgh Hybrid 2 days onsite Im on the lookout for an experienced Data Engineer who can help build and scale a robust, multi-tenant data platform. This role focuses on designing, building, and maintaining reliable data pipelines using Python ...

Sheffield (3 days on-site per week) Candidates must be eligible for BPSS clearance We're looking for a highly skilled Senior Data & Observability Engineer to design, build, and evolve large-scale telemetry and observability capabilities across modern cloud platforms. This role is ideal for someone who thrives … high-volume, real-time data environments and enjoys solving complex engineering challenges. What You'll Do Design, implement, and maintain scalable data pipelines to ingest and process telemetry (metrics, logs, traces) from OpenShift and Kubernetes platforms. Stream telemetry via Kafka-managing producers, topics, schemas, and building ...

responsible for: Delivering strategic advisory, engineering delivery, and enablement services to Government and commercial clients. Solve user problems by drawing source and captured data into a single foundational data model and building operational workflows that materially change the decisions made Work closely with clients, to tailor … products and services Design solutions for client problem sets based on their descriptions and by exploration of the data landscape Assist in the design, development, and maintenance of data pipelines and ETL processes to build data and action models to address the workflow needs ...

behalf a large healthcare consultancy, partnering with some of the worlds most ambitious life sciences, biotech, and healthcare organisations to solve complex, data-driven challenges at scale. Our work directly contributes to improving patient outcomes, accelerating research, and enabling smarter, more efficient healthcare systems. We are proud … Pinecone and Weaviate . Build multi-modal workflows spanning text, voice, and video, with a strong focus on accuracy, explainability, and performance. Design data ingestion and enrichment pipelines that meet healthcare-grade quality and compliance expectations. Drive Engineering Excellence Build scalable microservices and platforms using Python and/ ...

Opportunity Overview The KTP aims to transform CARA_EPS into a data-driven retrofit company, positioning them the preferred partner for housing associations in their retrofit programs. The project aims to develop an integrated digital asset management (DAM) platform to address issues of unreliable property data and poor housebuilding procurement practices, which are common within the industry. Key Responsibilities Key responsibilities include enhancing retrofit assessment data transparency and accessibility, forecasting property energy performance, and decision-making in housing stock refurbishment. The platform will enable early verification of property/project specifications ...

works. You will help us automate manual tasks, connect AI agents to our CRM and ERP systems, strengthen customer interactions with data-driven “proof packages,” and explore new AI-enabled capabilities that strengthen our position as a UK leader in custom battery technology. This role is ideal … documentation, production forecasting). Create AI workflows that integrate with our CRM and ERP systems (including IFS Cloud), enabling smarter insights, rapid data retrieval, and improved traceability. Maintain AI agents that support internal teams, including text-analysis bots, report generators, and knowledge-retrieval tools. Data engineering ...

Augmented Generation (RAG) AI agents and orchestration workflows Designing and maintaining backend services and APIs (primarily Python, FastAPI/Flask) Building and scaling data pipelines and search systems using: ElasticSearch, Weaviate, Pinecone, and other vector databases Engineering secure, containerised architectures using Docker and Kubernetes Ensuring high availability, fault … Experience in defence, government, healthcare, or other regulated domains Familiarity with DevOps practices, CI/CD pipelines, and infrastructure as code Exposure to data engineering, ETL pipelines, or high-volume data processing Experience working in SME or start-up environments , comfortable wearing multiple hats Salary ...

development of cutting-edge technologies that address future business needs. Key Responsibilities: Technical lead in designing full-cycle AI solutions, including data collection and pipelines for reusable analysis, and monitoring and maintenance regimes for continuous development. Research, develop, deploy, and maintain scalable AI-powered solutions for Cambridge English … ensure prompt delivery Deep experience in Python and PyTorch/TensorFlow. Industry experience in building scalable, production-grade AI services and automated data pipelines with tools like Docker, Kubernetes, and cloud platforms (e.g., AWS). Proven ability to collaborate effectively in Agile/Scrum teams and contribute ...

current hybrid working arrangement requires office attendance on Mondays, Tuesdays, and Thursdays. Summary of the Role We’re looking for a Data & Automation Engineer to join our AI team. This hybrid technical role blends coding expertise with advanced low-code orchestration to design, build, and maintain data … Head of AI, you’ll play a critical role in connecting agency systems with Generative AI models, ensuring secure, efficient, and accurate data flow. What You Can Expect to Be Doing: Design and deploy sophisticated automation workflows using tools like n8n and Dify. Develop custom scripts (Python/ ...

Title: AWS Site Reliability Engineer (Data Platform) Role Summary We are looking for an AWS Site Reliability Engineer (SRE) to support and scale a cloud-native data platform built on AWS, Snowflake, and Databricks . The role focuses on driving reliability through automation, disaster recovery …/CD. Lead resiliency and disaster recovery planning , including regular DR drills, failure testing, and recovery validation across AWS and data platform components. Define, implement, and manage SLIs, SLOs, and SLAs for critical data pipelines and platform services; use error budgets to guide reliability improvements. Build ...

leading provider of catering and support services to the education sector in Oxford are seeking a talented and detail-oriented Junior Data Analyst to join their team. KEY DUTIES Collect, clean, and analyse large datasets to uncover trends, patterns, and insights. Develop and maintain data pipelines … workflows to ensure data integrity and accessibility. Design and implement data models and visualizations to communicate findings to stakeholders. Collaborate with cross-functional teams to identify business requirements and translate them into data analysis solutions. Conduct ad-hoc analyses and deep dives to support ...

integration with Zero Trust. Harden AKS clusters and cloud-native apps through role-based access control (RBAC), secrets management, and runtime protection. Secure data pipelines and models in AI workloads, implementing guardrails for model integrity and prompt injection defence. Cloud & Network Security Engineer secure access and posture … automation. Strategic, yet pragmatic: balances best practice with business context. Proactive, communicative, and passionate about elevating security posture through innovation. MS Engineer & MS Data Scientist Associate Ability to exercise independent judgment and act on it. Benefits for our U.K. teams include: The choice to work flexibly from home ...

growing our AI capabilities and we’re looking for curious problem solvers who love turning data into decisions. If you enjoy spotting patterns, learning to write code, and testing ideas with evidence, this is where you’ll make a difference. Role During this programme, you’ll work … areas such as below work as part of team to become expert-level data and AI talent: AI Adoption & Enablement – helping shape, test and improve ML models, data pipelines, or automation solutions Data & Insights – partnering with product, operations or customer facing teams to understand ...