201 to 225 of 267 Data Pipeline Jobs in the UK excluding London

guidelines for Airflow clusters and DAG's, working knowledge of performance tuning of the DAG's and task implementation, developing DAG - data pipeline to on-board and change management of datasetsExperience with installing, configuring and monitoring Airflow cluster, strong PythonUnderstanding of Airflow rest services and integration ...

helping organisations transform and deliver high-quality, scalable solutions that truly matter. Overview: We are looking for a highly skilled AWS DevOps/Data Engineer to design, build, and maintain scalable, resilient, and high-performing cloud-based data solutions. The ideal candidate will have strong DevOps … expertise combined with data engineering capabilities, working across modern cloud technologies and automation frameworks. Key Responsibilities: Translate technical requirements into effective DevOps toolchains to enable product delivery Design and implement resilient, scalable, and highly available services Automate infrastructure, deployments, and testing using Infrastructure as Code (IaC) Ensure deployment ...

etc.) with libraries/frameworks such as PyTorch, TensorFlow etc.) * Experience with RAG and LLM-based solutions * Solid understanding of AI architectures and data pipelines * Knowledge of security, compliance, and governance principles from a development perspective * Proven mentoring and team development capabilities * Excellent communication and stakeholder management skills ...

evidence to identify emerging risks, shape policy and improve decision making at a national level.Rather than focusing on building machine learning models or data pipelines, this role is centred around designing robust statistical methodologies , helping establish best practice and developing the long-term capability of the organisation ...

Staff Data Engineer/Leeds or London 1x a month/£120k - £140k Shape the Future of Data Engineering Within a Leading SaaS Product Business This is an opportunity to step into one of the most influential technical roles within a high-growth SaaS organisation where … engineering plays a central role in shaping the future of the product. Rather than focusing solely on building data pipelines, you'll provide technical leadership across the engineering function, helping define the architectural direction of the company's data platform while influencing how engineering teams solve ...

technology - serving more than 300,000 customer locations across the Globe. As a company undergoing significant business transformation, Avantor is investing in the data capabilities and operational discipline needed to drive sustainable growth. This UK/Ireland-based role offers the opportunity to shape global workforce strategy. About … Role As Senior Manager, Workforce Planning & Data Analytics, you will build and lead the enterprise workforce planning capability, ensuring Avantor's global talent strategy is anchored in rigorous data and aligned to its business transformation agenda. You will translate complex workforce dynamics across our business segments ...

architectural leadership by leading through example. Your responsibilities will include: Lead the development and implementation of a modern cloud foundation and data platform that is robust, scalable, fully automated, secure, and can support the growth of the business. Build Scalable Architectures: Leverage modern technologies to design and implement … build secure RESTful and GraphQL APIs, ensuring seamless integration with core banking systems (e.g., Mambu) and external services like Open Banking platforms. Data Engineering and Analytics: Work closely with data teams to define robust data pipelines and scalable cloud-based data platforms ...

technical leadership to a team of engineers. Your responsibilities will include: Lead the development and implementation of a modern cloud foundation and data platform that is robust, scalable, fully automated, secure, and can support the growth of the business. Build Scalable Architectures: Leverage modern technologies to design … build secure RESTful and GraphQL APIs, ensuring seamless integration with core banking systems (e.g., Mambu) and external services like Open Banking platforms. Data Engineering and Analytics: Work closely with data teams to define robust data pipelines and scalable cloud-based data platforms ...

Data Software Engineer - Azure Databricks London, United Kingdom Contract 6 to 8 years of experience in data software engineering with a strong focus on Azure technologies Expertise in Azure Databricks development, including efficient handling of large datasets Strong working knowledge of Azure Data Factory … building complex ETL workflows Experience with performance optimization of data pipelines and distributed processing setups Proficiency in Scala development (hands-on experience highly desirable) Solid understanding of Azure services relevant to data engineering and cloud architecture Strong problem-solving ability with effective communication skills for team ...

deep technical engine behind their internal AI deployment team. Where the AI Deployment Strategist scopes and builds agents, you own the infrastructure underneath data pipelines, system integrations, and everything that ensures deployed AI solutions work reliably at scale. You will take prototypes to production, ensure all business systems … storage supporting current and future AI deployments Troubleshoot integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft ...

deep technical engine behind their internal AI deployment team. Where the AI Deployment Strategist scopes and builds agents, you own the infrastructure underneath — data pipelines, system integrations, and everything that ensures deployed AI solutions work reliably at scale. You will take prototypes to production, ensure all business systems … storage supporting current and future AI deployments • Troubleshoot integration failures, data inconsistencies, and production issues Key Skills • Python & SQL (production-grade pipeline development) • REST APIs, webhooks, OAuth, event-driven architecture • Orchestration tools: Airflow, Prefect, or Dagster • Cloud platforms: AWS, GCP, or Azure • Docker & Kubernetes Microsoft 365 & Microsoft ...

Data Engineer Opportunity, DV-cleared only London Manchester Bristol A progressive, leading-edge UK consulting and technology organisation is hiring Data Engineers to deliver mission-critical work across defence and security programmes, building modern data platforms and production-grade pipelines that enable better decisions … Process large structured and unstructured datasets, integrating multiple sources Collaborate with stakeholders and delivery teams to drive outcomes Core skills (indicative) Production pipeline design and deployment experience Strong engineering capability with Python, SQL, plus big data tooling (e.g., Spark, and Java/Scala where relevant ...

real difference. Whether you are coming from insurance pricing, actuarial, financial services, consulting, retail pricing, or another analytical environment, if you enjoy turning data into commercial decisions, we would love to hear from you. Why join us? Make an impact: help determine how we price, grow, and optimise … products used by millions of customers across the UK. Work on innovative data science problems: use the right tools for the job, including Python, SQL, Power BI, and advanced analytical techniques, to solve real commercial pricing challenges. Develop your actuarial career: if you are on the actuarial path ...

real difference. Whether you are coming from insurance pricing, actuarial, financial services, consulting, retail pricing, or another analytical environment, if you enjoy turning data into commercial decisions, we would love to hear from you. Why join us? Make an impact: help determine how we price, grow, and optimise … products used by millions of customers across the UK. Work on innovative data science problems: use the right tools for the job, including Python, SQL, Power BI, and advanced analytical techniques, to solve real commercial pricing challenges. Develop your actuarial career: if you are on the actuarial path ...

from design to live production, collaborating in a small squad led by a Principal engineer. You’ll work across LLM integration, backend services, data pipelines, and frontend elements, using Claude Code daily to shape robust, production-ready AI systems. #J-18808-Ljbffr ...

executing the migration to AWS Bedrock. Utilise your strong C# background to ensure AI agents integrate seamlessly with core enterprise systems, APIs, and data pipelines. Refine prompt engineering, agent autonomy, and guardrails to ensure high performance, security, and accuracy. Proven experience building Agentic AI systems (autonomous agents, multi ...

outcomes. WHAT YOU'LL DO Architect and Scale Global AI Frameworks: Design and develop end‐to‐end AI/ML solution patterns, robust data pipelines, and model architectures that seamlessly integrate with Everpure’s core enterprise offerings. Drive Pilot‐Led Innovation: Spearhead high‐impact MVPs and Proofs … influencing services growth and high‐value deal success. Establish Domain Excellence and Governance: Define organization‐wide best practices for the AI model lifecycle, data governance, and responsible AI usage to ensure long‐term solution scalability and compliance. WHAT YOU BRING Production‐Level AI Expertise: Proven success building, deploying ...

Data Engineer needed in Lincoln Paying £27.95 per hr ref 376144 Full time hours on a temporary basis Key Responsibilities Design, develop, and maintain scalable data pipelines and data integration solutions. Build and optimise ETL/ELT processes to extract data from ...

title Data Science Engineer Job Type Contract (6 Months) Client/Industry Software Location Reading (Hybrid/Remote) Key roles and responsibilities Design and deliver a sophisticated EDU rSAM (remaining sales addressable market) model to quantify untapped revenue opportunities across global education markets. Partner cross-functionally with … deeply understand the complexities of the education sector, including diverse subscription models, institutional procurement structures, and consortium-based purchasing dynamics. Collaborate with Data Engineering teams to integrate and operationalise third-party datasets e.g. student population and institutional metrics to create rich, actionable customer intelligence profiles. Work closely with ...

have a bachelor’s degree and proven multi-platform SIEM expertise including Splunk Enterprise Security and Microsoft Sentinel, plus automation and data pipeline experience. #J-18808-Ljbffr ...

hands-on delivery and cross-functional collaboration to deliver measurable ROI. You will partner with an AI Business Analyst, applying best practices in data pipelines, AI governance, and security while guiding rapid time-to-value projects #J-18808-Ljbffr ...

fault management strategies for radiation effects, SEUs, redundancy, and safe-mode protocols. Experience with GNC/AOCS systems, including sensor and actuator data pipelines, control algorithms, and estimation filters. Exposure to digital twins, simulation environments, and SIL/HIL testing for end-to-end system validation. Experience supporting ...

nationwide site visits Rate: £500/day Outside IR35 Roc Search is recruiting for an experienced Scrum Master to join a large-scale data and analytics transformation programme. This is a fantastic opportunity to work with a highly technical team delivering modern cloud-based data solutions … fast-paced Agile environment. This is a technically-focused Scrum Master role. You'll be working closely with Data Engineers, BI Developers and technical stakeholders to drive delivery, remove blockers and ensure Agile best practices are embedded across the programme. Key Responsibilities Lead Scrum ceremonies and manage sprint ...

+ excellent benefits A leading technology-focused organisation is seeking an experienced AI Deployment Engineer to join its growing AI and data function. This is a highly technical, hands-on role forming the backbone of the company’s AI deployment capability, responsible for taking AI-driven solutions from … reliably across the business. The role is predominantly remote, with occasional on-site collaboration days in London. Key Responsibilities: Own and develop the data and integration infrastructure supporting AI deployments, including pipelines, storage and data delivery layers Design and implement robust integrations between AI solutions ...

deliver and operate production-grade capabilities that power cyber operations at scale. You will build reliable, secure and observable services, tools, platforms, or data pipelines aligned to shared engineering standards. You will own the service lifecycle, drive architectural decisions with stakeholders, and push platform maturity through automation ...