726 to 750 of 1,241 Data Pipeline Jobs in the UK

to maximize performance while maintaining cost efficiency Governance and Versioning Systems : Establish best practices for model versioning, reproducibility, and responsible AI deployment Integrated Data Pipelines : Utilize Databricks to construct and manage sophisticated data and ML pipelines Monitoring Ecosystems : Implement comprehensive monitoring systems to ensure reliability and … Azure) and managed AI/ML services Practical experience with Docker, Kubernetes, and container orchestration Databricks expertise, including ML workflows and data pipeline integration Familiarity with MLflow, DVC, Prometheus, and Grafana for versioning and monitoring Bachelor's or Master's degree in Computer Science, Engineering, or related … enjoy mentoring junior team members and elevating the entire technical organization What Makes This Opportunity Special You'll be working with a modern data stack designed to process large-scale information, automate analysis pipelines, and integrate seamlessly with AI-driven workflows. This is your chance to make a More ❯

directed, action-oriented, and comfortable making high-impact decisions quickly. Your focus will include consulting on new product developments, refining complex real-time data pipelines handling millions of data points, and driving operational excellence across our SaaS platform. Key Responsibilities: Production Ownership: Manage and optimise our … integrating these into the existing production infrastructure with minimal disruption. Performance Optimisation: Continuously monitor, benchmark, and enhance system performance and reliability. Data Pipeline Management: Oversee and improve our extensive real-time data ingestion pipelines, ensuring they are resilient, scalable, and future proof. Process Innovation: Identify … startup or scale-up businesses is highly desirable. Deep technical expertise in: Databases such as MySQL, Singlestore, and BigQuery. Real-time, high-throughput data pipelines. Cloud platforms, particularly Google Cloud. Strong understanding of software engineering best practices, feature deployment processes and testing. Ability to drive rapid decision-making More ❯

BI Developer. This is a unique opportunity to join an industry leader that has invested heavily into the latest Microsoft technologies to drive data-led decision making and deliver automated insights across the business. You'll play a key role in building and optimising ETL/data pipelines, developing modern reporting solutions, and shaping how data is used across the organisation. Key Responsibilities: Build and optimise ETL processes using Microsoft SQL Server. Develop and maintain data pipelines feeding into data lakes and warehouses. Design and deliver dashboards and reports … to understand and deliver data requirements. Explore AI-powered enhancements for automated insights and report generation. Required Experience: Demonstrable experience in ETL pipeline development and data modelling. Strong SQL Server skills, including performance tuning and stored procedures. Hands-on experience with Power BI, SSRS, and More ❯

want to hear from you if you are experienced with taking LLM models into production and have any exposure to using large behavioural data . Our client is a cutting-edge technology startup redefining authentication through groundbreaking behavioural biometrics and a new field of AI . Due to … hands-on role. As the most Senior Machine Learning Engineer, you will be instrumental in managing the entire lifecycle of ML models—from data exploration and development to deployment, monitoring, and ongoing enhancement. We are looking for candidates with expertise in sensor-based data , large behaviour … data , or behavioural biometrics. Ideal applicants will have experience analysing and interpreting complex behavioural data to drive insights and innovation. You will be a skilled professional with a strong history of turning prototypes into robust, production-ready solutions that drive meaningful impact. Reporting to the Chief More ❯

IDEXX is looking for a Senior Data Engineer to join the Veterinary Software team in London! This is an exciting opportunity for a a data engineer looking to expand their technical leadership skills and make an impact in a meaningful industry. This role will be hybrid … days onsite in our Wimbledon office, London. As a Senior Data Engineer, you will lead data migration strategies and implementation across hybrid cloud environments (primarily AWS), enabling smooth and secure movement of legacy and modern systems. As a senior engineer, you'll design, develop, and optimise … scalable and efficient data pipelines integrating PHP, and C applications using .NET framework backend services and React frontends. You'll utilise tools such as Terraform for infrastructure-as-code ( IaC ), AWS (Lambda, EC2, EKS, Step Functions, VPC etc.) for ETL, Airflow pipelines, Snowflake, and ensure architectural alignment with More ❯

leading mid-market funds. With this investment, the business is going through a digital transformation strategy and is now looking to drive a data-focused culture across the organisation. As the first data hire, you will work closely with the CFO and CTO to help navigate … and develop the data roadmap in both the short term and long term. You will serve as the hands-on expert responsible for optimising data structures, performing cleansing from diverse sources to enable concise reporting in Tableau, and constructing as well as maintaining pipelines. Role: As … the technical expert, you will be responsible for creating a transformation layer mapping raw data to data marts to support analysis. You will have the opportunity to work closely with the executives, including the CFO and CTO as well as the PE Fund. You will be More ❯

leading mid-market funds. With this investment, the business is going through a digital transformation strategy and is now looking to drive a data-focused culture across the organisation. As the first data hire, you will work closely with the CFO and CTO to help navigate … and develop the data roadmap in both the short term and long term. You will serve as the hands-on expert responsible for optimising data structures, performing cleansing from diverse sources to enable concise reporting in Tableau, and constructing as well as maintaining pipelines. Role: As … the technical expert, you will be responsible for creating a transformation layer mapping raw data to data marts to support analysis. You will have the opportunity to work closely with the executives, including the CFO and CTO as well as the PE Fund. You will be More ❯

tool for drug discovery. Our patient data, biobank and biomarker technology have led to the creation of an in-house drug discovery pipeline for cancers with high unmet need. We are a VC-backed start-up based in London, England and Natick, Massachusetts. To grow our company … we’re looking for a Senior Data Scientist with experience in biotech to join our early-stage team. Will you be the one? Job Description This role offers an opportunity to apply your expertise in data science, machine learning, and statistical modeling to oncology-focused target … identification and drug discovery . You will work with diverse datasets, from different sources including genomics, transcriptomics, proteomics, and high-content imaging data, to support the identification of cancer targets and the development of novel therapeutics. You will be part of the Software Team and support wet-lab More ❯

tool for drug discovery. Our patient data, biobank and biomarker technology have led to the creation of an in-house drug discovery pipeline for cancers with high unmet need. We are a VC-backed start-up based in London, England and Natick, Massachusetts. To grow our company … we’re looking for a Senior Data Scientist with experience in biotech to join our early-stage team. Will you be the one? Job Description This role offers an opportunity to apply your expertise in data science, machine learning, and statistical modeling to oncology-focused target … identification and drug discovery . You will work with diverse datasets, from different sources including genomics, transcriptomics, proteomics, and high-content imaging data, to support the identification of cancer targets and the development of novel therapeutics. You will be part of the Software Team and support wet-lab More ❯

projects Help shape the data roadmap from early foundations to advanced maturity. What you'll bring: Strong SQL and data pipeline understanding (you will work with data engineering teams rather than being hands on coding pipelines). Proven experience building dashboards and defining … KPIs. Comfortable with data flows, table mapping, and metric definitions. Stakeholder confidence – you’ll work with both editorial teams and global tech leads. Experience in media, publishing, or mobile product analytics preferred. Ability to manage ambiguity and thrive in a fast-changing environment. Why apply? High-impact, high More ❯

projects Help shape the data roadmap from early foundations to advanced maturity. What you'll bring: Strong SQL and data pipeline understanding (you will work with data engineering teams rather than being hands on coding pipelines). Proven experience building dashboards and defining … KPIs. Comfortable with data flows, table mapping, and metric definitions. Stakeholder confidence – you’ll work with both editorial teams and global tech leads. Experience in media, publishing, or mobile product analytics preferred. Ability to manage ambiguity and thrive in a fast-changing environment. Why apply? High-impact, high More ❯

Job Title Senior Data Engineer (Ms Fabric) Location Hybrid (Manchester ) Salary Up to £70K + Benefits + Bonus Job Type Perm UK ONLY Please note we cannot offer sponsorship for this position A fast-growing technology-driven organisation is looking for a Senior Data Engineer to … play a key role in delivering multiple data and cloud projects . As they expand their engineering capability , they are seeking a technical expert with strong problem-solving skills and a deep understanding of data architecture, engineering, and governance . Why Join? Work on cutting-edge … data and cloud projects in a fast-paced environment. Develop expertise across Data Architecture, Solution Architecture, and Data Governance . Excellent career progression opportunities and a collaborative team culture. Key Responsibilities Design, build, and maintain scalable data solutions to support business objectives. More ❯

I'm seeking a Data Engineer with extensive hands-on experience in Databricks, Azure Data Factory, Python, and DBT. The suitable candidate will develop data models, troubleshoot pipelines, and deliver production-ready code. Effective communication skills and the ability to articulate technical decisions clearly … Responsibilities: Develop and maintain SQL models and macros using DBT Collaborate with stakeholders to gather and refine modelling requirements Debug and enhance existing data models Design and implement ETL/ELT workflows and data pipelines using Azure Data Factory Automate ETL workflows to support … machine learning features Apply best practices for data governance, quality, and performance Utilise Azure Databricks and adhere to code-based deployment practices Essential Skills: Over 3 years of experience with Databricks (including Lakehouse, Delta Lake, PySpark, Spark SQL) Strong proficiency in SQL with 5+ years of experience Extensive More ❯

platforms. We primarily work with central government agencies and adhere to the Government Digital Service standard. We are looking for a product-focused Data Scientist to join our engineering team, passionate about delivering high-quality data-driven solutions. Role Overview: As a Data Scientist … at Zaizi, you will be responsible for analysing data to find and bring value to our clients. This involves a mixture of business acumen and technical expertise in data science disciplines. You will be digging through data to deliver data-driven, machine … things done" attitude. Develop scrappy exploratory code and code that adheres to best practices and production-ready standards. Design and implement workflows. Optimise data pipelines to improve clients' performance and scalability whilst maintaining their data integrity. Monitor and ensure observability of deployed solutions. Collaborate with technical More ❯

JobDescription: Areyou passionate about Data and Analytics (D&A) and excited abouthow it can completely transform the way an enterprise works? Do youhave the strategic vision, technical expertise, and leadershipskills to drive data-driven solutions? Do you want to work in adynamic, fast-growing category? If … so, you might be the idealcandidate for the role in the Data and Analytics function forGlobal Pet Nutrition (PN) atMars. Pet Nutrition (PN) isthe most vibrant category in the FMCG sector. As we work totransform this exciting category, a new program, Digital First, hasbeen mobilized by the Mars … thought leadership, quality decisionmaking, deep technical know-how, and an ability to navigate complexbusiness challenges while building and leading a team of worldclass data and analyticsleaders. With Digital First,PN is moving to a Product based model to create business facingdigital capabilities. Develop and maintain robust data More ❯

join our consulting practice. In this role, you will act as a strategic advisor to clients, leading the design and delivery of advanced data solutions leveraging the Databricks Lakehouse Platform. You will partner with business and technical stakeholders to modernize data ecosystems, enabling data … Strategy: Serve as the primary Databricks expert in client engagements, advising on architecture, best practices, and use cases. Lead discovery sessions to assess data maturity and define tailored data modernisation strategies. Translate business requirements into scalable technical architectures and roadmaps using Databricks and cloud technologies. Solution … Leadership: Design and oversee implementation of data platforms and AI/ML solutions on Databricks. Drive end-to-end project delivery in collaboration with cross-functional teams (data engineering, cloud, analytics). Ensure solution quality, performance, and governance through robust architecture design and code reviews. Team More ❯

join our consulting practice. In this role, you will act as a strategic advisor to clients, leading the design and delivery of advanced data solutions leveraging the Databricks Lakehouse Platform. You will partner with business and technical stakeholders to modernize data ecosystems, enabling data … Strategy: Serve as the primary Databricks expert in client engagements, advising on architecture, best practices, and use cases. Lead discovery sessions to assess data maturity and define tailored data modernisation strategies. Translate business requirements into scalable technical architectures and roadmaps using Databricks and cloud technologies. Solution … Leadership: Design and oversee implementation of data platforms and AI/ML solutions on Databricks. Drive end-to-end project delivery in collaboration with cross-functional teams (data engineering, cloud, analytics). Ensure solution quality, performance, and governance through robust architecture design and code reviews. Team More ❯

Contract Data Engineers (x3) – Defence Sector – Bristol Inside IR35 | DV Clearance Required or Willing to Undergo Location: Bristol (Onsite/Hybrid) | Contract Duration: 6-12 months+ We are recruiting three experienced Data Engineers for a leading defence client in Bristol. This is a contract opportunity operating … hold UK DV Clearance , or be eligible and willing to undergo the vetting process. Key Responsibilities: Design, develop, and manage complex, large-scale data pipelines in secure or regulated environments Ingest, process, index, and visualise data using the Elastic Stack (Elasticsearch, Logstash, Kibana) Build and maintain … robust data flows with Apache NiFi Implement best practices for handling sensitive data, including encryption, anonymisation, and access control Monitor and troubleshoot real-time data pipelines to ensure high performance and reliability Write efficient, secure scripts and code using Python, Bash, or similar languages More ❯

Machine Learning Engineer, JP Science and Data Job ID: Amazon Japan G.K. - A43 The JP Retail Science team is looking for a Machine Learning Engineer to expand our efforts in vendor tooling. To help vendors grow their business, Amazon provides multiple programs such as deals, ads, etc. In … scientists to design and develop scalable ML infrastructure that supports model training, deployment, and monitoring across hundreds of vendors Implement efficient data pipeline and architectures that enable automated ML workflows for our eCommerce partners Build ML debugging and analysis tools to ensure model reliability and performance Utilizing … technical roadmap for vendor growth tooling at AMZ. About the team In this position, you will be part of the JP Science and Data team, consisting of scientists, business intelligence engineers, data engineers, and machine learning engineers, collaborating with Product Managers and Software Developers worldwide. Our More ❯

is a dynamic digital agency based in the heart of London. As we continue to grow, we're looking for a passionate AI & Data Lead to join our team. About the Role: We are seeking a highly motivated and experienced AI & Data Lead to spearhead our … agency's adoption and application of artificial intelligence and data-driven strategies. This is a pivotal role that will shape our future direction, driving innovation and enhancing our creative output. The ideal candidate will be a strategic thinker with a deep understanding of AI and data … a passion for creative problem-solving and a strong commercial acumen. Key Responsibilities: Strategy Development: Develop and implement the agency's AI and data strategy, aligning it with business objectives and client needs. This includes identifying opportunities for AI and data integration across all agency functions More ❯

Data Engineer – Azure/C#/ETL – (£600 - £700 Per Day) – Outside IR35 – Remote Overview: An exciting opportunity has emerged with a well-established financial services organisation for a skilled Data Engineer to join a greenfield project. This role focuses on building scalable data … cloud-based solutions using Microsoft Azure technologies. You’ll work independently in a fully remote setup, supporting the design and implementation of modern data platforms using the latest Azure services. Key Responsibilities: Design and develop robust data pipelines and workflows using Azure Data Factory … Develop and manage ETL processes across various Azure components Build and optimize Azure SQL databases and Function Apps for data transformation Collaborate with developers using .NET/C# for data integration and automation tasks Leverage Excel and Access VBA for legacy data manipulation and More ❯

Lead Data Engineer ( Databricks ) London - Hybrid - Remote Permanent £100,000 - £130,000 plus up to 20% bonus based on performance and commercial contribution About the Role We’re looking for a Lead Data Engineer to spearhead some of our clients most strategic Databricks engagements. This is … build capacity for future growth. What You’ll Be Doing Act as the technical lead on client engagements, owning design and delivery of data solutions in Databricks. Architect robust, scalable data platforms using the medallion architecture. Translate business requirements into scalable workflows, advising on data governance, quality, and security. Design and implement complex data pipelines using tools like Delta Live Tables (DLT) and Unity Catalog. Guide teams in implementing best practices across engineering, DevOps, and model deployment. Support pre-sales activity, including shaping proposals, estimates, and technical roadmaps. Provide technical leadership More ❯

Lead Data Engineer ( Databricks ) London - Hybrid - Remote Permanent £100,000 - £130,000 plus up to 20% bonus based on performance and commercial contribution About the Role We’re looking for a Lead Data Engineer to spearhead some of our clients most strategic Databricks engagements. This is … build capacity for future growth. What You’ll Be Doing Act as the technical lead on client engagements, owning design and delivery of data solutions in Databricks. Architect robust, scalable data platforms using the medallion architecture. Translate business requirements into scalable workflows, advising on data governance, quality, and security. Design and implement complex data pipelines using tools like Delta Live Tables (DLT) and Unity Catalog. Guide teams in implementing best practices across engineering, DevOps, and model deployment. Support pre-sales activity, including shaping proposals, estimates, and technical roadmaps. Provide technical leadership More ❯

Techyard is supporting a growing Databricks-partnered consultancy in securing a Databricks Champion Solution Architect to lead the design and delivery of advanced data and AI solutions on the Databricks Lakehouse Platform. This strategic, high-impact role is ideal for a seasoned professional who can operate as both … bridging business vision with technical excellence. You’ll play a pivotal part in driving innovation, embedding best practices, and shaping the consultancy’s data engineering capabilities as they scale their team and client base. Key Responsibilities: Architect and implement end-to-end, scalable data and AI … solutions using the Databricks Lakehouse (Delta Lake, Unity Catalog, MLflow). Design and lead the development of modular, high-performance data pipelines using Apache Spark and PySpark. Champion the adoption of Lakehouse architecture (bronze/silver/gold layers) to ensure scalable, governed data platforms. Collaborate More ❯

Amazon Last Mile DSP is seeking an extraordinary Data Engineer to join the DSP Analytics team. At Amazon, we are working to be the most customer-centric company on earth including how we fulfill and deliver customer orders. The goal of Amazon's Service Partner (DSP) Team is … candidate we are looking for! You should have deep expertise in creating, managing, and utilizing large and small datasets across a variety of data platforms. You should have excellent business and interpersonal skills to work with business owners, understand their data requirements, and build scalable and … robust data solutions. You should be an expert at crafting, implementing, and operating stable, scalable, and cost-effective solutions to manage data flow from multiple production systems. Above all, you should be passionate about working with data and enjoy bringing varied datasets together to More ❯