76 to 100 of 107 Data Pipeline Jobs in the South East

Cambridge, Gatwick, Milton Keynes, Reading Business Line Enabling Functions Job Type Permanent/FTC Date published 10-Jun-2025 19366 Connect to your Industry Are you passionate about data science and AI? Do you want to apply your skills and knowledge to help shape the future of Deloitte? If so, we have an exciting opportunity for you to … join our Enabling Functions team as an AI Architect & Data Scientist. You be part of an exciting innovative team that delivers cutting edge GenAI solutions. As an AI Architect & Data Scientist, you will work on innovative projects that leverage data and AI to enhance our internal capabilities and deliver value to our employees. You … values lead every decision wemake and action we take, guiding us to deliver impact how and where it mattersmost . Connect to your opportunity As an AI Architect & Data Scientist, you will be expected to: Design and build Agentic AI and GenAI solutions, from PoC to production, using agile methodologies and best practices. Apply advanced analytical techniques, such More ❯

Cambridge, Gatwick, Milton Keynes, Reading Business Line Enabling Functions Job Type Permanent/FTC Date published 10-Jun-2025 19366 Connect to your Industry Are you passionate about data science and AI? Do you want to apply your skills and knowledge to help shape the future of Deloitte? If so, we have an exciting opportunity for you to … join our Enabling Functions team as an AI Architect & Data Scientist. You be part of an exciting innovative team that delivers cutting edge GenAI solutions. As an AI Architect & Data Scientist, you will work on innovative projects that leverage data and AI to enhance our internal capabilities and deliver value to our employees. You … values lead every decision wemake and action we take, guiding us to deliver impact how and where it mattersmost . Connect to your opportunity As an AI Architect & Data Scientist, you will be expected to: Design and build Agentic AI and GenAI solutions, from PoC to production, using agile methodologies and best practices. Apply advanced analytical techniques, such More ❯

We're working with a growing music & royalties business that provides services across royalty data, rights management, audit, and transactional services. This is an exciting opportunity to join a specialist team in a collaborative, agile environment and gain hands-on experience in the music space! Responsibilities: Build, maintain, and optimise data pipelines for ingestion, transformation, and … analysis of royalty datasets Clean, standardise, and structure raw, inconsistent datasets to support valuation and due diligence work Develop, test, and document transformation processes, applying data quality checks and validation Collaborate with the Transaction Services and central data teams to deliver accurate, actionable insights Contribute to internal projects, including development of dashboards, front-end applications, and … analytical tools Apply best practices in version control, CI/CD, and cloud infrastructure management Identify opportunities to improve processes, systems, and data quality across the business Requirements: Strong experience in SQL, Python, Spark, and cloud infrastructure (AWS, Databricks preferred) Hands-on experience building production data pipelines, coding transformations (e.g., dbt), and implementing data More ❯

vector database experience Previous experience in educational technology, AI/ML applications, or digital learning product development (desirable) Proven experience in database design, API development, and data pipeline management Cross-cultural competency in diverse educational contexts Deep ELT methodology knowledge and curriculum development expertise (desirable) We care about work/life balance here at OUP. With this More ❯

Role: Data Engineering Manager Location: Brighton (Hybrid - 1 day per week in-office) Salary: £90,000 An innovative, product-led tech company is hiring a Data Engineering Manager to lead a growing data function at the centre of a cutting-edge AI transformation. You'll be managing a team of three Engineers, working on … to join during an exciting scale-up phase and have real influence on architecture, systems, and product evolution. Key Responsibilities: Line manage and mentor a team of 3 Data Engineers Own and support delivery across both legacy and new AI-driven products Collaborate with Data Science, Engineering, and Product teams to build real-time data pipelines and infrastructure Be hands-on when needed: supporting with code, architecture, and stakeholder requirements Help shape and scale a modern data stack, and act as a technical coach within the team What You'll Be Working On: Building and supporting "Humara" - an intelligent, AI-driven virtual sales advisor that enables more human-like customer interactions More ❯

in high-velocity environments. Deep knowledge of modern ML (supervised, unsupervised, generative, self-supervised learning, etc.). Strong understanding of ML systems at scale: inference pipelines, observability, MLOps, data pipelines, etc. Background in building ML for real-world infra, distributed systems, or low-latency environments is a big plus. Confident working at both strategic and technical depth, exec More ❯

solutions and consultancy projects to clients across a range of sectors (travel, manufacturing, microfinance, engineering, services etc.). Typical responsibilities will include: Manage both scheduled and ad hoc data harvesting processes. Write efficient regular expressions to extract data from raw HTML, JSON, and XML. Clean and validate harvested data using automated processes. Troubleshoot and … resolve complex data processing issues. Maintain and enhance existing data pipelines and services. Improve automation for data processing and delivery. Core Requirements: Solid understanding of HTTP protocols and request/response architecture. Proficiency in writing Regular Expressions for data extraction. Working knowledge of Python, SQL, and either C# or Java. Strong problem … solving and debugging skills. Effective communication skills for understanding project needs and working with the team. Advantageous: Experience with automated data harvesting using Python, C#, or Java. Familiarity with front-end technologies (JavaScript, React). Experience with cloud platforms like AWS or GCP (especially BigQuery). Exposure to Big Data tools such as Elasticsearch. Background in More ❯

Job Title: Data Engineer Location: London (Hybrid - 2 days in office) Type: Full-time Agency: Representing a leading digital media group We're representing a well-established digital media company with a growing portfolio of high-traffic content brands. As they expand their data capabilities, they're looking for a Data Engineer to join … their London-based team. Key Responsibilities: Design and maintain scalable data pipelines across diverse sources. Automate and optimise workflows using tools like Airflow, dbt, and Spark. Support data modelling for analytics, dashboards, and A/B testing. Collaborate with cross-functional teams to deliver data-driven insights. Work with cloud platforms (GCP preferred) and … A proactive, collaborative approach and strong communication skills. This is a fantastic opportunity to work in a fast-paced, content-rich environment and help shape the future of data in digital publishing. Diversity, equity and inclusion are at the heart of what we value as an organisation. Boston Hale is an equal opportunities employer, and all qualified applicants More ❯

Data Engineer (SC cleared) Start: ASAP Duration: 12 months Location: Mostly Remote - must have access to London or Bristol Pay: negotiable, INSIDE IR35 Responsibilities: - Design, implement robust ETL/ELT data pipelines using Apache Airflow - Build ingestion processes from internal systems and APIs, using Kafka, Spark, AWS - Develop and maintain data lakes and warehouses … AWS S3, Redshift) - Ensuring governance using automated testing tools - Collaborate with DevOps to manage CI/CD pipelines for data deployments and ensure version control of DAGs - Apply best practice in security and compliance Required Tech Skills: - Python and SQL for processing - Apache Airflow, writing Airflow DAGs and configuring airflow jobs - AWS cloud platform and services like S3 … Redshift - Familiarity with big data processing using Apache Spark - Knowledge of modelling, schema design and partitioning strategies - Understanding batch Vs streaming data paradigms - Docker or Kubernetes (containerization More ❯

Location: London OR Bristol 2- days per week Pay: negotiable, INSIDE IR35 We're looking for an experienced Integration Engineer to join our team and ensure seamless, secure data flow across systems, APIs, and backend services. You'll play a key role in connecting platforms, optimizing data pipelines, and maintaining integration health across our architecture. What … You'll Do: - Design and implement integrations between internal systems and third-party platforms via APIs and other technologies - Build and manage scalable data pipelines for data ingestion, transformation, and delivery - Collaborate with engineering, DevOps, and product teams to define and deliver integration solutions - Troubleshoot and resolve issues such as data inconsistencies, auth errors … of both SQL and NoSQL databases - Familiarity with AWS and its integration services - Strong problem-solving, communication, and collaboration skills - Agile team experience Nice to Have: - Awareness of data privacy and compliance standards (e.g., GDPR) - Exposure to CI/CD pipelines and DevOps practices More ❯

remote. Key Requirements: Strong experience with Server-Side Tag Management Google Tag Manager (GTM) - Server-Side essential Google Meridian - MUST HAVE Google Cloud Platform (GCP) Experience setting up data pipelines, event tracking, and tag execution environments Bonus Experience: Experience working with travel or airline clients (huge plus) eCommerce or Shopify knowledge is an advantage Timeline: Start Date: September More ❯

Job Title: Software Engineer – (AI Automation & LLM ) Location: London- Hybrid role Duration: 31st December 2025- possible extension Role: We're looking for a skilled Data Engineer/Software Engineer to support a pioneering AI research project focused on Large Language Model (LLM) agents and intelligent automation. You’ll play a central role in preparing and filtering text, image … and video datasets used to train advanced AI systems, helping to ensure data quality, relevance, and safety.. Key Responsibilities Data Onboarding: Convert raw datasets into structured formats suitable for automated processing. Filtering: Apply data … cleaning, quality checks, and filtering pipelines to identify and mitigate risks in datasets. Post-Processing: Integrate filtering results to modify original datasets, repackage them, and prepare for re-ingestion. Pipeline Engineering: Design, build, and maintain scalable and efficient data pipelines. Collaboration: Work closely with researchers and technical teams to refine processes and contribute to ongoing AI automation More ❯

Job Description: Are you passionate about Data and Analytics (D&A) and excited about how it can transform enterprise operations? Do you possess strategic vision, technical expertise, and leadership skills to drive data-driven solutions? If so, you might be the ideal candidate for the Data and Analytics role within Mars's Global Pet … achieve growth, profitability, and customer centricity. This role requires demonstrating thought leadership, technical proficiency, and the ability to navigate complex challenges while leading a team of top-tier data and analytics professionals. The role involves transitioning to a product-based model to develop digital capabilities. Responsibilities include developing and maintaining robust data pipelines and storage solutions … to support analytics and machine learning initiatives. You will report to the Director of Data Engineering Solutions and collaborate globally with engineering teams across core products. Key Responsibilities: Technical Leadership: Provide leadership to data and DevOps engineers. Collaborate on designing and evolving scalable data platforms. Promote best practices and foster a high-performance culture. More ❯

AWS Data Engineer – Contract Location: Reading (Hybrid – 1–2 days/month onsite) Rate: £500-550/day (Inside IR35) Start Date: ASAP Duration: 5 months (with potential for extension)A leading financial services organisation is seeking an experienced AWS Data Engineer to join their Compliance Reporting team. This backend-focused role involves designing and deploying … scalable data solutions that support the delivery of regulatory compliance reports across the business.You’ll work with a modern AWS stack and infrastructure-as-code tools to build robust data pipelines and applications that process complex datasets from multiple operational systems.Key Responsibilities: Build and maintain AWS-based ETL/ELT pipelines using S3, Glue (PySpark/… Python), Lambda, Athena, Redshift, and Step Functions Develop backend applications to automate and support compliance reporting Process and validate complex data formats including nested JSON, XML, and CSV Collaborate with stakeholders to deliver technical solutions aligned with regulatory requirements Manage CI/CD workflows using Bitbucket, Terraform, and Atlantis Support database management and improve data ingestion More ❯

to humanity's enduring challenges. We are looking for a Senior DevSecOps Engineer to join the Pathogen Programme at EIT. In this role, you'll help ensure our data platform is built to the highest standards, with a strong emphasis on automation across the development lifecycle. You'll work closely with engineers to deploy data pipelines … their workflows. You'll be responsible for maintaining infrastructure, designing secure automation pipelines, managing cloud environments, and ensuring security and compliance. You'll collaborate with cross-functional teams, data engineers, backend, and full-stack developers, to build robust, automated deployment pipelines across our environments. Key Responsibilities Design, implement, and maintain secure cloud infrastructure using Oracle Cloud Infrastructure (OCI … tools like Terraform to enable secure, repeatable deployments. Implement and manage CI/CD pipelines, focusing on automated security testing, deployment, and monitoring. Ensure all aspects of the data platform OCI infrastructure, data ingest pipelines, tool deployments, access controls, and monitoring are developed, tested, and deployed using automation best practices. Support bioinformaticians in building pipelines that More ❯

working £300 to £350 a day A top timer global consultancy firm is looking for an experienced Hadoop Engineer to join their team and contribute to large big data projects. The position requires a professional with a strong background in developing and managing scalable data pipelines, specifically using the Hadoop ecosystem and related tools. The role … will focus on designing, building and maintaining scalable data pipelines using big data hadoop ecosystems and apache spark for large datasets. A key responsibility is to analyse infrastructure logs and operational data to derive insights, demonstrating a strong understanding of both data processing and the underlying systems. The successful … candidate should have the following key skills Experience with Open Data Platform Hands on experience with Python for Scripting Apache Spark Prior experience of building ETL pipelines Data Modelling 6 Months Contract - Remote Working - £300 to £350 a day Inside IR35 If you are an experienced Hadoop engineer looking for a new role then this is More ❯

Job title: Power Platform & Azure Data Developer (AI focused) Location: Central London (hybrid - 2 days on site) Salary: Up to £70,000 DOE About the role/company: This is a great opportunity to join a Pharmaceutical company in London who are growing their team. The business are in the process of delivering some of their AI strategies … and are seeking a dynamic and skilled Power Platform & Azure Data Developer with hands-on experience or exposure to AI Key skills/responsibilities: Design, build, and maintain Power Apps (integrate them with Power Automate and in Power BI). Integrate Power Platform solutions with other Microsoft services (e.g SharePoint, Dynamics) Build and manage data pipelines … using Azure Data Factory. Support the deployment of cognitive services (e.g., Azure OpenAI, Azure Machine Learning) Work closely with business stakeholders, analysts, and other developers to gather requirements. Proven experience designing and deploying Power Automate workflows. AI/ML exposure with basic understanding of AI concepts Click 'apply now' or get in touch with Joe Bigsby on More ❯

control and branching methodologies using GIT Application integration using SOAP web services and REST APIs OWASP Top 10 security framework Agile and SCRUM Significant experience developing and implementing data solutions in a high-volume data loading environment Excellent understanding of SSIS framework, ADF data pipelines, administration, maintenance, and configuration Significant experience and clear demonstration More ❯

enables all full-time employees to work up to 2 days a week from home as work permits. POSITION SUMMARY Infleqtion is recruiting a passionate and detail-driven Data Scientist to join our team, focusing on time series modelling and control systems. You'll be developing predictive algorithms, identifying dynamic patterns and anomalies, and building models that help … automate decision-making in real-time systems involving quantum sensors. This is a high-impact role bridging data science, physics and engineering disciplines. JOB RESPONSIBILITIES The duties and responsibilities outlined below include essential functions of the role. Depending on business needs, this role may perform a combination of some or all of the following duties. Design and implement … to integrate predictive models into live quantum sensors. Evaluate performance of deployed models and develop tools/pipelines to continuously refine them. Analyse large-scale sensor and telemetry data from quantum sensor systems Effectively manage technical priorities, meet deadlines, and deliver on project objectives. Masters degree in a STEM field (maths, science, engineering etc.) or equivalent Strong programming More ❯

for an extra person to join them to work on statistical method development and application to real-world drug discovery & development problems. Your new role As a Senior Data Scientist, your work will centre on methodological innovation. You will: Design and implement novel statistical approaches to interrogate large-scale genomic datasets Develop new models to quantify genetic contributions … reproducibility in all method development Translate complex statistical outputs into meaningful insights for technical and non-technical audiences Collaborate with engineering teams to embed new methods into scalable data pipelines Contribute to peer-reviewed publications that showcase methodological advancements Stay ahead of emerging techniques in statistical genetics and bioinformatics, integrating them into ongoing research While pharma/biotech … work from one of the company's offices across the UK. What you'll need to succeed PhD (or Master's with substantial experience) in statistics, maths, physics, data science, computing, statistical genetics or a related field with a strong methodological focus (or equivalent experience) Demonstrated ability to create and validate new statistical/analytical models or workflows More ❯

a market leading financial services (fintech) organisation to recruit a AI Engineer on a contract basis - Hybrid working - London based The AI Engineer will join a cross-functional data team to design, develop, and deploy AI/ML models that tackle real business problems - from prediction and automation to customer personalisation and decision intelligence. You'll play a … practices across the AI lifecycle. Role The AI Engineer will build and deploy scalable machine learning models in a cloud-native environment (AWS, GCP or Azure) Collaborate with data engineers, analysts, and product teams to translate business needs into AI-driven solutions Contribute to the development of data pipelines and feature engineering workflows Integrate models into … are dependent upon your experience, qualifications and training. If you wish to apply, please read our Privacy Notice describing how we may process, disclose and store your personal data: gb/en/privacy-notice. More ❯

Azure Databricks Platform Lead Hybrid | Slough Are you a Databricks expert with deep knowledge of both data infrastructure and engineering We’re looking for a Databricks Champion to take ownership of an enterprise-scale Azure Databricks environment — driving performance, governance, connectivity, and cost optimisation across the platform. This is a client-facing, hands-on role that combines cloud … engineering, data architecture, and platform operations. You’ll work closely with security, DevOps, data, and networking teams — and must be comfortable engaging senior stakeholders and driving best practices across the board. Location : Hybrid with travel to Slough (a few days per week) Start : ASAP Type: Full time permenant What You’ll Be Doing: Manage & optimise Azure … Databricks workspaces (VNet Injection, private endpoints, cluster config, performance tuning, cost control) Design and implement Unity Catalog for governance, access control, data lineage, and auditing Integrate Entra ID for SSO, SCIM, credential passthrough, Key Vault, and ACLs Build secure networking (VNet peering, NSGs, DNS/routing, firewalls) Orchestrate data pipelines with ADF Databricks Configure and troubleshoot More ❯

Data Insights Analyst Up to £55,000 | Hybrid - 3 Days in Oxfordshire A rapidly scaling e-commerce and logistics company is looking for a Data Insights Analyst to help shape the future of its customer experience and commercial strategy. With sustainability and tech innovation at its core, the business operates multiple online brands and is one … focused on driving smarter decision-making across marketing, product, and customer experience teams. You'll work closely with stakeholders to uncover behavioural trends, improve digital performance, and develop data-led strategies that support business growth. Key Responsibilities: Analyse customer journeys, engagement patterns, and conversion funnels to identify opportunities for UX and performance improvement Support development of personalisation and … deep-dive analysis using large datasets (customer, product, marketing, transactional) Contribute to the creation of a Single Customer View across brands Collaborate with tech teams to ensure clean data pipelines and reliable tracking Present findings to non-technical stakeholders and influence strategic decisions at senior levels Continuously test, iterate, and refine data-driven hypotheses to support More ❯

financial organisation you will be decommissioning a legacy commodities platform and then become part of new platform team! What you'll need to succeed Great experience as a Data Engineer and strong recent Financial Services or Insurance experience - working in regulated environments. You must be an expert with Databricks - Databricks certification (desirable). Experience with architecting and engineering … Databricks solutions from scratch. Strong experience with Python or Spark Expertise with modern data pipelines and running into production. Fantastic communication skills - Consultative/stakeholder management experience - you will help stakeholders use the platform. What you'll get in return Flexible working options available. What you need to do now If you're interested in this role, click More ❯

Do: Develop responsive and intuitive front-end interfaces using Blazor/Razor Components Translate UX designs and wireframes into performant, user-friendly features Build and integrate APIs and data pipelines across mobile and desktop environments Contribute to a platform-wide adoption of AI-enhanced development (e.g. automated testing, design audits) Support scalable cloud architecture using Docker, Kubernetes, and … DevSecOps workflows (GitHub, CI/CD, automated testing) Experience in the enterprise infrastructure, construction, or engineering sectors is highly desirable, especially where digital workflows, field operations, or geospatial data have played a role. Bonus Points For: Exposure to event streaming platforms (e.g. Kafka, SQS, EventHub) Familiarity with SQL/NoSQL, CosmosDB, or Kusto (KQL) Knowledge of IoT protocols More ❯