76 to 100 of 141 Data Pipeline Jobs in the South East

Data Engineering Lead - Summary A leading global construction firm is seeking a Data Engineering Lead to drive the evolution of its hybrid Azure/Databricks data platform. This role is central to building scalable, high-performance data solutions that bridge infrastructure and business intelligence. Key Responsibilities:* Design and maintain robust data pipelines for ingestion, transformation, and delivery* Leverage Azure, Databricks, and Azure Data Factory to integrate and enrich data from diverse sources* Implement orchestration, monitoring, and error-handling best practices* Optimize hybrid cloud architecture for performance, scalability, and cost-efficiency* Lead a team of BI Developers, ensuring timely delivery of dashboards and data products* Promote adoption of best practices and continuous learning within the team* Enforce data quality, governance, and security standards Technical Requirements:* Deep experience with Azure, SQL, Databricks, Azure Data Factory, and medallion architecture* Strong ETL design and hybrid cloud integration skills* Proficiency in Power BI and DAX for dashboard development* Solid understanding of KPI More ❯

Data Engineer - Microsoft Fabric - Azure - London - Hybrid - 70k A well-established multinational organisation is undergoing a major data transformation, moving from legacy systems to a modern Microsoft ecosystem built on Azure and Fabric. We're looking for a Data Engineer with hands-on Microsoft Fabric experience to lead this migration, design scalable architecture, and … support analytics, forecasting, and AI enablement. Key Responsibilities: Spearhead the migration to Microsoft Fabric Build robust data pipelines using OneLake, Dataflow and Data Factory Deliver full-stack data engineering solutions What You'll Bring: Proven experience in BI or Data Engineering roles Strong skills in Microsoft Fabric and Azure Data Services Solid knowledge of SQL and Power BI Experience with data migration and modern data platforms What's on Offer: Competitive salary up to £70,000 (DOE) Hybrid working - just 2-4 days per month onsite in London A pivotal role in a high-impact transformation project Supportive, flexible, and collaborative team culture Interviews More ❯

A well-established business in Brentford is seeking an experienced BI Developer to join their BI & Data Team as they embark on a major transformation journey, moving from legacy systems towards a modern Microsoft ecosystem built on Azure and Fabric. The team operates mostly remotely, with office collaboration up to once per week alongside the wider BI & Data … and ensure the new set-up supports analytics, forecasting, and AI enablement. Key responsibilities will include: Designing and delivering scalable BI solutions using Microsoft Fabric Building and managing data pipelines with OneLake, Dataflows Gen2, and Data Factory Developing semantic models in Power BI and integrating with DirectLake or Warehouse datasets Optimising data models for … performance and reusability Supporting governance, security, and compliance best practices in a modern data platform Tooling for migration and reconciliation of data from legacy systems or acquisitions Providing technical support and hands-on data engineering across the full stack We are looking for: Strong experience as a BI Developer, Data Engineer, or More ❯

Data Engineering Lead - Hybrid - Azure/Databricks - London Permanent Salary: 90,000 Are you a seasoned Data Engineering professional ready to lead the charge in building scalable, high-performance data platforms? An industry leading, global firm is seeking a Data Engineering Lead to drive innovation across their hybrid Azure/Databricks environment. … About the role: Shape data strategy for global projects Lead a talented team of BI Developers Enjoy a flexible working culture and industry-leading benefits Work with cutting-edge tech in a collaborative, inclusive environment Key Responsibilities: Architect and optimize robust data pipelines for large-scale ingestion, transformation, and delivery Leverage Azure and Databricks to enable … seamless data integration and enrichment Implement best practices for orchestration, monitoring, and error handling Lead and mentor a BI team, ensuring timely delivery of dashboards and data products Champion data governance, quality, and security across platforms Requirements: Proven expertise in Azure, SQL, Databricks, and Azure Data Factory Experience implementing medallion architecture frameworks More ❯

Data Engineering Lead - Hybrid - Azure/Databricks - London Permanent Salary: £90,000 Are you a seasoned Data Engineering professional ready to lead the charge in building scalable, high-performance data platforms? An industry leading, global firm is seeking a Data Engineering Lead to drive innovation across their hybrid Azure/Databricks environment. … About the role: Shape data strategy for global projects Lead a talented team of BI Developers Enjoy a flexible working culture and industry-leading benefits Work with cutting-edge tech in a collaborative, inclusive environment Key Responsibilities: Architect and optimize robust data pipelines for large-scale ingestion, transformation, and delivery Leverage Azure and Databricks to enable … seamless data integration and enrichment Implement best practices for orchestration, monitoring, and error handling Lead and mentor a BI team, ensuring timely delivery of dashboards and data products Champion data governance, quality, and security across platforms Requirements: Proven expertise in Azure, SQL, Databricks, and Azure Data Factory Experience implementing medallion architecture frameworks More ❯

Data Engineering Lead - Hybrid - Azure/Databricks - London Permanent Salary: £90,000 Are you a seasoned Data Engineering professional ready to lead the charge in building scalable, high-performance data platforms? An industry leading, global firm is seeking a Data Engineering Lead to drive innovation across their hybrid Azure/Databricks environment. … About the role: Shape data strategy for global projects Lead a talented team of BI Developers Enjoy a flexible working culture and industry-leading benefits Work with cutting-edge tech in a collaborative, inclusive environment Key Responsibilities: Architect and optimize robust data pipelines for large-scale ingestion, transformation, and delivery Leverage Azure and Databricks to enable … seamless data integration and enrichment Implement best practices for orchestration, monitoring, and error handling Lead and mentor a BI team, ensuring timely delivery of dashboards and data products Champion data governance, quality, and security across platforms Requirements: Proven expertise in Azure, SQL, Databricks, and Azure Data Factory Experience implementing medallion architecture frameworks More ❯

Data Engineering Lead - Hybrid - Azure/Databricks - London Permanent Salary: £90,000 Are you a seasoned Data Engineering professional ready to lead the charge in building scalable, high-performance data platforms? An industry leading, global firm is seeking a Data Engineering Lead to drive innovation across their hybrid Azure/Databricks environment. … About the role: Shape data strategy for global projects Lead a talented team of BI Developers Enjoy a flexible working culture and industry-leading benefits Work with cutting-edge tech in a collaborative, inclusive environment Key Responsibilities: Architect and optimize robust data pipelines for large-scale ingestion, transformation, and delivery Leverage Azure and Databricks to enable … seamless data integration and enrichment Implement best practices for orchestration, monitoring, and error handling Lead and mentor a BI team, ensuring timely delivery of dashboards and data products Champion data governance, quality, and security across platforms Requirements: Proven expertise in Azure, SQL, Databricks, and Azure Data Factory Experience implementing medallion architecture frameworks More ❯

Data Engineering Lead - Hybrid - Azure/Databricks - London Permanent Salary: £90,000 Are you a seasoned Data Engineering professional ready to lead the charge in building scalable, high-performance data platforms? An industry leading, global firm is seeking a Data Engineering Lead to drive innovation across their hybrid Azure/Databricks environment. … About the role: Shape data strategy for global projects Lead a talented team of BI Developers Enjoy a flexible working culture and industry-leading benefits Work with cutting-edge tech in a collaborative, inclusive environment Key Responsibilities: Architect and optimize robust data pipelines for large-scale ingestion, transformation, and delivery Leverage Azure and Databricks to enable … seamless data integration and enrichment Implement best practices for orchestration, monitoring, and error handling Lead and mentor a BI team, ensuring timely delivery of dashboards and data products Champion data governance, quality, and security across platforms Requirements: Proven expertise in Azure, SQL, Databricks, and Azure Data Factory Experience implementing medallion architecture frameworks More ❯

Cloud Data & Full Stack Engineer - SC Cloud, Data Location: UK Wide Salary: £72,000 - £100,000 (depending on experience) + attractive benefitsFlexibility: Happy to consider candidates who can commute into London from other UK locationsSecurity: Essential that you can undergo security clearanceAbout the RoleWe're looking for a Cloud Data & Full Stack Engineer to … join our team and help build scalable, cloud-native platforms and applications that enable analytics, automation, and digital transformation. This role combines advanced data engineering with full stack development, giving you the chance to work on cutting-edge solutions that make a real impact.Key Responsibilities* Data Engineering: Design and implement data pipelines, lakes, and … warehouses using tools like Spark, Airflow, or dbt.* API & Microservices Development: Build secure, efficient APIs and microservices for data integration.* Full Stack Development: Deliver responsive, high-performance web applications using React (essential), plus Angular or Vue.js.* Back-End Expertise: Develop server-side logic with Java and Node.js; Python or Ruby experience is a bonus.* Cloud & Containers: Deploy microservices More ❯

Lead Data Engineer Salary: £90,000 - £115,000 Location: West London (3 Days in Office) Using data science at their core, they transform complex property and customer data into insights that enhance targeting, retention, and performance. The Opportunity This flexible, data-driven start-up places Data Science at the heart … of its business, using cutting-edge UK property data to deliver powerful insights for real estate partners and retailers. By leveraging advanced analytics, predictive modelling, and customer segmentation, the company helps clients better understand and engage with their audiences, improving marketing precision, timing, and overall performance. With a strong technical foundation and a collaborative team led by experienced … data leaders, the company is building efficient, automated data pipelines and scalable systems to power intelligent decision-making. This is an exciting opportunity for forward-thinking professionals to join a growing team, shape best practices, and make a real impact in a fast-evolving data and technology environment. The Role and Responsibilities This is More ❯

/DevOps/MLOps capability to develop, productionise, and scale systematic investment models and sustainability analytics. You will own the end to end research to production lifecycle : automating data operations, building and validating quant models, and integrating them into scalable platforms, while providing timely responses to internal and external client queries. The role is a key contributor to …/robustness analysis; implement model risk controls , documentation, and reproducibility. Scale and commercialise proprietary metrics for investment use‐cases and new revenue lines (e.g., indices, APIs/subscriptions, data products). 2) Data Operations & Quality Control (Automation‐First) Work with systematic team and IT to automate ingestion, validation, and monitoring of 30+ multi‐vendor datasets (schema … evolution, mapping, entity resolution, referential integrity, SLA oversight). Run impact analysis of data changes on models and reporting; manage deployment pipelines and data lineage/audit trails ; support data procurement and budgeting. 3) Platform Development & Process Management Co-develop with systematic team and IT a scalable data/quant architecture (data More ❯

Data Engineer (SC) Croydon - 2-4 days per month on site £500/day outside IR-35 Contract until the end of the financial year We are seeking an experienced AWS Data Engineer to support a major government programme delivering secure, scalable data solutions. Key Responsibilities Design and implement data pipelines on … AWS using services such as Glue, Lambda, S3, and Redshift. Develop ETL processes and optimise data workflows for performance and security. Collaborate with analysts and architects to ensure compliance with government security standards. Troubleshoot and resolve issues in complex cloud environments. Essential Skills Strong experience with AWS services (Glue, Lambda, S3, Redshift, IAM). Proficiency in Python and … SQL for data engineering tasks. Knowledge of data modelling, ETL frameworks, and best practices. Familiarity with security and compliance in government or regulated environments. Excellent communication and problem-solving skills. Active SC clearance (mandatory). Desirable Experience with Terraform or CloudFormation. Exposure to CI/CD pipelines and DevOps practices. Reasonable Adjustments: Respect and equality are More ❯

emissions for a cleaner, brighter future. The role: As a Senior SAP Functional Business Analyst, you will help drive our goals by: Acting as a bridge between the data team (e.g., SAP BW, data engineering, and BI reporting colleagues), the business, and the SAP technical team by helping to build the data foundation and … develop reporting solutions based on SAP processes and data flows. Engaging directly with business stakeholders to understand and clarify reporting and analytics requirements, primarily in the finance (Record-to-Report) domain, and translating them into technical specifications aligned with SAP Business Warehouse (BW/4HANA) and SAP Analytics Cloud (SAC) architecture and development standards. Analysing SAP ERP (ECC … transactions to identify relevant extractors, tables, and logic for reporting, while acting as a subject matter expert in Financial and other SAP processes, transaction flows, and data structures. Collaborating with data team to design, develop, and validate robust data pipelines and models, ensuring data integrity, consistency, and completeness through validation against ECC More ❯

Data Engineer Salary: £70,000 Location: London (3 Days per Week in Office) Join a fast-paced analytics team driving data innovation at a global tech powerhouse! Take ownership of modern data pipelines, shape the future of reporting, and make an immediate impact in a collaborative, forward-thinking environment. The Opportunity This is an … exciting opportunity to step into a senior-level data engineering role within a global technology business that's transforming the way customers connect with everyday services. You'll play a key part in building and optimizing modern data pipelines, supporting a major migration from Tableau to Looker, and ensuring the delivery of clean, reliable, and scalable … data solutions. The team operates with a strong focus on collaboration, innovation, and autonomy, giving you the chance to influence data strategy and mentor others. You'll work closely with analytics and marketing stakeholders to bring data-driven insights to life and help shape the future of customer experience. This is a role for More ❯

Data Visualisation & Analytics Experience Specialist Overview We are seeking a highly skilled Data Visualisation & Analytics Experience Specialist to bridge the gap between data, design, and user experience. This role combines analytical capability with front-end development and UX skills to create intuitive, impactful interfaces for data-driven decision making. You will work … closely with Business Analysts, Product Teams, Engineers, and Data Practitioners to turn complex outputs into accessible, engaging visual products. Key Responsibilities Data Analysis & Insight Delivery Collaborate with Business Analysts and Product Teams to identify high-impact business problems that can be solved through data analysis and visualisation, utilising tools such as Microsoft Fabric and … pipelines and feature engineering activities in partnership with Data Engineers. Design and develop a unified, user-friendly front-end interface that integrates dashboards, reports, models, and pipeline outputs into a single cohesive experience. Optimise front-end application performance, ensuring fast load times, responsive layouts, and smooth, intuitive interactions. User Experience & Accessibility Ensure all interfaces and visual More ❯

Lead Data Operations BI Analyst Up to £83,000 + 10% bonus Watford Junction - Hybrid (ideally 3 days per week, 2 in Watford and 1 in London) Sponsorship available The Company This global technology-driven organisation is on a major data transformation journey, building the infrastructure and intelligence behind one of the UK's most high … profile platforms. They've created a collaborative, fast-paced environment where data, analytics and engineering teams play a critical role in ensuring operational excellence and continuous improvement across business systems. The Role Sitting within the wider Business Intelligence function, this role reports into the BI Data Operations Manager and acts as the technical lead across data monitoring, automation, and issue resolution. You'll work closely with the offshore team of engineers and analysts, ensuring data pipelines and workflows run seamlessly within an AWS environment. Day to day, you'll: Oversee end-to-end data operations, ensuring all loads complete successfully and identifying and resolving workflow issues within Airflow Manage and More ❯

Data Analyst: Global Customer Success (10-Month Contract) Are you an 8+ year Data Analyst ready to drive global customer success for an industry leader in financial software? We're looking for a highly motivated Data Analyst to join the Global Customer Success Data Science Team in London, UK. This is a … critical 10-month contract role focused on establishing reliable data foundations and enabling actionable insights across multiple international markets. What You'll Do: Build the Foundation: Collaborate on data pipelines, improve integrity, and establish metric definitions tailored for non-US markets. Automate & Visualize: Create scalable, user-friendly dashboards to track core KPIs (tNPS, AHT, repeat contacts … improve the customer journey. Experiment: Support A/B testing and digital experience evaluations by analyzing performance and quantifying business impact. What You'll Bring: 8+ years in Data Analytics/Science. Expertise in SQL and dashboarding tools ( Tableau/Qlik ). Familiarity with big data tools ( Snowflake, Databricks ) and ETL . Experience with A/ More ❯

Data Engineer Watford | £85,000 + Excellent Benefits | Office-Based The Opportunity An excellent opportunity for a talented Data Engineer to join a well-established and growing financial services organisation based in Watford . Offering a salary of £85,000 plus a superb benefits package , this role is ideal for someone seeking a technically hands-on … position within a forward-thinking and data-driven environment. The company has been operating successfully for over 20 years, employs more than 300 staff, and is recognised as a market leader in specialist financial services. You'll be joining a collaborative data team where innovation, accuracy, and analytical thinking are highly valued. Duties & Responsibilities Design, build … and maintain robust data pipelines and reporting solutions using modern cloud technologies. Work closely with senior leadership to shape data and reporting strategies. Develop and enhance MI & BI reports, ensuring accuracy and integrity across all data outputs. Collaborate with Product Owners, Developers, and other technical teams to support business goals. Mentor junior members of More ❯

Technical Delivery Manager, Delivery Manager We are working with a leading central government organisation to find a Technical Delivery Manager with a strong data delivery expertise and active SC Clearance. You will be required to lead cross functional teams to deliver a data focused technology project that supports high impact public services, ensuring outcomes are aligned … day inside ir35 Location - London - Once a week onsite Duration - 6 Months with the opportunity of extension Key Responsibilities and Key Skills Deliver end to end technical and data driven projects within a complex government environment Manage project scopes, timelines, risks, budgets, dependencies and reporting in line with government delivery frameworks Work closely with data engineers … architects, analysts and stakeholders to ensure effective data integration, transformation and governance across programmes Coordination of multi disciplinary agile and hybrid teams ensuring delivery milestones are met Produce detailed project documentation Ensure compliance with security, privacy and assurance processes required within SC Cleared environment Active SC Clearance (mandatory to start). Proven experience delivering data-focused More ❯

DATA SCIENTIST - E-DV CLEARED NEW CONTRACT OPPORTUNITY FOR A DATA SCIENTIST IN LONDON WITH ENHANCED DV CLEARANCE New contract opportunity with a leading consultancy for a Data Scientist London based, 3-4 days on site Enhanced DV Clearance required to start 12 month rolling contract Daily rate up to £850 To apply, email … About We're seeking an experienced Data Scientist with enhanced Developed Vetting (DV) clearance to join a leading consultancy supporting mission-critical UK Government projects . This is a hands-on, high-impact role within a secure environment, where your analytical expertise will directly contribute to national operational outcomes. The Role As a Data Scientist, you … ll work within a multidisciplinary team to extract insight from complex, high-volume datasets. You'll design and implement advanced analytical and machine learning models, develop robust data pipelines, and deliver actionable intelligence to inform decision-making at the highest levels. This role offers the opportunity to apply your technical skills to highly operational challenges in a fast More ❯

CONTRACT PALANTIR DATA ENGINEER - DV CLEARED NEW OUTSIDE IR35 CONTRACT OPPORTUNITY AVAILABLE WITHIN A HIGHLY SECURE DELIVERY PROGRAMME FOR A DV CLEARED PALANTIR FOUNDRY DATA ENGINEER. Contract opportunity for a Palantir Data Engineer to support cutting-edge National Security projects Up to £650 per day (Outside IR35) DV clearance is essential Based full-time … using Foundry tools like Workshop Strong communication skills to collaborate with technical and military/non-technical stakeholders Experience integrating data from multiple sources into Foundry via Pipeline Builder and custom code Comfortable working on-site within a secure environment and at pace with end-users TO BE CONSIDERED... . Please either apply by clicking online or … consent for us to process and submit your application to our client in conjunction with this vacancy only. KEY SKILLS: PALANTIR/FOUNDARY/DATA ENGINEER/PIPELINE BUILDER/ONTOLOGY/PYTHON/TYPESCRIPT/FULL STACK/DV CLEARED/LONDON/CONTRACT More ❯

We’re seeking a Lead Data Engineer to design and deliver scalable data and AI solutions across internal and client projects. You’ll own the full data lifecycle — from ingestion and processing to deployment and monitoring of machine learning models — while leading a small, high-performing team. What You’ll Do Build and maintain … data pipelines and cloud infrastructure (AWS/GCP). Develop AI-driven applications including LLM tools, RAG pipelines, and automation solutions. Lead on client communication , project delivery, and stakeholder alignment. Collaborate with Data Science and Analytics teams to improve data access, automation, and governance. Drive innovation through proofs-of-concept , model monitoring , and adoption … of emerging AI tech. What We’re Looking For 5+ years in data engineering, with team or project leadership experience. Advanced Python (Pandas, PyTorch, TensorFlow, Scikit-learn). Strong AWS/GCP , MySQL , and CI/CD experience; Docker/Kubernetes a plus. Excellent communication, organisation, and problem-solving skills. Passion for innovation and ethical, scalable AI. Nice More ❯

a Technical Project Manager to join a scaling startup in the Education sector Salary up to £60k , depending on experience Fully remote role (UK-based preferred) Tech environment: Data Engineering | AI | Python | GCP | Agile | Jira If you wish to keep your CV/data private, feel free to WhatsApp your details/CV to me, Dan … WHO WE ARE: We're an EdTech startup using AI and data-driven technology to transform how students learn and educators teach. Our intelligent learning platform empowers users through personalised insights, automating complex processes and enabling smarter decision-making. As we continue to scale, we're strengthening our technical delivery capability and looking for an organised, tech-savvy … Technical Project Manager to help keep our engineering efforts running smoothly. WHAT YOU WILL BE DOING: Coordinate and track technical projects across data, engineering, and AI teams. Facilitate Agile delivery - managing sprint planning, daily stand-ups, retrospectives, and backlog prioritisation. Act as a bridge between engineering and product , ensuring goals are aligned and progress is transparent. Monitor project More ❯

Fabric Consultant Location: Hybrid in London Salary: Up to 90,000 + Benefits Are you passionate about data engineering? Join a leading Microsoft partnered consultancy delivering cutting-edge solutions with Azure, Power BI, and Microsoft Fabric. This is your chance to work on exciting projects, design robust data architectures, and help organisations unlock the full potential … of their data. Key Responsibilities: Build and optimise data pipelines and ETL workflows using Microsoft Fabric and Azure Synapse or Databricks. Implement scalable solutions for data ingestion, storage, and transformation. Develop clean, reusable Python code for data engineering tasks. Research and integrate the latest cloud-based technologies. Requirements: Proven experience in data engineering with Azure tools (Synapse, Data Factory, Databricks). Experience with Microsoft Fabric. Strong Python programming skills. Knowledge of data modelling and Kimball methodology. Excellent communication and problem-solving skills. Fabric certifications are highly desirable Benefits: Competitive salary and bonus scheme Private medical cover, life assurance, and pension. Hybrid working. And more More ❯

Fabric Consultant Location: Hybrid in London Salary: Up to £90,000 + Benefits Are you passionate about data engineering? Join a leading Microsoft partnered consultancy delivering cutting-edge solutions with Azure, Power BI, and Microsoft Fabric. This is your chance to work on exciting projects, design robust data architectures, and help organisations unlock the full potential … of their data. Key Responsibilities: Build and optimise data pipelines and ETL workflows using Microsoft Fabric and Azure Synapse or Databricks. Implement scalable solutions for data ingestion, storage, and transformation. Develop clean, reusable Python code for data engineering tasks. Research and integrate the latest cloud-based technologies. Requirements: Proven experience in data engineering with Azure tools (Synapse, Data Factory, Databricks). Experience with Microsoft Fabric. Strong Python programming skills. Knowledge of data modelling and Kimball methodology. Excellent communication and problem-solving skills. Fabric certifications are highly desirable Benefits: Competitive salary and bonus scheme Private medical cover, life assurance, and pension. Hybrid working. And more More ❯