1 to 25 of 79 Permanent Data Pipeline Jobs in the South East

To Work. This prestigious award is based entirely on what current employees say about their experience working at Circana. Job description: We are seeking a skilled and motivated Data Engineer to join a growing global team. In this role, you will be responsible for designing, building, and maintaining robust data pipelines and infrastructure on the Azure … cloud platform. You will leverage your expertise in PySpark, Apache Spark, and Apache Airflow to process and orchestrate large-scale data workloads, ensuring data quality, efficiency, and … scalability. If you have a passion for data engineering and a desire to make a significant impact, we encourage you to apply! Job Responsibilities ETL/ELT Pipeline Development: Design, develop, and optimize efficient and scalable ETL/ELT pipelines using Python, PySpark, and Apache Airflow. Implement batch and real-time data processing solutions using More ❯

Job Description: Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in a dynamic, fast-growing category? If so, you might be … the ideal candidate for the role in the Data and Analytics function for Global Pet Nutrition (PN) at Mars. Pet Nutrition (PN) is the most vibrant category in the FMCG sector. As we work to transform this exciting category, a new program, Digital First, has been mobilized by the Mars Pet Nutrition (PN) leadership team. Digital First places … similar data processing tools. Strong technical proficiency in data modeling, SQL, NoSQL databases, and data warehousing. Hands-on experience with data pipeline development, ETL processes, and big data technologies (e.g., Hadoop, Spark, Kafka). Proficiency in cloud platforms such as AWS, Azure, or Google Cloud and cloud-based data More ❯

Job Description: Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in a dynamic, fast-growing category? If so, you might be … the ideal candidate for the role in the Data and Analytics function for Global Pet Nutrition (PN) at Mars. Pet Nutrition (PN) is the most vibrant category in the FMCG sector. As we work to transform this exciting category, a new program, Digital First, has been mobilized by the Mars Pet Nutrition (PN) leadership team. Digital First places … similar data processing tools. Strong technical proficiency in data modeling, SQL, NoSQL databases, and data warehousing. Hands-on experience with data pipeline development, ETL processes, and big data technologies (e.g., Hadoop, Spark, Kafka). Proficiency in cloud platforms such as AWS, Azure, or Google Cloud and cloud-based data More ❯

Engineer, you will operate within an Agile delivery environment, working closely with the Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from our wide-ranging dataset to support executive and operational decision making that will underpin sustained growth of BFS business units … solutions. You will also have management responsibilities for a small team of Data Engineers, who you will coach, support and organise to ensure we sustain a predictable pipeline of delivery, whilst ensuring all appropriate governance and best practice is adhered to. Your responsibilities as our Senior Databricks Data Engineer will include: Understand the business/… Azure DevOps, collaborating on Sprint cycles, code deployment, version control, and development practices Develop and maintain data models, schemas, and documentation Monitor and troubleshoot data pipeline issues, ensuring timely resolution Stay up-to-date with the latest industry trends and technologies in data engineering and recommend improvements to existing systems or processes as More ❯

Data Engineer/Scientist Department: Consultancy Employment Type: Full Time Location: United Kingdom/Hybrid Description As a Data Engineer/Scientist at Actica, you will have the opportunity to design, build, and maintain data pipelines while developing advanced analytics solutions to unlock business problems for high-profile UK public sector organisations. Your expertise … will enable organisations to maximise the value of their data assets through robust data infrastructure and sophisticated analysis, playing a key role in nationally critical projects that make a real difference to people's everyday lives. Locations: London, Guildford, Bristol, M4 corridor Hybrid working Roles and Responsibilities Actica recognises that data engineering, analytics, and … data science are distinct but interconnected disciplines. Our data engineers focus on building and maintaining the data infrastructure that enables analytics and data science work, while our data scientists and analysts focus on deriving insights and developing models. At Actica, we're at the forefront of the UK government's More ❯

To Work. This prestigious award is based entirely on what current employees say about their experience working at Circana. Job description: We are seeking a skilled and motivated Data Engineer to join a growing team Global Team based in the UK. In this role, you will be responsible for designing, building, and maintaining robust data pipelines … and infrastructure on the Azure cloud platform. You will leverage your expertise in PySpark, Apache Spark, and Apache Airflow to process and orchestrate large-scale data workloads, ensuring data quality, efficiency, and scalability. If you have … a passion for data engineering and a desire to make a significant impact, we encourage you to apply! Job Responsibilities Data Engineering & Data Pipeline Development Design, develop, and optimize scalable DATA workflows using Python, PySpark, and Airflow Implement real-time and batch data processing using Spark Enforce best practices More ❯

Data Engineer Bond street 50-70k ROLE OBJECTIVE This role requires a strong understanding of data architecture, data modelling, and performance optimisation. The role will contribute significantly to the company's success. There are opportunities for professional development, such as training programs, certifications, and career advancement paths. KEY RESPONSIBILITIES Design, develop, and maintain … scalable data pipelines SQL, Azure ADF, Azure Functions, DBT Collaborate with analysts and stakeholders to understand … their data needs, scoping and implementing solutions Optimising and cleaning of data warehouse, cleaning existing codebase and creating documentation. Monitor and troubleshoot data pipeline issues Provide technical support and guidance to junior team members Provide ongoing support for business applications. Provide help with data analysis using SQL, Qube, Tableau, Hubspot and More ❯

and building platforms that handle the complexity of modern cruising - all while keeping things collaborative and fun. Your Mission: Are you passionate about unlocking the power of customer data? We're looking for a talented Lead Data Engineer to spearhead the design, development, and optimisation of our critical CRM and customer data transformation. You … ll play a pivotal role in building the data foundations for advanced analytics, personalised customer experiences, and effective marketing activation. As a senior member of the team, you will lead data engineers and champion best practices within our data environment. What You'll Do: Design & Build: Architect, build, test, and deploy robust, scalable, and … Guide: Provide technical leadership to data engineers, fostering their growth and ensuring high standards in coding, testing, and documentation. Optimise: Continuously monitor and improve data pipeline performance, reliability, and cost-efficiency. Document: Create and maintain clear, comprehensive documentation for data models, pipeline architecture, and processes. What You'll Be Responsible For: The More ❯

Data Engineer - 18-Month Fixed-Term - London About the Role My client is seeking an experienced Data Engineer to join their Technology team on an 18-month fixed-term contract. You'll play a key role in building and optimising data pipelines, enabling analytics and AI, and integrating enterprise systems like CRM, HR, and … Finance. Key Responsibilities Design and develop scalable data pipelines using Azure Data Factory, dbt, and Synapse Build enriched datasets for Power BI and AI/ML use cases Implement CI/CD workflows Ensure GDPR compliance and secure data handling Requirements: 5+ years in data engineering or Strong experience with Azure, Databricks … Microsoft Fabric, and Snowflake Proficiency in SQL, Python, and tools like dbt and Airflow Familiarity with DevOps practices in a data context Benefits: Work on impactful, enterprise-wide data projects Collaborate with architects, analysts, and data scientists Be part of a supportive, innovative, and forward-thinking team Competitive salary And more Please Note: This More ❯

Location: Central London (Hybrid - 3 days in office) Salary: Up to £62,000 per annum Contract Type: Fixed Term (18 months) Are you a skilled Data Engineer looking to make a real impact in a world-renowned cultural institution? We're working with a prestigious organisation undergoing a major digital and data transformation - and they're … looking for a hands-on Data Engineer to help build the foundations of their future data estate. This is a unique opportunity to join a high-impact transformation programme at a pivotal moment. You'll play a key role in designing and building robust, scalable data pipelines that bring together data from … across the organisation - including CRM, HR, Finance, and Commercial systems - into clean, trusted, and AI-ready datasets. You'll be part of a growing data team, working closely with architects, analysts, and business stakeholders to unlock insights and enable innovation. What You'll Be Doing: Design and build robust, automated data pipelines using Azure Data More ❯

Location: Central London (Hybrid - 3 days in office) Salary: Up to £62,000 per annum Contract Type: Fixed Term (18 months) Are you a skilled Data Engineer looking to make a real impact in a world-renowned cultural institution? We're working with a prestigious organisation undergoing a major digital and data transformation - and they're … looking for a hands-on Data Engineer to help build the foundations of their future data estate. This is a unique opportunity to join a high-impact transformation programme at a pivotal moment. You'll play a key role in designing and building robust, scalable data pipelines that bring together data from … across the organisation - including CRM, HR, Finance, and Commercial systems - into clean, trusted, and AI-ready datasets. You'll be part of a growing data team, working closely with architects, analysts, and business stakeholders to unlock insights and enable innovation. What You'll Be Doing: Design and build robust, automated data pipelines using Azure Data More ❯

a range of benefits to support your finances, wellbeing and family. Working Pattern: Permanent Full Time Flexible First options available The Role We're looking for a Senior Data Engineer to join the Asset Data & Insights value stream. You'll lead the design, development and operations of robust, scalable data pipelines using modern cloud … technologies-empowering real-time analytics, advanced BI, and data-driven decision-making across SSE Thermal. You'll need hands-on expertise with the Databricks ecosystem, a strong command of data engineering best practices, and a growth mindset that balances technical excellence with collaboration and continuous improvement. Experience in an asset-intensive or energy environment is desirable … Delta Live Tables, Unity Catalog) to serve downstream analytics and reporting. - Optimise data architecture for performance, scalability, and reliability, proactively monitoring data health and troubleshooting pipeline issues. - Mentor junior engineers and foster a knowledge-sharing culture through documentation, pairing, and architectural guidance. - Collaborate with analysts, product teams, and business stakeholders to translate requirements into performant More ❯

and a range of benefits to support your finances, wellbeing and family. Working Pattern: Permanent Full Time Flexible First options available The Role Were looking for a Senior Data Engineer to join the Asset Data & Insights value stream. Youll lead the design, development and operations of robust, scalable data pipelines using modern cloud technologies … empowering real-time analytics, advanced BI, and data-driven decision-making across SSE Thermal. Youll need hands-on expertise with the Databricks ecosystem, a strong command of data engineering best practices, and a growth mindset that balances technical excellence with collaboration and continuous improvement. Experience in an asset-intensive or energy environment is desirable, as is … Delta Live Tables, Unity Catalog) to serve downstream analytics and reporting. - Optimise data architecture for performance, scalability, and reliability, proactively monitoring data health and troubleshooting pipeline issues. - Mentor junior engineers and foster a knowledge-sharing culture through documentation, pairing, and architectural guidance. - Collaborate with analysts, product teams, and business stakeholders to translate requirements into performant More ❯

Data Engineers - Permanent Opportunity Location Reading: 1-2 days per week A major UK organization is expanding their Data Engineering team to drive forward their ambitious data platform transformation initiative. Are you an experienced Data Engineer ready to tackle complex data challenges at scale? This established organization is building their … next-generation Enterprise Data Platform and needs skilled professionals to accelerate their journey toward becoming a data-centric organization. About Our Client Our client operates critical infrastructure serving millions of customers across the UK. They're investing significantly in modernizing their data capabilities, creating a centralized data lakehouse architecture that will transform … Help establish and maintain enterprise-wide data governance standards Core Technical Requirements of the Data Engineer Azure Data Factory mastery - demonstrable expertise in pipeline development and management Proficient SQL and/or Python capabilities - minimum 2-3 years hands-on experience Comprehensive Data Engineering background - proven track record in enterprise data More ❯

Senior Data Engineer - London/Hertfordshire - Hybrid Are you a technically strong Azure Data Engineer looking to join a forward-thinking, innovative company? Here's your chance to work with a leading construction and infrastructure company who foster a culture of collaboration, support, and strong communication. You'll be responsible for ensuring seamless integration and optimization … of diverse business data streams - from CRM, finance, procurement to fleet and commercial data - enabling reliable, high-quality data for analytics and business insights. While this role won't be client-facing, strong communication and collaboration skills within their agile team are essential. Requirements: Experience orchestrating data pipelines with Azure Data Factory (ADF) Proficiency managing API data ingestion and orchestration using Azure Databricks Working with Azure Data Lake (ADL) Storage, on-prem SQL, and Python Vast Azure data engineering experience Proficient in ADF, Databricks, ADL, SQL, Python Desirable: Exposure to D365 or similar CRM environments Azure Synapse experience Power BI knowledge Benefits: Salary More ❯

The AI Engineer is a key role in the Data Platform Portfolio team, building the data platform, driving value from data across the business. With strong technical skills and business acumen to help turn millions of potential data points into actionable insights that can drive product improvements, make our customer acquisition more … efficient, improve our customer retention rates, and drive operating efficiencies across the business. The primary goals of the team are: To build and run a data platform which can create and deliver analytics to colleagues and deliver regulatory reporting Ingest and transform data from multiple systems, modelling data and engineering data marts … outliers. Proficient in designing and implementing ELT pipelines using tools like dbt, with strong knowledge of data warehousing, data lake concepts, and data pipeline optimization. Skilled in SQL for data manipulation, analysis, query optimisation, and database design. Artificial Intelligence and Machine Learning Understanding of machine learning algorithms (classification, regression, clustering) and More ❯

Senior Machine Learning Operations Engineer Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: £65,000 - £75,000 - depending on experience Department: Technology and Data We're 1st Central, a market-leading insurance company utilising smart data and technology at pace. Rapid growth has been based on giving our 1.4 million customers exactly what they … our colleagues too; we won Insurance Employer of the Year at the British Insurance Awards 2024 and our Glassdoor score is pretty mega too! We're big on data: it gives us the insights we need to give the right cover to the right customers at the right price. But it's the people inside and outside our … Deploying Infrastructure as Code (IAC) across various environments such as dev, uat and prod Handling large volumes of data in various stages of the data pipeline, from ingestion to processing Proven experience with feature stores, using them for both offline model development and online production usage Building integrations between cloud-based systems using APIs, specifically More ❯

to achieve this for a 7th time across the coming period. Due to these increasing levels of high activity, they are generating more and more customer and performance data than ever before and are embarking on a key project to best manage & utilise this. An agile and robust data management and utilisation plan will be key … e.g., star/snowflake schemas) Administer Microsoft Fabric Lakehouse and Azure services Optimise performance across queries, datasets, and pipelines Apply data validation, cleansing, and standardisation rules Document pipeline logic and contribute to business-wide metadata catalogues Collaborate with BI, Change, and Product teams to align data outputs to needs Support groundwork for future data … requirements clearly across business teams will be key to success in this role. You’ll be a technically strong data professional with hands-on experience in pipeline development, cloud-based data infrastructure, and performance optimisation. You will be at your best when working collaboratively in fast-paced environments and when translating complex business needs More ❯

Join Ten10 as a Software Development Engineer in Test (Data Automation Focus) Job Type: Permanent Location: London (Hybrid working with occasional client travel) Passionate about data and automation? At Ten10, were at the forefront of helping organisations harness the power of data through quality engineering and innovative test solutions. Were looking for a Software … Development Engineer in Test (SDET) with a strong focus on data automation to join our dynamic team. If you're driven by complex data challenges and thrive in high-performance environments where ETL testing has become second nature, this is the role for you. ?? Important: All applicants must be eligible for SC clearance Why Ten10? At … one of the UKs leading independent Quality Engineering, Software Testing, and DevOps consultancies. We empower top-tier organisations to accelerate software delivery while maintaining confidence in qualityespecially in data-driven environments. Heres why youll love working with us: Data-Driven Projects: Collaborate on exciting and complex data pipelines, platforms, and automation solutions across industries More ❯

Principal Data Engineer Up to £107,500 + Bonus + Benefits London - Hybrid Are you a hands-on data engineering leader who thrives on building robust platforms and driving technical excellence? I'm partnering with a well-loved consumer tech business that brings joy and storytelling into people's lives-and they're now scaling their … data and ML platform to support the next phase of growth. This is a chance to be at the forefront of a data transformation journey-owning architectural direction, mentoring engineers, and shaping a decentralised data model that will serve marketing, product, finance, and AI functions across the group. Why Join? Up to … base salary Hybrid working with a London base (flexibility offered) High-impact role with a growing, values-driven data team Platform-focused, mission-led engineering Work with a modern cloud-native stack (Snowflake, DBT, Airflow, Terraform, AWS) What You'll Be Doing Serve as the technical lead for cross-functional data initiatives Define and champion best More ❯

Job Description: We are seeking a highly skilled Data Architect for Gen AI Engineering to lead the design and implementation of robust data architectures that support our generative AI initiatives across various business functions, including Marketing, Product Development, and Customer Insights. This role is transformation-funded and therefore has an expected end date of 31st December … 2026. What are we looking for? A seasoned data architect with 8+ years of experience in data architecture and engineering, specifically in the context of generative AI and machine learning applications. Strong expertise in designing scalable data models and architectures, with proficiency in … cloud platforms such as AWS, GCP, or Azure, and familiarity with data warehousing solutions. Experience with data integration tools, ETL processes, and data pipeline development, along with a solid understanding of data governance and security best practices. Proven leadership skills in managing cross-functional teams, driving data strategy, and More ❯

Are you an experienced Data Engineer looking to make a real impact in a high-growth, data-driven environment? We're exclusively partnered with a major UK construction and infrastructure company to find a technically strong and collaborative Data Engineer to join their team. Why This Role? This is a chance to shape the … future of data engineering within a business that's investing heavily in its data strategy. With a newly appointed Head of Data and a strong architectural lead in place, you'll be working side-by-side with other data professionals and senior stakeholders to build scalable, modern data solutions that … either in their Central London office or in their Hertfordshire office to collaborate with your team and business stakeholders. What You'll Be Doing: Designing and orchestrating robust data pipelines using Azure Data Factory and Databricks. Ingesting and transforming data from diverse sources including CRM (D365), finance, procurement, and fleet systems. Collaborating with a More ❯

etc Annual Leave 23 days rising to 27 with length of service Sick Pay Increasing with length of service The Role: Senior AI Engineer Overview of role AI Data Engineering: Design, build, operate, and deploy real-time data pipelines at scale using AI techniques and best practices. Support AI R&D efforts by applying advanced data warehousing, data science, and data engineering technologies. Aim for automation to enable a faster time-to-market and better reusability of new AI initiatives. Collaboration: Work in tandem with the AI Product Owner and other team members to create, curate, and maintain high-quality AI assets. Ensure alignment of data architecture and … outliers. Proficient in designing and implementing ELT pipelines using tools like dbt, with strong knowledge of data warehousing, data lake concepts, and data pipeline optimization. Skilled in SQL for data manipulation, analysis, query optimisation, and database design. Artificial Intelligence and Machine Learning Understanding of machine learning algorithms (classification, regression, clustering) and More ❯

Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: from £70,000 depending on experience Department: Technology and Data We're 1st Central, a market-leading insurance company utilising smart data and technology at pace. Rapid growth has been based on giving our 1.4 million customers exactly what they want: great value insurance with an … design, through development and testing, to deployment, production, monitoring and continuous improvement optimising cutting-edge AI solutions across the business. One day you might be collaborating with our Data Scientists, Software Engineers and Architects on a new GenAI capability. The next, you're fine-tuning prompts or experimenting with Computer Vision and Natural Language Processing (NLP). You … control, and continuous integration. Develop and maintain automated frameworks for testing, validation, and monitoring of AI model accuracy, performance, and reliability. Establish best practices for AI data pipeline development and integration with infrastructure to support scalable deployment. Collaborate closely with Data Scientists, Data Engineers, Software Developers, and Architects for seamless execution of AI More ❯

Senior Data Engineer| AWS/Databricks/PySpark | London/Glasgow (Hybrid) | August Start Role: Senior Data Engineer Location: This is a hybrid engagement represented by 2 days/week onsite, either in Central London or Glasgow. Start Date: Must be able to start mid-August. Salary: £80k-£90k (Senior) | £90k-£95k (Lead) About The Role … Our partner is looking for a Senior Data Engineer to join a high-impact engineering team delivering scalable data solutions for complex marketing and customer insight use cases. This is an opportunity to work on cutting-edge data pipelines, cloud-native platforms and real-time data flows in a collaborative, forward-thinking … with cross-functional teams including Data Science, Software Engineering and Product. Champion best practices in data governance, observability and compliance. Contribute to CI/CD pipeline development and infrastructure automation (Terraform, AWS DevOps). Provide input into technical decisions, peer reviews and solution design. Requirements Proven experience as a Data Engineer in cloud More ❯