1 to 25 of 85 Remote Data Pipeline Jobs in the South East

Databricks Architect (Contract) - Greenfield Data Platform Location: Hybrid working (London) Duration: 12-month initial contract Are you a visionary Databricks Architect with a passion for building cutting-edge data platforms from the ground up? Do you thrive on shaping strategy and driving technical excellence in a greenfield environment? Our client is embarking on a pivotal journey … to establish a brand-new, enterprise-grade data platform using the full power of Databricks. This is a unique opportunity to lead the architectural design and implementation of a truly greenfield data ecosystem that will underpin all future data-driven initiatives, from advanced analytics to AI/ML. We are looking for a hands … who can translate business needs into robust, scalable, and secure Databricks solutions. The Role: As our Databricks Architect, you will be instrumental in defining and delivering our new data strategy and architecture. This is a greenfield project, meaning you'll have the exciting challenge of building the entire Databricks Lakehouse Platform from scratch. You will provide critical technical More ❯

To Work. This prestigious award is based entirely on what current employees say about their experience working at Circana. Job description: We are seeking a skilled and motivated Data Engineer to join a growing global team. In this role, you will be responsible for designing, building, and maintaining robust data pipelines and infrastructure on the Azure … cloud platform. You will leverage your expertise in PySpark, Apache Spark, and Apache Airflow to process and orchestrate large-scale data workloads, ensuring data quality, efficiency, and … scalability. If you have a passion for data engineering and a desire to make a significant impact, we encourage you to apply! Job Responsibilities ETL/ELT Pipeline Development: Design, develop, and optimize efficient and scalable ETL/ELT pipelines using Python, PySpark, and Apache Airflow. Implement batch and real-time data processing solutions using More ❯

Data Engineer/Scientist Department: Consultancy Employment Type: Full Time Location: United Kingdom/Hybrid Description As a Data Engineer/Scientist at Actica, you will have the opportunity to design, build, and maintain data pipelines while developing advanced analytics solutions to unlock business problems for high-profile UK public sector organisations. Your expertise … will enable organisations to maximise the value of their data assets through robust data infrastructure and sophisticated analysis, playing a key role in nationally critical projects that make a real difference to people's everyday lives. Locations: London, Guildford, Bristol, M4 corridor Hybrid working Roles and Responsibilities Actica recognises that data engineering, analytics, and … data science are distinct but interconnected disciplines. Our data engineers focus on building and maintaining the data infrastructure that enables analytics and data science work, while our data scientists and analysts focus on deriving insights and developing models. At Actica, we're at the forefront of the UK government's More ❯

To Work. This prestigious award is based entirely on what current employees say about their experience working at Circana. Job description: We are seeking a skilled and motivated Data Engineer to join a growing team Global Team based in the UK. In this role, you will be responsible for designing, building, and maintaining robust data pipelines … and infrastructure on the Azure cloud platform. You will leverage your expertise in PySpark, Apache Spark, and Apache Airflow to process and orchestrate large-scale data workloads, ensuring data quality, efficiency, and scalability. If you have … a passion for data engineering and a desire to make a significant impact, we encourage you to apply! Job Responsibilities Data Engineering & Data Pipeline Development Design, develop, and optimize scalable DATA workflows using Python, PySpark, and Airflow Implement real-time and batch data processing using Spark Enforce best practices More ❯

We are supporting a fast-growing, high-performing investment firm as they expand their lean, agile technology team. This newly created role sits at the intersection of data engineering and business intelligence — offering real ownership, front-to-back exposure, and close interaction with the business. The team is small, collaborative, and strategically embedded within the firm’s data … opportunity to have impact from day one while helping shape the future of reporting and analytics. Key Responsibilities: Maintain and extend ETL pipelines and support the firm’s data warehouse infrastructure. Work closely with business teams to understand data and reporting needs. Develop and enhance Power BI dashboards and reporting solutions. Assist with modernising and automating … legacy processes. Play a key role in the design, testing, and deployment of new data solutions. Technology Stack: Azure SQL, SSIS, Power BI, PowerShell C# (used for testing automation) Excel/Power Query CI/CD tooling What We’re Looking For: A hands-on data engineer or BI engineer with solid SQL skills and experience More ❯

Partners, senior staff, and Oaktree Capital. Find out more about us by visiting our website at: http://www.hartreepartners.com/ROLE OVERVIEW: Hartree Partners is growing its data-driven analytics team and is hiring a Statistical Data Scientist to sharpen our view of weather-driven risk and support wider supply-and-demand modelling for power … gas trading. You will play a pivotal role in shaping our data strategy and driving the development of our modelling approaches. You will collaborate with cross-functional teams to design, implement, and optimize data pipelines and predictive models that inform trading decisions and enhance operational efficiency. RESPONSIBILITIES: Primary Focus: Probabilistic Weather Modelling: Research and prototype probabilistic … inference, state-space filtering, change-detection tests, etc.) that flag when fresh weather guidance materially diverges from prior outlooks. Continuous Development and Improvement: Calibrate confidence metrics with historical data and measure their value to trading & risk learning models. Continuously improve these models based on real-time data and feedback from trading activities. Model Communication: Explain uncertainty More ❯

twentyAI are partnering with a globally renowned law firm currently undergoing a major digital and data transformation. With a deep-rooted legacy in legal excellence and a strong global footprint, the firm is currently modernising their data capability and building a new data platform. About the Role You will be part of a diverse … group of engineers and machine learning experts, working with cutting-edge Azure cloud technologies, including Microsoft Fabric and related services. Your mission is to design reliable, efficient data pipelines that enable the business to access trusted, well-structured data. You will also focus on building and scaling the core data infrastructure that supports advanced analytics and … machine learning efforts across the business. Responsibilities Design and develop end-to-end data pipelines that ingest, transform, and prepare data for analytics and machine learning workflows. Work with Infrastructure as Code, primarily Terraform, to automate and manage cloud infrastructure, enabling repeatable and reliable deployment processes. Collaborate closely with data scientists, MLEs, and business More ❯

Join a Leading Growth business on its transformative journey to become the UK's most recommended businesses in their industry Lead Data Engineer Salary: £70-90k Location: Reading – Hybrid Benefits: Competitive Role Overview: As Lead Data Engineer , you will be part of the Corporate Data team, reporting directly to the Senior Leadership team. … You will lead a small team of data engineers, overseeing the design, build, and maintenance of the infrastructure and systems that collect, store, and process corporate data for reporting and analysis. Key Responsibilities: • Lead and manage a team of data engineers. • Design, build, and maintain data infrastructure and systems. • Ensure effective use … of Microsoft Fabric and Synapse analytics platform. • Collaborate with various stakeholders, including Data and Systems owners, Data Analysts, Data Architects, Infrastructure Architects, and external suppliers. Experience required • Dynamic and hands-on leader who excels in managing and coaching a team. • Thrives in a collaborative environment and adapts to changing priorities with ease. • Excellent communication More ❯

join their team. They are one of the leaders within the Insurance space. They are currently going through a period of growth and are looking for an experienced Data Engineer to join their team. They are backed by a huge Multi National equity firm who are looking to bolster my client's financial position. They are expected to … and utilise the most in-demand and cutting-edge technology on the market right now. Your role will include: Collaborate closely with stakeholders to understand and interpret their data and analytics needs. Design and deploy end-to-end data solutions using Microsoft Fabric, encompassing data ingestion, transformation, and visualisation workflows. Construct and refine data models, pipelines, and ETL frameworks within the Fabric ecosystem. Leverage Fabric’s suite of tools to build dynamic reports, dashboards, and analytical applications. Maintain high standards of data integrity, consistency, and system performance across the data environment. Advise on data governance, security protocols, and compliance strategies aligned with industry best practices. Diagnose and More ❯

The AI Engineer is a key role in the Data Platform Portfolio team, building the data platform, driving value from data across the business. With strong technical skills and business acumen to help turn millions of potential data points into actionable insights that can drive product improvements, make our customer acquisition more … efficient, improve our customer retention rates, and drive operating efficiencies across the business. The primary goals of the team are: To build and run a data platform which can create and deliver analytics to colleagues and deliver regulatory reporting Ingest and transform data from multiple systems, modelling data and engineering data marts … outliers. Proficient in designing and implementing ELT pipelines using tools like dbt, with strong knowledge of data warehousing, data lake concepts, and data pipeline optimization. Skilled in SQL for data manipulation, analysis, query optimisation, and database design. Artificial Intelligence and Machine Learning Understanding of machine learning algorithms (classification, regression, clustering) and More ❯

Senior Machine Learning Operations Engineer Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: £65,000 - £75,000 - depending on experience Department: Technology and Data We're 1st Central, a market-leading insurance company utilising smart data and technology at pace. Rapid growth has been based on giving our 1.4 million customers exactly what they … our colleagues too; we won Insurance Employer of the Year at the British Insurance Awards 2024 and our Glassdoor score is pretty mega too! We're big on data: it gives us the insights we need to give the right cover to the right customers at the right price. But it's the people inside and outside our … Deploying Infrastructure as Code (IAC) across various environments such as dev, uat and prod Handling large volumes of data in various stages of the data pipeline, from ingestion to processing Proven experience with feature stores, using them for both offline model development and online production usage Building integrations between cloud-based systems using APIs, specifically More ❯

Senior Data Engineer | Python, AWS, AI/ML | Data-Driven Insights Company My client is a leading provider of intelligence and data solutions to major players in the asset management industry. With a workforce of over 600 employees, they are currently expanding their tech and product offerings during a period of rapid growth. As … a Senior Data Engineer, you will be a key member of their automation team. You will play a central role in integrating cutting-edge technology with strategic data solutions. This newly created role reports directly to the Head of Data Science and Engineering. You will have the opportunity to work in a team of … Data, DevOps, and Software engineers, guiding projects in the direction you believe will ensure success. My client is based in St. Pauls and they operate a hybrid working model expecting people in the office 2 times a week. The company culture is energetic and social with daily complimentary breakfast, all-day drinks, and a calendar full of social More ❯

Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: from £70,000 depending on experience Department: Technology and Data We're 1st Central, a market-leading insurance company utilising smart data and technology at pace. Rapid growth has been based on giving our 1.4 million customers exactly what they want: great value insurance with an … design, through development and testing, to deployment, production, monitoring and continuous improvement optimising cutting-edge AI solutions across the business. One day you might be collaborating with our Data Scientists, Software Engineers and Architects on a new GenAI capability. The next, you're fine-tuning prompts or experimenting with Computer Vision and Natural Language Processing (NLP). You … control, and continuous integration. Develop and maintain automated frameworks for testing, validation, and monitoring of AI model accuracy, performance, and reliability. Establish best practices for AI data pipeline development and integration with infrastructure to support scalable deployment. Collaborate closely with Data Scientists, Data Engineers, Software Developers, and Architects for seamless execution of AI More ❯

for musicians and artists to use to promote their music. At un:hurd music, we've created a music-tech platform that uses proprietary technology to turn music data into stronger promotional campaigns for artists. We've created both mobile and web applications which act as the 'digital marketing companion' for independent artists, facilitating data-led … responsible for the software development and technical infrastructure needed to design, train, deploy and scale machine learning models in production environments. You will harness a diverse array of data sources, from streaming services to social media, revolutionizing the way artists promote their music. Our Data Team members are full stack data professionals who independently … manage projects across the complete spectrum, from raw data ingestion to deployment, monitoring, and optimization. While this role emphasizes data engineering and ML deployment, you will have the responsibility of managing projects with autonomy across the full data lifecycle. Responsibilities As an ML Engineer at Un:hurd music, you'll be: MLOps and Model More ❯

Job Title: Data Engineer (with AI Engineering Experience) Location: Hybrid (UK-based) Outside IR35 Contract role - 6 months rolling Industry: Energy/Wholesale Markets & Trading We’re looking for a skilled Data Engineer with Artificial Intelligence (AI) experience to join a cutting-edge team within one of the UK’s leading energy companies. This role sits … within the organisation’s Wholesale Market Services and Trading division, a key area focused on enabling smarter, data-driven decision-making in fast-paced energy markets. You’ll play a pivotal role in designing and delivering complex, cloud-native data solutions that support trading operations, risk management, market forecasting, and AI-led optimisation initiatives. Key Responsibilities … Design, build, and maintain robust data pipelines and cloud-based infrastructure to support large-scale data processing and analytics workloads. Collaborate with data scientists and AI engineers to productionise machine learning models and support intelligent automation solutions. Contribute to the development and enhancement of the enterprise data platform with scalable, high-quality More ❯

Senior Data & AI Consultant Homebased - Must be happy to travel to customer sites and London Full-time, Permanent Eligible for SC Clearance/Actively SC Cleared Be part of a growing consulting team with a leading Microsoft Partner. This role is for a Senior Azure Data & AI Consultant, who will deliver high-impact solutions that transform … You’ll play a critical role in guiding organisations through the AI journey — from identifying opportunities to architecting and deploying real-world solutions using modern cloud platforms and data technologies. You’ll work in a dynamic, fast-paced environment alongside experienced consultants and technical teams, helping clients unlock the full value of AI through innovation, collaboration, and expertise. … What You'll Do Lead and deliver complex AI and data projects using Azure Data Services , Microsoft Fabric , and other modern data platforms Work closely with clients to understand business challenges and identify where AI and data-driven solutions can drive value Architect and implement scalable, production-ready solutions using machine learning More ❯

Data Engineer - Python, SQL, DBT, DWH, Airflow, Kafka, Snowflake - Challenger Digital Bank Location: London (Hybrid, 2 days per week in office) Type: Permanent Salary: Competitive + 20% bonus + excellent benefits Are you a Data Engineer ready to shape the future of a digital-first banking experience? Join a fast-growing start-up challenger bank where … data is at the core of every decision. You’ll play a key role in building the bank’s next-generation data infrastructure and enable insight-led growth from the ground up. What you will be doing: Build and optimise robust data pipelines using Python , Kafka , Airflow , and DBT Design and manage Snowflake data … Airflow, and modern ELT/ETL frameworks Familiarity with data visualisation tools like Sisense, Looker, or Tableau Solid understanding of data architecture, transformation workflows, and pipeline orchestration Clear communicator who can collaborate with both technical and non-technical stakeholders Comfortable working in a fast-paced, agile and product-focused environment Why join us? Shape the More ❯

Principal Data Engineer – AWS Location : London (Hybrid) Salary : £105,000 Job Type : Permanent A top-tier digital consultancy is making a key strategic hire at the Associate Director level for a hands-on, stakeholder-savvy Principal Data Engineer. You will take the helm of public and private sector data transformation programmes, drive technical delivery … shape a growing practice at scale. This is a greenfield opportunity for a high-impact leader with deep AWS expertise and a consulting mindset as an experienced Principal Data Engineer. Key Responsibilities • Lead and deliver enterprise-grade data engineering solutions using core AWS services (S3, Glue, Lambda, Redshift, Matillion, etc.). • Define architecture, mentor large technical … cloud data lake architectures, ETL/ELT patterns, and metadata/data quality management. • Expertise in Matillion, Redshift, Glue, Lambda, DynamoDB, and data pipeline automation. • Familiarity with data visualisation platforms such as Quicksight, Tableau, or Looker. • Knowledge of CI/CD processes and infrastructure-as-code. • Eligible for SC clearance (active More ❯

Data Engineer (Databricks & Azure) - Clean Energy Location: South East England (Hybrid - 1 day onsite per week) Salary: £60k - £70k + benefits package 18 months. That’s all the time it took for the client’s Databricks platform to evolve into a key driver of innovative green technologies. Now, they’re looking for someone to take it even further. … solid oxide platform is transforming energy systems and helping decarbonise emissions-heavy industries like steelmaking and future fuels. What’s the Role? You’ll join a highly skilled data team, part of a broader department focused on modelling and digitalisation. This team develops and maintains a cutting-edge Azure Databricks Data Lakehouse platform to support all … core business functions. Your primary goal will be building and maintaining robust, secure data pipelines and models that deliver trusted datasets to internal and external stakeholders, enabling data-driven decisions across the organisation. As a Data Engineer, you will maintain, monitor, and enhance the Databricks platform that powers the client’s data More ❯

Lead Data Engineer | Location: London (Hybrid - 1-2 day a month in office) | Salary: £90,000-£110,000 | London Market Insurance The Opportunity Are you passionate about building cutting-edge data platforms that drive business growth? Our client is seeking who is a Speciality Insurer a skilled and motivated Lead Data Engineer with Insurance … experience to play a key role in the creation of a brand-new data platform within the Azure ecosystem and Databricks . This is an exciting opportunity to be at the forefront of data innovation, working within a newly formed Data & Analytics team in a long standing London Market Insurer . You’ll help … shape the data strategy, improve data quality, and empower the business to make data-driven decisions. As a Lead Data Engineer, you'll work closely with both technical and business stakeholders, leveraging your expertise to design, develop, and optimize a high-performance data platform built on Databricks . This platform More ❯

Data Engineer (one day a week in the central London office) Sports Analytics This role is a unique opportunity for a Data Engineer to combine technical challenges with creativity in a collaborative, high-standard work environment. By joining this team, you’ll not only be part of a creative and open work culture focused on innovation … it an ideal setting for those passionate about leveraging cutting-edge technology to make meaningful contributions in the world of sports analytics. Key responsibilities for the role of Data Engineer include: Design, construct, install, test, and maintain highly scalable data management systems. Ensure systems meet requirements and industry practices for data quality and integrity. … Integrate data management technologies and software tools into existing structures. Create data tools to support sport data modelling, prediction and analytics. Work with data and analytics experts to strive for greater functionality in our data systems. As the selected Data Engineer, your background will include: 3+ years industry More ❯

Company: My client are a fast-growing fintech transforming how people manage and move money. As a Data Engineer, you’ll play a pivotal role in building and scaling the data infrastructure that powers everything from real-time fraud detection to personalized financial insights. Key Responsibilities: Design, build, and maintain scalable data pipelines (batch … streaming) Work with Data Scientists and Analysts to enable fast, reliable access to data Optimize our data architecture for performance, reliability, and cost Collaborate with engineers to integrate data best practices across teams Champion data quality, governance, and documentation Key Requirements: Strong experience with Python, SQL, and modern ETL tools … e.g., Airflow, dbt) Solid grasp of cloud platforms (AWS/GCP/Azure) and data warehouses (e.g., BigQuery, Snowflake) Familiarity with streaming technologies (Kafka, Kinesis, etc.) Passion for clean, maintainable code and robust data systems Previous experience in a fintech or regulated environment is a plus Benefits: Join a collaborative, mission-driven team at the cutting More ❯

Are you a Data Engineer who is looking for a more purposeful career? Feel like your current role, limits you to what you can achieve and that there isn’t any more room to grow? If the answer is yes, I could have just the role for you. Recruit with Purpose are supporting one the UK’s largest … and leading housing associations to help them find a Data Engineer to join their growing team. This organisation is at the forefront for how housing associations modernise and utilise their data. Constantly evolving and leading the way within the sector. This is a fantastic opportunity for a senior/lead data engineer to come in and … put their stamp on how this organisation utilises its data. In this role, you’ll be at the heart of designing, building, and maintaining trusted, reusable data products that create better decision-making and deeper insights across the organisation. You'll help bring their data strategy to life, opening up access, enhancing quality, strengthening governance, and More ❯

We're looking for a skilled Cloud Engineer with experience in Cloud technologies to collaborate with our Data Platforms Engineering team, on advancing our Enterprise Data Lakehouse platform. If you're passionate about developing scalable data solutions and automating infrastructure, this role is for you. At Nationwide we offer hybrid working wherever possible. More … the advert earlier than the advertised date, so please apply as soon as you can. What you’ll be doing In this role, you'll work in the Data Platforms Engineering team, utilising cutting-edge Cloud technology within Microsoft Azure to enhance our Enterprise Data Lakehouse platform. Using tools such as Databricks, Python, and Terraform , you … ll design and develop key platform functionality and data analytics solutions. Expect a hands-on experience where you'll contribute significantly to our daily operations, making a tangible impact with your expertise. In this position, you'll be at the forefront of ensuring the efficiency and accuracy of our data platforms and processes. Your responsibilities will More ❯

Job Title: Contract Data Architect Location: Reading (Hybrid or On-site as required) Contract Length: 3 Months Sector: Financial Services Overview: We are seeking an experienced Data Architect to support a leading financial services organisation based in Reading. This is a 3-month contract role focused on designing and implementing scalable, secure, and compliant data solutions in a highly regulated environment. Key Responsibilities: Design and deliver end-to-end data architecture solutions to support business intelligence, reporting, and regulatory needs Lead the integration and optimisation of large-scale data platforms using Azure Synapse and Databricks Build and maintain robust data pipelines using Python (PySpark) and SQL Collaborate … with data engineers, analysts, and stakeholders to ensure data quality, governance, and security Ensure all solutions adhere to financial regulations and internal compliance standards Key Skills & Experience: Proven experience as a Data Architect within the financial services sector Hands-on expertise with Azure Synapse Analytics and Databricks Strong programming and data engineering More ❯