1 to 25 of 387 Data Pipeline Jobs in the South East

engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities: Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources … effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote ...

Location: London, Potters Bar or Bristol (Hybrid working options available) The Data Engineer is a hands-on technical role responsible for designing, developing, and maintaining data pipelines within the IT department. The pipelines will be realised in a modern lake environment and the engineer will collaborate … guiding and upskilling more junior data engineers and setting data standards and guidelines. What you'll do Data Pipeline Development Design, model, develop and maintain data pipelines to ingest, store, process, and present data. Ensure data quality, accuracy ...

Robert Half Technology are assisting a market leading AI-driven data platform organisation to recruit a Data Platform Engineer paying c£125 - £150k per annum. Remote working (UK based). Role The Data Platform Engineer will be an expert level engineer within Snowflake … build scalable solutions within the Data Platform Perform root cause analysis on often complex errors to help ensure data pipeline availability Help drive technical & architectural decisions on the data platform including decisions on data architecture, data engineering processes, data ...

Role Overview We are seeking a Geospatial Data Pipeline Engineer to design, build, and operate scalable, secure, and auditable data pipelines for high-volume geospatial and spatial attribute datasets. This role focuses on end-to-end pipeline engineering—from data ingestion … Analysts, QA teams, Automation Engineers, and Programme Managers to ensure reliable, high-quality data delivery at scale. Key Responsibilities Data Pipeline Design & Engineering Design, implement, and maintain end-to-end geospatial data pipelines Support ingestion of: o Spatial datasets (point, line, polygon ...

developers (quants and strategists) in a unique hands-on opportunity within a focused, high-impact team. You'll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with … leading global Investment Banks' front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We're Looking For 5 + ...

unique opportunity to remain hands-on in engineering while also leading a focused, high-impact team. You'll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with … leading global Investment Banks' front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We're Looking For 10 + ...

Data Engineer £Up to £65,000 GBP Hybrid WORKING Location: London; Norwich; Watford; Colchester; Chelmsford; Woking; Chatham; Slough, Central London, Greater London - United Kingdom Type: Permanent Must Have: Active SC Clearance Join a world-class organisation delivering mission-critical data solutions for Defence, National Security … architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data ...

Data Engineer £Up to £65,000 GBP Hybrid WORKING Location: London; Norwich; Watford; Colchester; Chelmsford; Woking; Chatham; Slough, Central London, Greater London - United Kingdom Type: Permanent Must Have: Active SC Clearance Join a world-class organisation delivering mission-critical data solutions for Defence, National Security … architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data ...

Data Engineer £Up to £65,000 GBP Hybrid WORKING Location: London; Norwich; Watford; Colchester; Chelmsford; Woking; Chatham; Slough, Central London, Greater London - United Kingdom Type: Permanent Must Have: Active SC Clearance Join a world-class organisation delivering mission-critical data solutions for Defence, National Security … architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data ...

Data Engineer £Up to £65,000 GBP Hybrid WORKING Location: London; Norwich; Watford; Colchester; Chelmsford; Woking; Chatham; Slough, Central London, Greater London - United Kingdom Type: Permanent Must Have: Active SC Clearance Join a world-class organisation delivering mission-critical data solutions for Defence, National Security … architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data ...

Data Engineer £Up to £65,000 GBP Hybrid WORKING Location: London; Norwich; Watford; Colchester; Chelmsford; Woking; Chatham; Slough, Central London, Greater London - United Kingdom Type: Permanent Must Have: Active SC Clearance Join a world-class organisation delivering mission-critical data solutions for Defence, National Security … architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data ...

Data Engineer £Up to £65,000 GBP Hybrid WORKING Location: London; Norwich; Watford; Colchester; Chelmsford; Woking; Chatham; Slough, Central London, Greater London - United Kingdom Type: Permanent Must Have: Active SC Clearance Join a world-class organisation delivering mission-critical data solutions for Defence, National Security … architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data ...

Data Engineer (Inside IR35) £425 Per day SC Cleared - A must to be considered for this opportunity London To support the formation of a new Data Team, there is a critical need to establish robust, scalable, and secure data infrastructure. Currently, data … performance, cost efficiency, and maintainability using modern data-engineering tools and platforms (e.g., Azure Data Factory, AWS Data Pipeline, Databricks, Apache Spark). Support the integration of data into visualisation platforms and analytical environments (e.g., Power BI, ServiceNow). Ensure adherence ...

Agile delivery environment, working closely with the Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from our wide-ranging dataset to support executive and operational decision making that will underpin sustained growth … management responsibilities for a small team of Data Engineers, who you will coach, support and organise to ensure we sustain a predictable pipeline of delivery, whilst ensuring all appropriate governance and best practice is adhered to. Your responsibilities as our Senior Databricks Engineer will include: Understand ...

Data Engineer - £350PD - Remote Required Technical Skills Data Pipeline & ETL Design, build, and maintain robust ETL/ELT pipelines for structured and unstructured data Hands-on experience with AWS Glue and AWS Step Functions Implementation of data validation, data quality frameworks, and reconciliation checks Strong error handling, monitoring, and retry strategies in production pipelines Experience with incremental data processing patterns (CDC, watermarking, upserts) AWS Data Services Amazon S3: data lake architectures, partitioning strategies, lifecycle policies DynamoDB: data modeling, secondary ...

seeking an experienced Data Architect to join our Group Technology team in Milton Keynes. You will help realise the value of the immense Connells Group dataset for the benefit of customers, colleagues and the overall business. You will use your expertise to define, evolve and communicate the existing … enterprise data architecture, to democratise safe access to data to help engineering teams create personal, relevant and timely customer experiences. The data architecture will also support colleagues in accessing data, getting insights and providing next best actions for faster and aligned decision ...

engineering mindset who enjoys improving data reliability, performance, and usability in a fast-moving fintech environment. Key Responsibilities Data Platform & Pipeline Engineering Design, build, and maintain scalable data pipelines to ingest, transform, and serve data from multiple internal and external sources. … cases. Orchestration & Workflow Management Develop and maintain data orchestration workflows to manage scheduled and event-driven pipelines. Improve monitoring and alerting around pipeline failures and data quality issues. Support deployment and operational best practices across the data platform. Collaboration & Delivery Work closely with ...

LatentView Analytics is a leading global analytics and decision sciences provider, delivering solutions that help companies drive digital transformation and use data to gain a competitive advantage. With analytics solutions that provide a 360-degree view of the digital consumer, fuel machine learning capabilities and support artificial intelligence … technical and non-technical audiences A proactive, growth-oriented mindset, passionate about improving data accessibility and scalability Responsibility: Data Engineering & Pipeline Development: Design, build, and maintain ETL/ELT data pipelines to consolidate and transform customer success data from multiple international ...

Title: Data Architect (Insurance Domain) Location: London UK Experience Level 15 years with at least 5 years in Azure ecosystem Role: We are seeking a seasoned Data Architect to lead the design and implementation of scalable data solutions for a strategic insurance project … ideal candidate will have deep expertise in Azure cloud services Azure Data Factory and Databricks with a strong understanding of data modeling data integration and analytics in the insurance domain Key Responsibilities · Architect and design end to end data solutions on Azure ...

Lead Data Engineer Lead Data Engineer required to join my Client’s Technology Team and contribute to their Group Architecture Board in building the next generation of their eCommerce data platform. You will play a pivotal role in designing and maintaining a modern, cloud … develop robust data models, ETL/ELT processes, and data marts that support complex analytics and operational use cases. Pipeline Optimization: Maintain and optimize data pipelines and warehouse environments to ensure top-tier performance, reliability, and scalability. Cross-Functional Collaboration: Effectively collaborate with ...

governance, and security across all platforms. Troubleshoot and enhance existing pipelines to improve scalability and resilience. Sample Projects You Work On Financial Asset Management Pipeline: Build and manage data ingestion from third-party APIs, model data using dbt, and support machine learning workflows for asset … production environments, emphasizing performance optimization and code quality. Python expertise: Advanced proficiency in Python for data engineering, data wrangling and pipeline development. Cloud Platforms: Hands-on experience working with Azure. AWS experience is considered, however Azure exposure is essential. Data Warehousing: Proven expertise ...

About The Role Team – Data Engineering Working Pattern - Hybrid – 2 days per week in the Vitality Bournemouth or London Office. Full time, 37.5 hours per week. We are happy to discuss flexible working! Top 3 skills needed for this role: Designing and delivering scalable data pipelines … with SQL Deploying and maintaining machine learning models as APIs in production environments What this role is all about: Join us as a Data Engineer and help build the data foundations that power machine learning and analytics across Vitality. You’ll work with rich datasets ...

Data Engineer £250 Outside IR35 - Remote Job Summary We are seeking a skilled Azure Data Engineer to design, build, and maintain scalable data solutions on Microsoft Azure. The ideal candidate has strong hands-on experience with Azure Databricks and Azure Synapse Analytics … stakeholders to deliver data solutions Implement security, governance, and compliance best practices (RBAC, data masking, encryption) Monitor, troubleshoot, and resolve pipeline and performance issues Document data architecture, pipelines, and operational processes Required Qualifications 3+ years of experience as a Data Engineer ...

Data Architect - Fully Remote role Data Architect sought by a well-established Data & Analytics consultancy. Our client is seeking a cloud professional to spearhead strategic leadership and technical expertise within their data solutions landscape, focussed on cloud data platforms … premise design expertise. At least 5 years in data engineering or business intelligence roles. Proficiency in ETL and data pipeline design, with a technology-agnostic approach. A solid understanding of data warehouse and data lake principles. Expert SQL skills and demonstrable ...

Lead Data Engineer (AWS Snowflake Python) London/WFH to £110k Are you a technologist Data Engineer with strong technical leadership skills? You could be progressing your career in an impactful, hands-on Lead Data Engineer role at a growing technology company as they … expand their UK presence. As a Lead Data Engineer you'll take responsibility for the team's engineering practices such as code reviews and CI/CD and for overall code quality and test coverage. You'll plan and manage the team's Sprints, collaborating closely with product ...