1 to 25 of 97 Data Pipeline Jobs in the South East

Location: London, Potters Bar or Bristol (Hybrid working options available) The Data Engineer is a hands-on technical role responsible for designing, developing, and maintaining data pipelines within the IT department. The pipelines will be realised in a modern lake environment and the engineer will collaborate … guiding and upskilling more junior data engineers and setting data standards and guidelines. What you'll do Data Pipeline Development Design, model, develop and maintain data pipelines to ingest, store, process, and present data. Ensure data quality, accuracy ...

Robert Half Technology are assisting a market leading AI-driven data platform organisation to recruit a Data Platform Engineer paying c£125 - £150k per annum. Remote working (UK based). Role The Data Platform Engineer will be an expert level engineer within Snowflake … build scalable solutions within the Data Platform Perform root cause analysis on often complex errors to help ensure data pipeline availability Help drive technical & architectural decisions on the data platform including decisions on data architecture, data engineering processes, data ...

Data Engineer (Inside IR35) £425 Per day SC Cleared London o support the formation of a new Data Team, there is a critical need to establish robust, scalable, and secure data infrastructure. Currently, data is dispersed across multiple systems with inconsistent formats … performance, cost efficiency, and maintainability using modern data-engineering tools and platforms (e.g., Azure Data Factory, AWS Data Pipeline, Databricks, Apache Spark). Support the integration of data into visualisation platforms and analytical environments (e.g., Power BI, ServiceNow). Ensure adherence ...

undertaking one of the largest transformation programmes in government, modernising the digital systems that support over £7 trillion of property ownership. As a Data Engineer, you will support the development of HMLR's data engineering capability by helping to build and maintain reliable data … and analysing source systems to produce clear, actionable insights. Desirable Skills Knowledge of DevOps processes: (e.g. Terraform) Knowledge of data pipeline testing (e.g. end-to-end testing, data quality testing, monitoring & alerting, unit & contract testing) Knowledge of the data lifecycle (e.g. ...

Agile delivery environment, working closely with the Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from our wide-ranging dataset to support executive and operational decision making that will underpin sustained growth … management responsibilities for a small team of Data Engineers, who you will coach, support and organise to ensure we sustain a predictable pipeline of delivery, whilst ensuring all appropriate governance and best practice is adhered to. Your responsibilities as our Senior Databricks Engineer will include: Understand ...

Data Engineer - £350PD - Remote Required Technical Skills Data Pipeline & ETL Design, build, and maintain robust ETL/ELT pipelines for structured and unstructured data Hands-on experience with AWS Glue and AWS Step Functions Implementation of data validation, data quality frameworks, and reconciliation checks Strong error handling, monitoring, and retry strategies in production pipelines Experience with incremental data processing patterns (CDC, watermarking, upserts) AWS Data Services Amazon S3: data lake architectures, partitioning strategies, lifecycle policies DynamoDB: data modeling, secondary ...

Description We are seeking an experienced Data Architect to join our Group Technology team in Milton Keynes. You will help realise the value of the immense Connells Group dataset for the benefit of customers, colleagues and the overall business. You will use your expertise to define, evolve … communicate the existing enterprise data architecture, to democratise safe access to data to help engineering teams create personal, relevant and timely customer experiences. The data architecture will also support colleagues in accessing data, getting insights and providing next best actions for faster ...

Lead Data Engineer Lead Data Engineer required to join my Client’s Technology Team and contribute to their Group Architecture Board in building the next generation of their eCommerce data platform. You will play a pivotal role in designing and maintaining a modern, cloud … develop robust data models, ETL/ELT processes, and data marts that support complex analytics and operational use cases. Pipeline Optimization: Maintain and optimize data pipelines and warehouse environments to ensure top-tier performance, reliability, and scalability. Cross-Functional Collaboration: Effectively collaborate with ...

Adword Job Title: Data Architect (Insurance Domain) Location: London UK Hybrid role Responsibilities: Experience Level 15 years with at least 5 years in Azure ecosystem Role: We are seeking a seasoned Data Architect to lead the design and implementation of scalable data solutions … strategic insurance project The ideal candidate will have deep expertise in Azure cloud services Azure Data Factory and Databricks with a strong understanding of data modeling data integration and analytics in the insurance domain Key Responsibilities Architect and design end to end data ...

Data Engineer £250 Outside IR35 - Remote Job Summary We are seeking a skilled Azure Data Engineer to design, build, and maintain scalable data solutions on Microsoft Azure. The ideal candidate has strong hands-on experience with Azure Databricks and Azure Synapse Analytics … stakeholders to deliver data solutions Implement security, governance, and compliance best practices (RBAC, data masking, encryption) Monitor, troubleshoot, and resolve pipeline and performance issues Document data architecture, pipelines, and operational processes Required Qualifications 3+ years of experience as a Data Engineer ...

have a new opportunity for an IT Data Engineer to join our existing team as part of a new phase of expansion. Whilst we're looking for individuals with previous experience working for a Professional Services organisation, as a medium sized business we operate with a much smaller … multi skilled IT Team than you will find at larger enterprise companies. Although the role is primarily an IT Data Engineer, you will also need to be able to display BI Developer and strong Analytical skills and play a crucial role in designing, building, and maintaining our data ...

summary The Data Test Engineer plays a key role in ensuring the quality, reliability, and performance of the Trust's digital applications, data platforms, and integrated clinical systems, focusing on areas such as Electronic Patient Records (EPR), Clinical Applications, Interoperability, Data Systems, Infrastructure … Digital Assurance. Reporting to the Data Assurance and Quality Lead, the role requires strong expertise in data testing and validation and involves designing, executing, and maintaining manual and automated testing processes, ensuring that all systems meet the highest standards of performance, security, and usability. The Test ...

pipelines and transformation workflows using Databricks and Python. Ensure data platforms support analytics, business intelligence, and emerging GenAI/ML needs. Pipeline Development & Workflow Management Develop, optimise, and monitor ETL/ELT pipelines across various data sources. Implement data validation, governance, quality … engineering within the organisation. Ensure data models and pipelines effectively support reporting and decision-making. Performance & Continuous Improvement Monitor system and pipeline performance, ensuring SLAs are met. Evaluate and implement new technologies and tools to improve data capabilities. Automate and streamline processes to enhance ...

experienced Data Engineer to design, build, and optimise data solutions within the Microsoft Azure ecosystem. The role focuses on pipeline development, data modelling, governance, and supporting analytics teams with high-quality, reliable data. Key Responsibilities: Develop and maintain scalable data pipelines … data security and governance, including Row-Level Security (RLS) and compliance standards. Maintain documentation and metadata for data assets and pipeline architecture. Collaborate with analysts, data scientists, and stakeholders to deliver trusted data products. Provide technical support and troubleshoot production issues. ...

Data Engineer (Snowflake) We are seeking an experienced Data Engineer (Snowflake) to join our clients team on a permanent basis. This role will focus on administering and developing our Snowflake data platform, building robust data pipelines, and transforming data to support analytics and marketing activation use cases. The successful candidate will initially work on projects involving the ingestion of multiple data sources - including Google Analytics 4 (GA4)- and transforming data to surface insights within Google Ads. Key Responsibilities Administer, maintain, and optimise ...

Senior Data Engineer Salary: Up to £70,000 I am working with a forward-thinking organisation that is modernising its data platform to support scalable analytics and business intelligence across the Group. With a strong focus on Microsoft technologies and cloud-first architecture, they are looking … bring on a Data Engineer to help design and deliver impactful data solutions using Azure. This is a hands-on role where you will work across the full data stack, collaborating with architects, analysts, and stakeholders to build a future-ready platform that drives ...

Senior Data Engineer Salary: Up to £70,000 I am working with a forward-thinking organisation that is modernising its data platform to support scalable analytics and business intelligence across the Group. With a strong focus on Microsoft technologies and cloud-first architecture, they are looking … bring on a Data Engineer to help design and deliver impactful data solutions using Azure. This is a hands-on role where you will work across the full data stack, collaborating with architects, analysts, and stakeholders to build a future-ready platform that drives ...

Senior Data Engineer (Databricks) Full Time/Permanent £70,000 - £80,000 + car allowance, generous bonus, private medical and other extensive benefits Hybrid - 1 day a week in the Oxfordshire head office required The Company: My client is an industry leading and award-winning financial services organisation … closely with the Head of Data, Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from a wide-ranging dataset to support executive and operational decision making that will underpin sustained ...

Data & Integration's Lead (Azure) I am recruiting for a rapidly growing, multi-site healthcare organisation in the middle of a major digital transformation. As their Data & Business Intelligence function continues to expand, they require a hands-on Data & Integration's Lead/Manager … Essential: Strong hands-on experience with the Azure data platform Proven experience delivering API and enterprise system integrations ETL/ELT pipelines, data modelling, and data warehousing Understanding of how data feeds into Power BI and analytics platforms Desirable: GCP exposure ...

Data Engineer (Databricks & Azure) - 3-Month Rolling Contract Rate: £400-£450 per day Location: Remote IR35 Status: Outside IR35 Duration: Initial 3 months (rolling) About the Company Join a leading Databricks Partner delivering innovative data solutions for enterprise clients. You'll work on cutting-edge projects … leveraging Databricks and Azure to transform data into actionable insights. About the Role We are seeking an experienced Data Engineer with strong expertise in Databricks and Azure to join our team on a 3-month rolling contract. This is a fully remote position, offering flexibility ...

Senior Data Engineer The Senior Data Engineer will play a crucial role in designing, implementing, and maintaining scalable data pipelines and infrastructure. This position is ideal for those with strong technical expertise and a passion for working in the Insurance/Financial services industry. … Client Details Senior Data Engineer The employer is a medium-sized organisation operating in the F sector. They focus on delivering innovative solutions and maintaining a strong reputation for excellence in analytics and data-driven decision-making. Description Senior Data Engineer Develop and maintain ...

largest transformation programmes in government, modernising the digital systems that support over £7 trillion of property ownership. As a Senior Data Engineer, you will help establish a new data engineering capability, contributing to the development of reliable data pipelines and products that improve data access, integrity and value across the organisation. Your work will support programmes that shape how HMLR manages and uses its data for years to come. Salary up to £60,800, 29% employer pension contribution plus full Civil Service benefits. Flexible, hybrid working from Plymouth, Croydon ...

supporting a large-scale data programme that requires an experienced Data Engineer to help transform complex, unstructured information into clean, reliable datasets suitable for analysis and reporting. The project involves working with sizeable JSON files and other mixed-format sources, standardising them, and preparing them … Have Background in sensitive or regulated data environments. Understanding of data protection considerations. Exposure to ETL or data pipeline development. ...

supporting a large-scale data programme that requires an experienced Data Engineer to help transform complex, unstructured information into clean, reliable datasets suitable for analysis and reporting. The project involves working with sizeable JSON files and other mixed-format sources, standardising them, and preparing them … Have Background in sensitive or regulated data environments. Understanding of data protection considerations. Exposure to ETL or data pipeline development. ...

Role title: Snr Data Engineer Security Clearance required: SC Clearance IR35 Status: Inside IR35 Pay Rates: £500/d - £525/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/… large data sets, and complex data pipelines. Understanding of Data Architecture and Design, and Data pipeline optimisation. Proven expertise with Data bricks, including hands on implementation experience and certifications. Reasonable Adjustments: Respect and equality are core values to us. ...