1 to 25 of 123 Data Pipeline Jobs in the South East

Delivering our strategy sustainably is fundamental to achieving our ambition to be a net zero company by 2050 or sooner! About the Role : We are seeking a Senior Data Engineer to join our growing data engineering team. In this role, you will be responsible for designing, building, and optimizing scalable data pipelines and infrastructure … that power analytics, machine learning, and business intelligence across the organization. You will work closely with data scientists, analysts, and platform engineers to ensure robust and efficient data delivery. This is an excellent opportunity for someone who is passionate about data, cloud technologies, and scalable systems, and wants to have a direct impact on … data-driven decision-making. Key Responsibilities: Design, build, and maintain scalable and reliable ETL/ELT data pipelines using Python, PySpark, and SQL. Develop and manage data workflows and orchestration using tools such as Airflow or similar. Optimize data processes for performance, scalability, and cost-efficiency, particularly in cloud environments. Collaborate with More ❯

Delivering our strategy sustainably is fundamental to achieving our ambition to be a net zero company by 2050 or sooner! About the Role : We are seeking a Senior Data Engineer to join our growing data engineering team. In this role, you will be responsible for designing, building, and optimizing scalable data pipelines and infrastructure … that power analytics, machine learning, and business intelligence across the organization. You will work closely with data scientists, analysts, and platform engineers to ensure robust and efficient data delivery. This is an excellent opportunity for someone who is passionate about data, cloud technologies, and scalable systems, and wants to have a direct impact on … data-driven decision-making. Key Responsibilities: Design, build, and maintain scalable and reliable ETL/ELT data pipelines using Python, PySpark, and SQL. Develop and manage data workflows and orchestration using tools such as Airflow or similar. Optimize data processes for performance, scalability, and cost-efficiency, particularly in cloud environments. Collaborate with More ❯

Delivering our strategy sustainably is fundamental to achieving our ambition to be a net zero company by 2050 or sooner! About the Role : We are seeking a Senior Data Engineer to join our growing data engineering team. In this role, you will be responsible for designing, building, and optimizing scalable data pipelines and infrastructure … that power analytics, machine learning, and business intelligence across the organization. You will work closely with data scientists, analysts, and platform engineers to ensure robust and efficient data delivery. This is an excellent opportunity for someone who is passionate about data, cloud technologies, and scalable systems, and wants to have a direct impact on … data-driven decision-making. Key Responsibilities: Design, build, and maintain scalable and reliable ETL/ELT data pipelines using Python, PySpark, and SQL. Develop and manage data workflows and orchestration using tools such as Airflow or similar. Optimize data processes for performance, scalability, and cost-efficiency, particularly in cloud environments. Collaborate with More ❯

DATA ARCHITECT - DV CLEARED NEW PERMANENT JOB OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A DATA ARCHITECT WITH ENHANCED DV CLEARANCE Permanent job opportunity for a Data Architect Leading National Security SME Salary up to £100,000 + Bonus London based organisation in an easily accessible location To apply please call or … email WHO WE ARE? We are recruiting multiple a Data Architect to support urgent National Security & Defence projects in London. Due to the nature of these projects you must hold enhanced DV Security Clearance. WHAT WILL THE DATA ARCHITECT BE DOING? You will be joining a leading SME who is working hard to support National Security … projects within UK Govt. Departments in London. As part of a team, you will be responsible for designing and implementing Data Solutions in Mission-Critical areas. WE NEED THE DATA ARCHITECT TO HAVE.... Current DV clearance - Enhanced Good at understanding complexity and abstracting that into a form that is consumable for a non-technical audience. Experience More ❯

Engineer, you will operate within an Agile delivery environment, working closely with the Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from our wide-ranging dataset to support executive and operational decision making that will underpin sustained growth of BFS business units … solutions. You will also have management responsibilities for a small team of Data Engineers, who you will coach, support and organise to ensure we sustain a predictable pipeline of delivery, whilst ensuring all appropriate governance and best practice is adhered to. Your responsibilities as our Senior Databricks Engineer will include: Understand the business/product strategy and … Azure DevOps, collaborating on Sprint cycles, code deployment, version control, and development practices Develop and maintain data models, schemas, and documentation Monitor and troubleshoot data pipeline issues, ensuring timely resolution Stay up-to-date with the latest industry trends and technologies in data engineering and recommend improvements to existing systems or processes as More ❯

Engineer, you will operate within an Agile delivery environment, working closely with the Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from our wide-ranging dataset to support executive and operational decision making that will underpin sustained growth of BFS business units … solutions. You will also have management responsibilities for a small team of Data Engineers, who you will coach, support and organise to ensure we sustain a predictable pipeline of delivery, whilst ensuring all appropriate governance and best practice is adhered to. Your responsibilities as our Senior Databricks Engineer will include: Understand the business/product strategy and … Azure DevOps, collaborating on Sprint cycles, code deployment, version control, and development practices Develop and maintain data models, schemas, and documentation Monitor and troubleshoot data pipeline issues, ensuring timely resolution Stay up-to-date with the latest industry trends and technologies in data engineering and recommend improvements to existing systems or processes as More ❯

Data Architect Salary : Up to £85,000 (Based on candidate experience) Employee Package: Outstanding employee benefits Hybrid Model : Once a week (London) Job Type: Permanent Job Summary Our client, a leading IT services and consulting firm, is looking for a skilled Data Architect to support enterprise-wide data architecture initiatives. This role focuses on … transforming core data assets into trusted, reusable resources that drive service delivery improvements, business growth, and AI innovation across multiple departments. Understanding of public sector compliance, regulations, and cross-departmental data sharing challenges Key Responsibilities: Build scalable data architecture for high-value organizational data assets Work with product teams and stakeholders to …/foundational data experience, cloud-native platforms knowledge, AI/ML data requirements understanding, data federation/cached services familiarity, data pipeline development, and Dell Data Lakehouse/DDAE administration experience. To be considered, please ensure you complete your application on the Computappoint website. Services offered by Computappoint Limited More ❯

hunt for builders . No, we've not ventured into construction in our quest to conquer the world, rather a designer and builder of systems for all things data related where we are conquering the Data World. We can offer an interesting insight into projects spanning a variety of sectors, which may include industries such as … to explore our dynamic team where you can truly act as an expert in your field in support of our clients and their challenges in the world of data and technology, read on! The Team We're Intuita - a fast growing consultancy that's making waves in both the consultancy and technology space. With our ambitious goals for … in the data pipelines. • Stay updated with the latest Azure technologies and industry best practices to continuously enhance data engineering capabilities. Looking at our current pipeline of work, we can also consider those with an Analytics Engineering lean, with experience within BigQuery (GCP) with data modelling in DBT and mobile/telecoms industry More ❯

AWS Data Engineer - VP Level Location: London (Canary Wharf)Salary: £115000 - 120000 A leading financial institution is seeking a Senior AWS Data Engineer to join a greenfield initiative focused on transforming market data access and utilization. This VP-level role offers strategic influence, technical leadership, and cross-functional collaboration across business units. Key Responsibilities … Design and maintain scalable data pipelines, warehouses, and lakes using AWS technologies.* Implement robust data governance, quality, and security practices.* Collaborate with data scientists to deploy machine learning models.* Lead or contribute to strategic planning, risk management, and stakeholder engagement.* Drive innovation through advanced analytics and research-based problem solving. To be successful you … should have: 10 years hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Previous experience in implementing best practices for data engineering, including data governance, data quality, and data security. Proficiency in data More ❯

Data Engineer Location: Hammersmith - Hybrid after probation Salary: £38,000 -£45,000 Join Our Team as a Data Engineer! About Us : Join a forward-thinking organisation dedicated to revolutionising retail through innovative solutions and operations. We leverage physical spaces to test new concepts and pilot initiatives, helping brands and retailers thrive in a dynamic marketplace. What … You'll Do : As our Data Engineer, you will play a pivotal role in the day-to-day management and maintenance of our data infrastructure, driving our large-scale programmes in the USA. Your primary responsibilities will include : Design & Development: Create, develop, and maintain scalable data pipelines and ETL processes across various systems. Data … star schema, data vault, dimensional modelling). Proficiency in Excel-based data workflows for various Agile Retail projects. Hands-on experience with data pipeline orchestration tools (Airflow, dbt, Prefect, or similar). Benefits: Unlimited holiday Annual Wellbeing Allowance Flexible work culture Monthly socials and events Complimentary snack bar Employer pension contribution If you More ❯

Role: Head of Data Management Location: Milton Keynes (Hybrid 2 Days On-site Weekly) Salary: £55,000 - £70,000 Benefits: Bonus, Company Cars, Flexible Benefit Package Network IT is recruiting for a Head of Data Management to join a leading automotive brand's UK IT function. In this senior role, youll lead a high-performing data management team, drive technical delivery across enterprise data platforms, and ensure compliance, scalability, and performance within a global standards-based ecosystem. Reporting to the Head of IT, youll take ownership of operational excellence, stakeholder engagement, and vendor performance across a broad data estate including Azure Data Factory and Data Lake … Strong experience in IT service management, strategy planning, and business process alignment. Hands-on knowledge of Azure Data Factory, Data Lake , and data pipeline architecture. Proven experience managing data security, access, and regulatory compliance. Familiarity with automation tools (AI, RPA) and Agile project delivery (e.g. Scrum, Kanban). Demonstrated ability to More ❯

the industry operates and putting customers at the heart of everything we do. Having completed our recent multi-year digital transformation, we’re now leveraging advanced technology and data-driven insights alongside exceptional service, to deliver personalised experiences that meet our customers ever-changing needs today and in the future. Job Description We are currently looking for a … Lead Data Engineer to join our forward thinking Data department. You’ll work in a team of data engineers, analytics engineers, data scientists and AI specialists to design and evolve scalable data platforms and modern data products that enable self-service analytics, advanced modelling, and AI-driven decision … making across our insurance business. What you’ll do: Lead on data engineering projects, managing a sub- team within engineering, building out our data warehouse & pipelines Collaborate effectively across data science, analytics, product, engineering and commercial business teams Build and support esure’s data products within our industry leading platform Work with More ❯

Role: Data Team Lead Location: Milton Keynes (Hybrid 3 Days On-site Weekly) Salary: £55,000 - £65,000 Benefits: Bonus, Company Cars, Flexible Benefit Package Network IT is recruiting for a Data Team Lead to join a leading organisations UK IT function. This role will see you managing and developing a high-performing data team, while ensuring the delivery of effective and scalable data solutions. Youll play a key role in setting direction, building capability, and aligning data initiatives with business priorities. The role requires strong leadership, stakeholder management, and the ability to oversee data programmes from strategy through to delivery. Key Responsibilities: Lead, mentor, and … data leadership roles within complex organisations. Proven track record of leading and developing technical teams. Good understanding of data warehousing, ETL, and data pipeline solutions. Experience with data governance, compliance, and security frameworks. Strong stakeholder engagement, vendor management, and planning skills. Experience managing budgets and resources effectively. Excellent communication, leadership, and More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector we are always in search of the best … you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and onpremises … with over 3 years of relevant experience in data engineering, platform engineering, or a related field, with demonstrated hands-on expertise in NiFi and data pipeline design in regulated environments. Responsibilities: Design, develop, and maintain robust and secure data pipelines using NiFi and related big data technologies. • Troubleshoot and optimize NiFi More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best … you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and onpremises … with over 3 years of relevant experience in data engineering, platform engineering, or a related field, with demonstrated hands-on expertise in NiFi and data pipeline design in regulated environments. Responsibilities: Design, develop, and maintain robust and secure data pipelines using NiFi and related big data technologies. • Troubleshoot and optimize NiFi More ❯

Senior Data Engineer Hybrid working - 1 day a week onsite in either London or Portsmouth Permanent - Up to £75k Overview: We are seeking a talented Senior Data Engineer specialising in Starburst (Trino) and Dell Data Lakehouse to join our AI & Data team. You will be responsible for deploying, maintaining and optimising Starburst … installations & Dell Data Lakehouse, enabling our clients to seamlessly access their data across multiple platforms. The ideal candidate will have excellent communication skills, an advanced understanding of Starburst & Dell Data Lakehouse, and proficiency with troubleshooting and root cause analysis. Responsibilities: Deploy and manage Starburst Enterprise/Galaxy and Dell Data Lakehouse installations … such as Ansible. Knowledge of data requirements for AI and machine learning workloads. Familiarity with Data Federation and Cached Services Familiarity with Data pipeline (Series of steps that move and transform data from one source to another for analyses and storage) Experience with Dell Data Lakehouse administration. Experience in More ❯

to-end testing of data ingestion, parsing, aggregation, and schema validation processes in AWS. Conduct stress testing to evaluate the performance of the entire data pipeline built on AWS. Develop and execute test plans, test cases, and test scripts to ensure data quality and integrity. Collaborate with data engineers and developers … to identify and resolve data quality issues. Monitor data quality metrics and implement improvements to enhance data reliability. Validate data transformations and ensure consistency across various data sources. Document test results and provide detailed reports on data quality findings. Required Skills and Experience: Proven experience with AWS services … such as EMR, Lambda, Redshift, Firehose, S3, Iceberg, Athena, and DynamoDB. Strong understanding of data ingestion, parsing, aggregation, and schema validation processes. Proficiency in SQL for data querying and validation. Experience with Python for scripting and automation. Familiarity with data quality tools and frameworks. Excellent analytical and problem-solving skills. Strong attention to detail More ❯

Senior Data Engineer Opportunity Salary: Up to £75k (Based on candidate experience) + Excellent benefits Work commute Model: Mostly remote with just 1 day per week on-site Location: London/Portsmouth Job Type: Permanent Ready to Shape the Future of Data? My client is a leading UK-based IT services and consulting company seeking an … experienced Senior Data Engineer who thrives on building robust, scalable systems that unlock the true potential of enterprise data. This is your opportunity to join their cutting-edge AI & Data team as a Senior Data Engineer . What You'll Be Doing Build & Optimise : Take ownership of Starburst (Trino) and Dell Data … and/or Java Automation Expertise with tools like Ansible AI/ML Data Requirements understanding Data Federation and caching technologies experience Data Pipeline design and optimisation Dell Data Lakehouse administration background To be considered, please ensure you complete your application on the Computappoint website. Services offered by Computappoint Limited are More ❯

Role Title: Data Solutions Architect (GCP) Business Unit: Data Management Role Overview This strategic, client-facing role sits within a growing Data Management practice and focuses on leading cloud-based data architecture initiatives, particularly on Google Cloud Platform (GCP). The Data Solutions Architect will collaborate with senior stakeholders to … modernise data platforms, accelerate cloud transformation, and unlock business value through data and AI. This is a technical leadership position-not hands-on coding-but one that requires deep architectural expertise, strong consulting skills, and the ability to influence C-suite decision-making. The role also involves significant pre-sales engagement, translating client needs into solution … and proposal narratives. Collaborate with sales, pricing, and delivery teams to shape commercially viable and technically sound solutions. Enterprise Solution Design Architect and oversee the delivery of scalable data platforms (data lakes, lakehouses, warehouses) using GCP technologies such as BigQuery, Cloud Storage, Databricks, and Snowflake. Cloud Data Strategy Lead cloud migration and modernisation strategies More ❯

Data Engineering Lead (Azure) London (3 days in office) Up to £105,000 + 5% Bonus + Benefits Are you ready to take the lead on building a cutting-edge data platform? We're seeking a Data Engineering Lead to shape and grow data engineering capabilities at a global, forward-thinking organisation. … a modern Azure-based platform, lead a talented team, and make a direct impact on business-wide decision making. Why this role? Spearhead the creation of a new data platform from the ground up. Balance hands-on engineering with team leadership and mentoring. Collaborate with analysts, data scientists, and business leaders to deliver impactful solutions. Hybrid … remote flexibility the rest. Join a supportive and inclusive culture, with a global footprint and a focus on innovation. What you'll be doing: Designing and building scalable data pipelines to support analytics, reporting, and machine learning. Leading the implementation of Azure data solutions and ensuring robust ETL/ELT processes. Defining architecture strategies that allow More ❯

Role: Data Engineer Location: City of London Salary: £87k Flexible working: 2 days a week in the office I am working with a leading insurance company who are looking for an experience Data Engineer to join their … team. This will be a great opportunity to work on exciting transformational projects and take your career to the next level. Skills and experience Experience using Data pipeline tools Strong experience with SQL and Python Azure Data stack & Azure SQL Hands on experience with Databricks Experienced with data lakes, data integration … and data warehousing Insurance experience ideally If this role sounds of interest and you would like to know more do not hesitate to contact me on Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse and inclusive community we have built, and we welcome applications from people of all backgrounds and More ❯

Data Engineer (Microsoft Azure) Location: London/Hybrid 2-3 days in office Type: 6 months contract Rate: £250 to £275/a day PAYE About the Role We are seeking a Data Engineer to support the design, build, and maintenance of modern data solutions on the Microsoft Azure platform. You will play a … key role in ensuring data is accessible, secure, and efficient, while collaborating with stakeholders to deliver solutions that meet strategic business needs. This is an exciting opportunity for someone motivated to learn, develop, and contribute to large-scale transformation initiatives in a dynamic environment. Key Responsibilities Support the development of data pipelines and storage solutions Assist … for front-office operations. Participate in Proof of Concept (PoC) development for data storage and transformation solutions. Support the Business Entity Mastering (BEM) to Fabric transformation through pipeline development and data modelling. Ensure all work aligns with governance, controls, and technical standards. Proven experience in data engineering. Strong understanding of data More ❯

Data Engineer Location: London - Hybrid Working Employment Type: Permanent Salary: £40k-£50k Data Engineer About the Role We're seeking a Data Engineer to collaborate with business stakeholders in designing, building, and maintaining innovative data solutions. You'll play a key role in developing their cloud-based analytics platform on Azure whilst … facilitating seamless integration with various systems, including AWS environments. Your expertise will be crucial in creating robust data pipelines that transform raw information into actionable insights for business intelligence, advanced analytics, and strategic modelling. Data Engineer - The Team & Environment Join during an exciting period of digital transformation as we harness data and analytics to … drive informed decision-making across the organisation. The Data, Analytics and Insight division is pioneering a customer-centric approach to data utilisation, ensuring we continuously evolve and deliver exceptional value. Their diverse team encompasses: Business Intelligence - crafting intuitive data visualisations and dashboards that make complex information accessible Customer Research - leveraging empathy-driven research, surveys More ❯

Are you passionate about shaping and delivering data strategies that drive real change and innovation? Do you excel at both setting the vision and rolling up your sleeves to design and implement cutting-edge data solutions? Could you be our next Data Technology Manager, leading by example to modernise our data landscape … while staying hands-on in the work that makes it happen? Data Technology Manager Up to £66,000 Permanent/Full Time Stevenage with Hybrid Working At the IET, making our world a better place starts by creating a better place for our people. At the IET youll have more than just a job. Our work is exciting … quality, scalable solutions, and the leadership skills to inspire and develop a high-performing team. Youll be just as comfortable in a design meeting as you are debugging a pipeline or building a proof-of-concept. Your ability to balance big-picture thinking with practical delivery will be key to your success. Wed love to get to know you More ❯

Robert Half Technology are assisting a market leading financial services organisation to recruit a Data Engineer on a contract basis - Hybrid working - London based Looking for a highly skilled Data Engineer to join our growing data platform team. You will be responsible for designing, building, and optimising large-scale data pipelines and … services that power analytics, reporting, and data-driven decision-making across the business. This role is heavily cloud-focused (AWS) and requires strong expertise in Python, Spark, and relational databases. Role Design, build, and maintain scalable, reliable, and high-performance data pipelines and workflows. Develop clean, maintainable, and testable code in Python for data ingestion, transformation, and processing. Optimise and fine-tune query performance on Aurora Postgres and other relational databases. Architect and manage data solutions on AWS using serverless technologies such as Lambda, Glue, Glue Data Catalog, EMR serverless, and API Gateway. Implement and manage large-scale data processing with Spark (Iceberg tables in S3 More ❯