1 to 25 of 46 Data Pipeline Jobs in the South West

Create and maintain optimal data pipeline architecture. Assemble large, complex data sets that meet functional/non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Build the infrastructure … variety of data sources using SQL and AWS 'big data' technologies. Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics. Work with stakeholders including the Executive, Product, Data … data tools: Hadoop, Spark, Kafka, etc. Experience with relational SQL and NoSQL databases, including Postgres and Cassandra. Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc. Experience with AWS cloud services: EC2, EMR, RDS, Redshift. Experience with stream-processing systems: Storm, Spark More ❯

Data Engineer required by our market leading, award winning, professional services organisation based in Yeovil. The successful Data Engineer, you'll play a vital role in designing, building, and maintaining sophisticated data pipelines and ensuring the integrity of our clients extensive customer data. Your … to develop your expertise in data engineering. Key Responsibilities Design & Build Data Pipelines: Create and maintain scalable data pipeline architecture that supports business needs. Data Management: Assemble large, complex data sets to meet business and technical requirements. Process Improvement … years of hands-on experience with big data tools and frameworks. Technical Skills: Proficiency in SQL, Python, and data pipeline tools such as Apache Kafka, Apache Spark, or AWS Glue. Problem-Solving: Strong analytical skills with the ability to troubleshoot and resolve data More ❯

being the dedicated point of contact . You will: Lead and mentor a team of 6+ Tech Leads overseeing product engineering squads delivering data-centric products Provide strategic guidance , coaching, and professional development opportunities for engineering managers to empower their teams and deliver results Establish and maintain a … industry trends and emerging technologies, continually improving data engineering capabilities Metrics-Driven Impact: Develop and track success metrics, including data pipeline reliability, availability, and time-to-insight, to evaluate and continuously improve team performance Communicate the impact of data engineering initiatives through clear … excellence through the implementation of efficient data engineering workflows, processes, and tools Drive the adoption of best practices for data pipeline development, continuous integration/continuous deployment (CI/CD), and data monitoring Identify and implement opportunities for automation and optimization, improving operational More ❯

Data Engineer (Security Clearance) Position Description The Space, Defence and Intelligence business unit in CGI is a true IT Systems Integrator. We work, build, and operate bespoke, technically complex, mission-critical systems which help our clients keep us all safe and secure. We bring innovation to our clients … collaboratively with global technology companies, cutting edge SMEs and academia to deliver the optimal solution for each client. We're seeking a Senior Data Engineer to design and lead the implementation of data flows between operational systems and BI/analytics platforms. Our Data Services team manages the ingestion, storage, and availability of diverse datasets, enabling analysts and data scientists to generate insights that support operational and tactical decisions. We're expanding our team of Data Engineers to ingest new datasets and maintain existing ones ensuring data More ❯

to your opportunity As a practitioner, you will bridge the gap between client needs and technical solutions. Your main responsibility is to make data valuable by developing pipelines that ingest, transform, and enrich high-volume, diverse data into accessible, trusted information assets for actionable insights. You … will manage deliverables and client relationships, delivering solutions using agile methodologies across multidisciplinary teams and industries. Your responsibilities may include: Applying data engineering tools and frameworks to create standardized data for AI and reporting. Designing and implementing high-quality data pipelines and stores. Introducing … candidates with both technical and business skills who can articulate the value of their work, with experience in: End-to-end data pipeline development. Working with diverse data types and sources. Architectural design for scalable data solutions. Data modeling, warehouse design More ❯

you ready to make a significant impact in the UK's leading investment services company? We are on a mission to revolutionize our data strategy by building core data hubs that serve as a single source of truth for our business. Our evolving data estate on AWS and Snowflake is at the heart of this transformation, consolidating key data areas such as client, account, and market reference data hubs. As a Senior Data Engineer, you will lead the engineering efforts for these data hubs … are a highly skilled data engineer with experience in working with high volume data/low latency data pipeline with the following skills. Data Engineering Skills Modelling Orchestration using Apache Airflow Cloud native streaming pipelines using Flink, Beam etc. DBT Snowflake More ❯

Data Engineer/Data Scientist (Imaging & Data Labelling) Location: Southwest UK (On-site/Hybrid) Clearance: Sole British National with active SC Clearance Engagement: Contract – Outside IR35 Overview: We are seeking a skilled Data Engineer/Data Scientist with strong … experience in data labelling and imaging datasets to support a critical defence-related project in the Southwest UK. The ideal candidate will have a solid background in building and managing data pipelines, annotating and labelling image-based data for machine learning workflows, and working … contributor in preparing high-quality datasets to support AI and ML initiatives in a mission-critical context. Key Responsibilities: Develop and maintain robust data pipelines to ingest, clean, transform, and store large volumes of imaging data. Lead and support the data labelling process—developing tools and More ❯

Data Engineer - Salisbury - £55,000-£65,000 - Hybrid My client is looking for a highly skilled Data Engineer to join our growing data team. This role is ideal for someone with strong experience in Azure data services and a passion for building … scalable, secure, and cost-effective data pipelines that support analytics and BI initiatives. You’ll work closely with data scientists, analysts, and business stakeholders to ensure our data infrastructure is reliable, performant, and compliant. Key Responsibilities Design, build, and maintain robust ETL/ELT … architecture and processes meet security, privacy, and compliance standards (e.g., GDPR, encryption, access controls). Support CI/CD and data pipeline automation best practices. Must-Have Skills Strong hands-on experience with Azure Synapse Analytics, Data Factory, and Data Lake Storage. More ❯

Data Engineer/Data Scientist (Imaging & Data Labelling) Location: Southwest UK (On-site/Hybrid) Clearance: Sole British National with active SC Clearance Engagement: Contract – Outside IR35 Overview: We are seeking a skilled Data Engineer/Data Scientist with strong … experience in data labelling and imaging datasets to support a critical defence-related project in the Southwest UK. The ideal candidate will have a solid background in building and managing data pipelines, annotating and labelling image-based data for machine learning workflows, and working … contributor in preparing high-quality datasets to support AI and ML initiatives in a mission-critical context. Key Responsibilities: Develop and maintain robust data pipelines to ingest, clean, transform, and store large volumes of imaging data. Lead and support the data labelling process—developing tools and More ❯

Next-gen diagnostics start-up Take ownership of data strategy and architecture from the ground up. Flexible, remote-first working with visits to a high-impact biotech lab. Cutting-edge work combining AI, genomics, and diagnostics to save lives. Were working with a biotech start-up thats tackling … looking for their very first Data Engineer. Youll take the reins on all things data - shaping the architecture, strategy, and pipeline design that supports a life-saving platform. Youll be leading on the design and implementation of scalable, secure data pipelines - getting experimental … and genomic data into formats that data scientists and machine learning engineers can easily work with. This role combines classic data engineering work with data management strategy, data lineage tracking, and storage infrastructure design; from cold archiving through to high More ❯

Senior Data Engineer Bristol 6-month contract Paying up to 80p/h (Outside IR35) Please note that due to the nature of the work you must be hold or be eligible to obtain DV level security clearance Responsibilities : Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. Implement data ingestion, transformation, and integration processes, ensuring data quality and security. Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards. … Manage and monitor large-scale data flows in real-time, ensuring system performance, reliability, and data integrity. Develop robust data models to support analytics and reporting within secure environments. Perform troubleshooting, debugging, and performance tuning of data pipelines and the Elastic Stack. More ❯

Group Enterprise Data Architect Location: Cheltenham, UK (Hybrid working) Benefits: 27 days holiday plus Wellbeing day, Private Medical Insurance, Bonus scheme, Sharescheme, Enhanced pension plan, Life assurance, Discount scheme. Role Overview: Join Spirax Group as a Group Enterprise Data Architect , reporting directly to the Group Head … of Data and Analytics. This hybrid role requires a minimum of four days per month in Cheltenham and occasional travel. You will shape our global data strategy, influence business unit architects, and promote consistency in enterprise data architecture. Your objectives and responsibilities will focus … on Leading the strategic direction for data solutions across the Spirax Group. Collaborating with stakeholders and IS/IT teams to understand needs and deliver tailored data solutions. Mentoring Data/IT colleagues via a Community of Excellence. Building relationships with technology partners, evaluating More ❯

Experience function hiring in: United Kingdom Work Preference Option(s): Hybrid model Work from Office Join Ascent and help shape the future of data solutions by becoming a valued team member where your skills matter, your impact is significant, and your career can reach new heights. About Us … and we help our customers solve problems, elevate, and do existing things better. We are on a mission to help our customers connect data, software, and purpose to create extraordinary outcomes. You could say we are a digital transformation business. We specialize in software product development, analytics, data science, IoT solutions, machine learning, DevOps optimization, and modernization of applications, data, and platforms. We work with incredible clients in all types of industries such as smart home devices, space exploration, beer manufacturing, finance, ecology, and logistics. We work with some of the sharpest minds in More ❯

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the More ❯

J Job Advertisement: Mid-Level Data Scientist/Data Analyst Location: Hybrid in Bristol (2-3 days a week) Contract Type: Permanent Salary: £50,000 - £60,000 depending on experience Are you passionate about data and excited to make a real impact in the … the next level, we want to hear from you! Position Overview: We are on the lookout for a talented and experienced Mid-Level Data Scientist/Data Analyst to spearhead the development and operation of our cutting-edge automated ingestion system. If you thrive in a … dynamic, agile environment and have a knack for designing sophisticated data ingestion pipelines with advanced ML components, this role is made for you! Key Responsibilities: Automated Ingestion System Development: Lead the design, development, and maintenance of advanced data ingestion pipelines, ensuring they are efficient, scalable, and More ❯

/non-tariff Department Insights It's an exciting time to join us! RWE Supply & Trading has recently introduced a new cloud-based data analytics platform. We are expanding our team and looking for an experienced Software Engineer to play a key role in enhancing our platform capabilities. … The Lead in Data (LiD) platform has revolutionized data searching, sharing, transforming, and onboarding within our organization. Data is central to our trading and analysis activities and will continue to be a vital asset across the group. You will apply your expertise in C# … and .NET to develop robust data services within the Mix system and help future-proof our architecture from security and technological perspectives. Experience with AWS environments is highly desirable. If this sounds appealing, then apply now! Your future responsibilities Design and develop scalable back-end services and APIs More ❯

GIS Data Engineer 35k to 50k Bath Hybrid I am working with a tech for good client who is looking for a data engineer, with skills in geospatial engineering for some exciting and large scale projects that they have underway. They are looking for skills in … that really enjoys problem solving and technical challenges. Benefits: 25 days holiday Company phone Health insurance Life insurance And more Role Responsibilities: Developing data pipelines Managing databases Analysing and visualizing GIS data Developing python scripting to automate data ingestion Testing and validating data Data manipulation and cleansing Creating tiles from datasets You will be working with a cross functional team including data analysts, developers, and other key stakeholders. Experience: Degree in Computer Science, GIS, Geography or related field Experience in data engineering Geospatial related software More ❯

GIS Data Engineer 35k to 50k Bath Hybrid I am working with a tech for good client who is looking for a data engineer, with skills in geospatial engineering for some exciting and large scale projects that they have underway. They are looking for skills in … that really enjoys problem solving and technical challenges. Benefits: 25 days holiday Company phone Health insurance Life insurance And more Role Responsibilities: Developing data pipelines Managing databases Analysing and visualizing GIS data Developing python scripting to automate data ingestion Testing and validating data Data manipulation and cleansing Creating tiles from datasets You will be working with a cross functional team including data analysts, developers, and other key stakeholders. Experience: Degree in Computer Science, GIS, Geography or related field Experience in data engineering Geospatial related software More ❯

GIS Data Engineer 35k to 50k Bath Hybrid I am working with a tech for good client who is looking for a data engineer, with skills in geospatial engineering for some exciting and large scale projects that they have underway. They are looking for skills in … that really enjoys problem solving and technical challenges. Benefits: 25 days holiday Company phone Health insurance Life insurance And more Role Responsibilities: Developing data pipelines Managing databases Analysing and visualizing GIS data Developing python scripting to automate data ingestion Testing and validating data Data manipulation and cleansing Creating tiles from datasets You will be working with a cross functional team including data analysts, developers, and other key stakeholders. Experience: Degree in Computer Science, GIS, Geography or related field Experience in data engineering Geospatial related software More ❯

Are you ready to join us and shape the future of data? As a Senior Data Engineer, you will play a critical role in designing, developing, and maintaining scalable data solutions using Microsoft Azure’s ecosystem—including Azure Synapse Analytics, Databricks, and Microsoft Fabric … to deliver innovative solutions across various projects. You will empower our clients to harness their data, drive decision-making, and achieve their business goals. What You’ll Do: Solution Design & Development : Architect and implement scalable, high-performance data solutions using Microsoft Azure technologies, adhering to best … practices for robust delivery. Design and maintain ETL/ELT pipelines, optimize storage solutions, and document technical workflows Data Management & Integration: Integrate structured and unstructured data, develop efficient data models for analytics and reporting, and ensure security, compliance, and data quality. Collaboration More ❯

If you're a Senior Data Scientist with expertise in predictive modelling, segmentation, and data automation, and you're excited about shaping the future of data in a cloud-first environment, we want to hear from you! This is a fantastic opportunity to join … our client at the start of its major data transformation journey. Their move to the cloud with Microsoft Azure, Fabric, and Synapse technologies will see you work with cutting-edge Cloud and AI-driven analytics, making this an exciting time to be part of its data science team. The role. We are looking for a Senior Data Scientist with expertise in predictive modelling, segmentation, and data automation. You will Lead and optimize automated reporting pipelines, ensuring high performance and quality assurance. Build and deploy predictive models, enhancing customer behaviour forecasting More ❯

other locations will still be considered. Overview of the role Our ideal candidate is one that has a solid technical background and demonstrable data engineering skills gained in a practical environment; as such, those with only theoretical viewpoints will not be considered. The ideal candidate will be able … maintain an end-to-end advanced analytics solution on cloud. You will have exposure to cloud, preferably in Microsoft Azure, build and maintain data pipelines, understand CI/CD for end-to-end cloud solutions, be able to manage and maintain pre-existing ecosystems, and be competent in … Build and maintain data pipelines (including those from 3rd party sources and APIs) Identifying and patching issues and bugs identified in the pipeline/architecture Working as part of a data specialist team to deliver quality service to engagement clients Providing access and identity management More ❯

What you'll be doing Are you ready to contribute to the evolution of our data pipelines for our B2C division? At Future, we are transforming our data-driven decision-making processes and we are looking for a passionate and experienced Data Engineer to … join us. This is an exciting opportunity for someone who excels in a creative environment, enjoys solving complex data challenges, and is eager to build impactful business insights, for this role you will directly report into the Head of Data Engineering The Data & Business … Intelligence team guides our organisation to become more data-driven. By ensuring visibility of objective performance data, we empower our teams to make rapid, informed decisions that enhance overall performance. Important Responsibilities include and not limited to; Develop and maintain new/current features of the More ❯

Your new role As Principal Data Engineer, you'll join a collaborative and agile data engineering team where you'll take responsibility for all aspects of data engineering while working with peers to ensure the efficient and sustainable function of the team. You'll … that organises as generalists to encourage sharing of projects, knowledge, and technical skills. The team maintains a large codebase of cloud-deployed, serverless data pipelines written to high software engineering standards. Your responsibilities will include: Working with stakeholders and delivery managers to build and deliver data … practices Experience with tools and techniques such as SQL, various database types, data formats, and data modelling Demonstrable experience of pipeline logging, usage tracking, and cost monitoring Previous experience deploying analytical/machine learning tools for users Experience leading a team or multiple projects and More ❯

Job Title: Snowflake Data Engineer Location: Somerset, 3 days per week onsite Contract- outside IR35 Overview: We are looking for a talented Snowflake Data Engineer to join our growing data team. Reporting to the BI & Data Manager, you will play a key … role in the technical build of our new enterprise data warehouse. While the overall BI and data strategy is being led by the manager, your focus will be on hands-on development: building scalable pipelines, designing efficient data models, and ensuring a high-quality … Snowflake environment to support business intelligence, analytics, and data-driven decision-making. Key Responsibilities: Develop and optimize data pipelines and ETL processes into Snowflake. Build, structure, and maintain the Snowflake data warehouse. Work closely with the BI & Data Manager to align technical More ❯