1 to 25 of 30 Contract Data Pipeline Jobs in the UK

Data Pipeline Development: Design and implement end-to-end data pipelines in Azure Databricks, handling ingestion from various data sources, performing complex transformations, and publishing data to Azure Data Lake or other storage services. Write efficient and standardized Spark … SQL and PySpark code for data transformations, ensuring data integrity and accuracy across the pipeline. Automate pipeline orchestration using Databricks Workflows or integration with external tools (e.g., Apache Airflow, Azure Data Factory). Data Ingestion & Transformation: Build scalable data … for performance by tuning configurations, partitioning data, and caching intermediate results to minimize processing time and resource consumption. Continuously monitor and improve pipeline performance, addressing bottlenecks and optimizing for cost efficiency in Azure. Automation & Monitoring: Automate data pipeline deployment and management using tools like More ❯

Join our client in embarking on an ambitious data transformation journey using Databricks, guided by best practice data governance and architectural principles. To support this, we are recruiting for talented data engineers. As a major UK energy provider, our client is committed to … in their Nottingham office, this is negotiable. It is a full-time role, 37 hours per week. Accountabilities: * Develop and maintain scalable, efficient data pipelines within Databricks, continuously evolving them as requirements and technologies change. * Build and manage an enterprise data model within Databricks. * Integrate new … warehousing. * Able to write repeatable, production-quality code for data pipelines, utilizing templating and parameterization where needed. * Can make data pipeline design recommendations based on business requirements. * Experience with data migration is a plus. * Open to new ways of working and new technologies. More ❯

Role: Data Engineer Location: Belfast Duration: Long Term Contract Opportunity - Rolling Contract Rate: Market Rates - Inside IR35 Job Description : Fujitsu's Decision Intelligence practice in the UK helps organisations bridge the gap between data and insights, empowering businesses to make smarter decisions, optimise operations, and drive … innovation. From AI-powered analytics to data governance and security, we offer a comprehensive suite of services to help our Public and Private Sector customers unlock the full potential of their data. Job Summary : We are seeking a skilled and experienced AWS Data Engineer to join … our team. The successful candidate will be responsible for implementing, and managing data architecture solutions for our customers, with a strong emphasis on Cloud technologies (Preferably AWS) and tooling. This role requires understanding of data modelling, database design, and data integration techniques. Key Responsibilities More ❯

Systems Architecture - AWS data engineer Job Description Location: UK - Will require travel to customer site ](Belfast) Job Summary: We are seeking a skilled and experienced AWS Data Engineer to join our team. The successful candidate will be responsible for implementing, and managing data architecture … solutions for our customers, with a strong emphasis on Cloud technologies (Preferably AWS) and tooling. This role requires understanding of data modelling, database design, and data integration techniques. Key Responsibilities -Data Ingestion and Extraction: Design and implement efficient data ingestion pipelines to … and from databases and file storage services. -Data Transformation and Cleaning: Transform and clean raw data to ensure data quality and consistency. -Data Pipelines: Build, maintain, and optimise data pipelines to automate data flows and enable real-time More ❯

About the role Reporting into the Data Systems Manager, the Data Systems Executive is a key position within the Operations arm of the Moorepay marketing team. The role will work with key stakeholders within the wider Data Operations team to enable and maintain the … Moorepay Data Platform' (MDP). This includes managing data quality as well as administration and maintenance of the ETL Engine and the Data Lakehouse, enabling the business to build value-adding data products at scale to support core business objectives. In particular … literate with strong data language expertise - SQL and Python primarily - for data manipulation and end-to-end data pipeline management Familiar with data-as-code methodology, specifically around version control and documentation/change controls Knowledge of working within GDPR and More ❯

University Healthcare Partners, which has been a driving force for the innovation and life sciences agenda nationally. We recognise that the Trust's data represents one of our most potent assets to support innovation, as well as research and quality improvement. But we have yet to fully exploit … its potential. Main duties of the job Our new Clinical Data Science and Engineering Unit will be the vital engine room for curating, sharing and utilising our data for novel purposes outside of the routine operations of hospital. It will work collaboratively with the Trust's … that extract, process, and analyse sensitive data for decision-making. Experience with test-driven development and continuous integration in data pipeline designs. Ability to communicate complex data insights to senior executives, using tact and diplomacy to influence decisions. Exceptional verbal and written communication More ❯

Data Engineer Hybrid working (2-3 days per week in Gloucester) £450-£550 per day (inside IR35) We are seeking a skilled, Security Cleared, Data Engineer to join our team. The ideal candidate will have a strong background in data architecture and engineering, specifically … within the AWS ecosystem. You will be responsible for designing and implementing data pipelines that support large-scale data processing and analytics. Key Responsibilities: - Develop, construct, test, and maintain data architectures and data processing systems. - Collaborate with data scientists and … analysts to understand data requirements and ensure data availability. - Design and optimise data models to support business intelligence and reporting needs. - Monitor and troubleshoot data systems to ensure high availability and performance. - Implement best practises for data management, security, and More ❯

Role: Snowflake Architect Location: Basildon, UK/Dublin, Ireland Work from Client office 5 days weekly Responsibilities: -Design, develop, and maintain robust data pipelines and ETL processes using Snowflake on AWS. -Implement data warehousing solutions, ensuring efficient storage, retrieval, and transformation of large datasets. -Collaborate with … data analysts, scientists, and other stakeholders to define and fulfil data requirements. -Optimize performance and scalability of Snowflake data warehouse, ensuring high availability and reliability. -Develop and … maintain data integration solutions, ensuring seamless data flow between various sources and Snowflake. -Monitor, troubleshoot, and resolve data pipeline issues, ensuring data quality and integrity. -Stay up-to-date with the latest trends and best practices in data engineering More ❯

Senior Data Engineering Manager London - Hybrid (2 days/week, usually Tues & Fri) 12-month contract Day rate: £700 (umbrella) About the Role Join a critical data remediation programme within the Surveillance Team/Lab, part of the Markets Platform area of a major financial services … You will lead design and delivery of stable, scalable, and performant data solutions within a complex GCP architecture, driving simplification and improving pipeline automation to enable timely, secure data delivery to surveillance systems. Key Responsibilities Engineer stable, scalable, performant, accessible, testable, and secure data … SQL expertise for querying, transforming, and troubleshooting data Experience building robust ETL pipelines from diverse source systems Familiarity with CI/CD pipeline automation and enhancement Understanding of Agile delivery frameworks, including Jira and sprint planning Knowledge of Terraform for infrastructure as code provisioning Experience with containerization More ❯

proficiency in PySpark, Hadoop, Python, Git - Strong understanding of data engineering concepts and ETL processes, Data warehousing, Data Pipeline, SQL - Additional Good To Have Skills: AWS Glue, Knowledge of implementing ETL and data pipelines in AWS -5+ years of experience More ❯

Job Title: Senior Data Engineer Location: London or Telford Contract: 6 months Clearance Requirement: Must hold or be eligible to obtain SC clearance About the Role We are seeking a Senior Data Engineer to join our team on a permanent basis, working on a high-profile … UK Government programme. You'll apply your expertise in data engineering and software development to solve complex data challenges, supporting critical infrastructure and decision-making through scalable, secure, and innovative data solutions. This is a fantastic opportunity to work with modern technologies in a … collaborative and mission-driven environment, with the freedom to innovate and grow your skills. Key Responsibilities Design and lead implementation of data pipelines from operational systems to BI and analytics platforms Build, maintain, and operate scalable data environments and real-time data streaming systems More ❯

technical solutions in Agile environments with expertise in JIRA and Confluence. Proven Solution Architect Experience: Deep technical expertise in various solution architectures, especially data platforms and complex data pipelines on public cloud platforms, with a focus on GCP. Robust Application Design Skills: Experience in designing scalable … low-latency solutions that meet diverse functional and non-functional requirements. Data Architecture Expertise: Proven experience in designing and delivering complex batch and streaming data pipelines, understanding orchestration, and database capabilities across multiple solutions. Problem-Solving Ability: A strategic and creative thinker who can tackle complex … Trade Surveillance Knowledge: Familiarity with Market Abuse Regulation and associated monitoring/reporting mechanisms. Industry Standards: Experience with TOGAF or equivalent, and GCP Data Engineer/Cloud Architect certifications. Cyber & Data Security Understanding: An appreciation for the architectural implications of application design and implementation. Modern Data More ❯

Eng London - Hybrid (2 days/week, usually Tues & Fri) 12-month contract Day rate: £800 (umbrella) About the Role Join a critical data remediation programme within the Surveillance Team/Lab, part of the Markets Platform area of a major financial services business. … You will lead design and delivery of stable, scalable, and performant data solutions within a complex GCP architecture, driving simplification and improving pipeline automation to enable timely, secure data delivery to surveillance systems. Key Responsibilities Engineer stable, scalable, performant, accessible, testable, and secure data … SQL expertise for querying, transforming, and troubleshooting data Experience building robust ETL pipelines from diverse source systems Familiarity with CI/CD pipeline automation and enhancement Understanding of Agile delivery frameworks, including Jira and sprint planning Knowledge of Terraform for infrastructure as code provisioning Experience with containerization More ❯

Job Title: Contract Data Architect Location: Reading (Hybrid or On-site as required) Contract Length: 3 Months Sector: Financial Services Overview: We are seeking an experienced Data Architect to support a leading financial services organisation based in Reading. This is a 3-month contract role focused … on designing and implementing scalable, secure, and compliant data solutions in a highly regulated environment. Key Responsibilities: Design and deliver end-to-end data architecture solutions to support business intelligence, reporting, and regulatory needs Lead the integration and optimisation of large-scale data platforms … using Azure Synapse and Databricks Build and maintain robust data pipelines using Python (PySpark) and SQL Collaborate with data engineers, analysts, and stakeholders to ensure data quality, governance, and security Ensure all solutions adhere to financial regulations and internal compliance standards Key Skills & Experience More ❯

Engineer specialised in the Agentic AI Space Required Skill Set- Python Machine Learning APIs (TEST, GraphQL) AWS/Azure cloud Experience Data Pipeline tools (Airflow/Prefect) Contract- 6 month contract Outside IR35 1 stage interview 1-2 days on site in London per month More ❯

in Glasgow on a contract basis. Key Responsibilities: Integrate PowerOn with other enterprise systems such as GIS, OMS, AMI, and SAP Develop APIs, data pipelines, and middleware Customise PowerOn configuration to align with business workflows Perform system testing a click apply for full job details More ❯

Our client is looking for a Azure Data Engineer to join their team on a six month rolling contract. They require a candidate to be able to be office based in Warwick, with the role also being deemed Inside IR35. Your responsibilities: * Collaborate with business stakeholders to manage … and deliver small enhancement requests for a data product. * Update solution designs and implement data workflows using Azure Data Factory (ADF), Logic Apps, and DataOps practices. * Develop and maintain data pipelines integrated with Snowflake. * Ensure effective communication and coordination across teams for … smooth project execution. Your Profile Essential skills/knowledge/experience: * ADF Expert Level * Logic Aps , * Data Ops , * Snowflake Desirable skills/knowledge/experience: * 6 to 8 years of Expierence LA International is a HMG approved ICT Recruitment and Project Solutions Consultancy, operating globally from the largest More ❯

teams' approach to testing - e.g. a framework for test driven development Design and develop automated test scripts with a focus on SQL-based data validation, ensuring the accuracy of data flows and transformation. Integrate automated tests into CI/CD pipelines. Be the person on-point … following: RESTful services, Message Queues, Relational Databases. Experience of SQL scripting and proven capability of writing sophisticated queries to analyse problems, and interrogate data pipelines Experience a range of testing methodologies and techniques including agile, with the ability to write test procedures, plans and report development/implementation. More ❯

teams' approach to testing - e.g. a framework for test driven development Design and develop automated test scripts with a focus on SQL-based data validation, ensuring the accuracy of data flows and transformation. Integrate automated tests into CI/CD pipelines. Be the person on-point … following: RESTful services, Message Queues, Relational Databases. Experience of SQL scripting and proven capability of writing sophisticated queries to analyse problems, and interrogate data pipelines Experience a range of testing methodologies and techniques including agile, with the ability to write test procedures, plans and report development/implementation. More ❯

teams' approach to testing - e.g. a framework for test driven development Design and develop automated test scripts with a focus on SQL-based data validation, ensuring the accuracy of data flows and transformation. Integrate automated tests into CI/CD pipelines. Be the person on-point … following: RESTful services, Message Queues, Relational Databases. Experience of SQL scripting and proven capability of writing sophisticated queries to analyse problems, and interrogate data pipelines Experience a range of testing methodologies and techniques including agile, with the ability to write test procedures, plans and report development/implementation. More ❯

support and assist the manufacturing process. The successful candidate will join the development team responsible for building and integrating Real Time test and data capture systems. The role will focus on developing and maintaining software in a Linux environment, working closely with high-speed multi-channel SFPDP cards … Real Time data flows, and complex multi-threaded processing pipelines. Proficient in C++ and with knowledge of C#, you will play a critical part in the design, development, integration, and verification of test software solutions that are central to our system validation and delivery capabilities. Responsibilities: Design and … develop Real Time test and capture software on Linux systems using C++. Integrate and interface with high-speed SFPDP cards across multi-channel data pipelines. Architect and implement robust, scalable, and maintainable multi-threaded applications. Define and implement data sequencing and synchronisation logic to support deterministic More ❯

support and assist the manufacturing process. The successful candidate will join the development team responsible for building and integrating real-time test and data capture systems. The role will focus on developing and maintaining software in a Linux environment, working closely with high-speed multi-channel SFPDP cards … real-time data flows, and complex multi-threaded processing pipelines. Proficient in C++ and with knowledge of C#, you will play a critical part in the design, development, integration, and verification of test software solutions that are central to our system validation and delivery capabilities. Responsibilities: Design and … develop real-time test and capture software on Linux systems using C++. Integrate and interface with high-speed SFPDP cards across multi-channel data pipelines. Architect and implement robust, scalable, and maintainable multi-threaded applications. Define and implement data sequencing and synchronisation logic to support deterministic More ❯

team in Stevenage, Hertfordshire. As a Software Engineer, you will play a crucial role in designing, developing, and integrating real-time test and data capture systems on Linux platforms. Responsibilities: - Designing and developing real-time test and capture software using C++ in a Linux environment - Integrating and interfacing … with high-speed SFPDP cards across multi-channel data pipelines - Architecting and implementing robust, scalable, and maintainable multi-threaded applications - Defining and implementing data sequencing and synchronisation logic to support deterministic capture and replay - Collaborating with hardware teams to ensure seamless integration and timing alignment between … development in a Linux environment - Proficiency in C# for tooling, test interfaces, or Windows components as needed - Proven experience working with real-time data systems, preferably involving SFPDP or other high-speed digital protocols - Demonstrable skills in multi-threaded and concurrent programming - Experience with low-level I/ More ❯

develop real-time test and capture software on Linux systems using C++. • Integrate and interface with high-speed SFPDP cards across multi-channel data pipelines. • Architect and implement robust, scalable, and maintainable multi-threaded applications. • Define and implement data sequencing and synchronisation logic to support deterministic … Essential: • C++ development in a Linux environment. • C# for tooling, test interfaces, or Windows components as needed. • Proven ability working with real-time data systems, preferably involving SFPDP or other high-speed digital protocols. • Demonstrable skills in multi-threaded and concurrent programming. • Low-level I/O, DMA … principles, memory management, and performance optimisation. • Version control systems (e.g., Git) and CI/CD pipelines. Desirable: • Knowledge of SFPDP (Serial Front Panel Data Port) specifically in defence, aerospace, or data acquisition contexts. • Knowledge of FPGA-based data acquisition systems and synchronisation methods. • Familiarity More ❯

be part of a collaborative environment where innovation, precision, and impact matter. What Youll Be Doing: Designing and developing real-time test and data capture software in a Linux environment using C++. Building scalable, multi-threaded applications that interface with high-speed SFPDP cards and multi-channel data pipelines. Implementing deterministic data sequencing and synchronisation logic for accurate capture and replay. Collaborating with hardware teams to ensure seamless software-to-hardware integration and timing alignment. Leading system-level integration, verification, and validation activities. Conducting performance analysis, debugging, and optimisation of real-time systems. Documenting … and handover. What Were Looking For: Strong C++ development experience in Linux environments; C# knowledge is a plus. Proven background in real-time data systems, ideally with SFPDP or similar high-speed protocols. Expertise in multi-threaded programming, low-level I/O, DMA, and PCIe driver interactions. More ❯