26 to 50 of 157 Data Pipeline Jobs in the UK excluding London

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

of BTs Strategic SIEM development. We are seeking a skilled Software Engineer ideally with knowledge of Elasticsearch (or other SIEM and/or Data platforms) to join our dynamic team. As a Software Engineer you will play a critical role in designing, developing, and maintaining our security information … will be on leveraging Elasticsearch and related technologies to enhance threat detection, incident response, and overall security posture. What you’ll be doing • Data Ingestion and Enrichment: o Configure data pipelines for data ingestion from various sources, primarily from Kafka o Enhance data … towards achieving these. You'll have the following experience • A strong background in Software Engineering, working on projects that involve large amounts of data • Understanding of ETL/Data pipelines Design, develop, and maintain data pipelines Automate data ingestion, transformation, and loading more »

Job Title : Band 6 Data Engineer Location : Gilford Duration : Temporary position, 12 months initially with possibility of extension Hours : Full Time. Mon-Fri, 37 hours per week. Salary : £18.10 per hour. On behalf of our client, we are seeking a Data Engineer who will be Working … within a team of data experts, support the provisioning of data platforms and tooling underpinning numerous corporate and customer facing services. You will aid the design, development, testing and operation of data products in an agile manner with an appropriate technology architecture. A key … opportunities to enhance the service offering by researching, piloting and establishing use cases to embrace leading edge solutions. Duties include: Build and maintain data pipelines and ensure our products are robust, performant, safe and cost effective. Undertake the extraction, transformation, loading, cleansing and testing of data more »

Lead Data Engineer (Contract/ASAP Start) Location| Workplace policy: Fully Remote (Must be based in the UK without visa/sponsorship requirements ) Day Rate: £575 to £625 (outside IR35) Contract Length: 6-month term initially An exciting market-leading tech-savvy organization that works directly alongside some … of the biggest brands in the retail sector is seeking a contract Lead Data Engineer with hands-on experience working directly with C-Suite execs. Technical Expertise Needed At least 4 years recent Databricks (must have) At least 10+ years experience in a Data Engineer role … recently operating at a hands on lead level Ideally educated to degree level in a QS top 100 university Proven experience delivering scalable data pipelines PySpark SQLDevOps/DataOps/CICD Expertise in designing, constructing, administering, and maintaining data warehouses and data lakes Data more »

Director of Data & Systems - SaaS - Up to £85,000 DOE + Benefits Hybrid Working - Sheffield Our Client's mission is dedicated to empowering educational institutions around the globe to achieve compliance and elevate their standards. They offer a comprehensive SaaS solution that merges top-tier professional development with … cutting-edge management tools, all accessible through a single, unified platform. Are you passionate about technology and data? Do you thrive in a fast-paced, collaborative environment? If so, we have the perfect opportunity for you. We are seeking a Director of Data and Systems to … lead our IT and data initiatives, ensuring high-quality, reliable solutions that delight our users and stakeholders. Key Responsibilities: Strategic Planning : Build and implement a roadmap for IT that supports our overall technology and business strategy, driving continuous improvement. Internal Infrastructure Management : Oversee all aspects of internal IT more »

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

new services? We are partnering exclusively with a large organisation to bring you an exciting opportunity for a highly skilled and experienced Lead Data Engineer to be responsible for leading data engineering projects, ensuring the efficient processing and management of data across the organisation. … other benefits. Flexible, hybrid working from Swansea with options for condensed hours. You will get to lead a team working on a modern data platform, collaborating with engineering product teams delivering services and working closely with reporting teams and data scientists to help understand and optimise … their digital services. This role will be instrumental in shaping their data infrastructure, ensuring it supports the strategic objectives. You will leverage cutting-edge technologies, primarily focusing on Microsoft Azure and Databricks, to develop and maintain robust, scalable data solutions. Responsibilities: Leadership and Strategy: Lead the more »

new services? We are partnering exclusively with a large organisation to bring you an exciting opportunity for a highly skilled and experienced Lead Data Engineer to be responsible for leading data engineering projects, ensuring the efficient processing and management of data across the organisation. … other benefits. Flexible, hybrid working from Swansea with options for condensed hours. You will get to lead a team working on a modern data platform, collaborating with engineering product teams delivering services and working closely with reporting teams and data scientists to help understand and optimise … their digital services. This role will be instrumental in shaping their data infrastructure, ensuring it supports the strategic objectives. You will leverage cutting-edge technologies, primarily focusing on Microsoft Azure and Databricks, to develop and maintain robust, scalable data solutions. Responsibilities: Leadership and Strategy: Lead the more »

Specialist/Architect to join our team. The ideal candidate will possess extensive knowledge and hands-on experience with Palantir Foundry, including its data integration, analytics, and operational decision-making capabilities. As a Palantir Foundry Specialist/Architect, you will be responsible for designing, developing, and maintaining data-driven solutions that leverage Palantir Foundry's platform to address complex business challenges. Key Responsibilities: - Design and architect end-to-end data solutions using Palantir Foundry, ensuring alignment with business goals and technical requirements. - Develop scalable and efficient data models, pipelines, and workflows within … the Foundry environment. - Collaborate with stakeholders to understand business needs and translate them into technical requirements and solutions. - **Data Integration:** - Implement data integration solutions to ingest, transform, and manage large datasets from various sources into Palantir Foundry. - Ensure data quality, consistency, and security throughout more »

Data Engineer 12 Month FTC £60,000 - £65,000 Nottingham - Hybrid A Data Engineeris needed at an established company in Nottingham as it continues a period of growth and expansion. The individual will come in at the start of an exciting project and will have the … ability to really make their mark on it. This position is to develop and the firms Central Data Platform. The candidate will have the skills needed to develop solutions using on-premise Microsoft BI and cloud-based Azure technologies and will have a proven experience in applying best … rapidly changing market as well as keeping it up to date with continuous improvement. What does the role actually involve? Design and develop data pipelines and data solutions on the Azure platform. Build and maintain data warehousing solutions on Azure. Develop and maintain data more »

Rivington Rd, Preston Brook, Runcorn, WA7 3DJ Salary: £46,000 - £49,000 Hours: 37.5 hours per week Are you a seasoned expert in data engineering looking to make a significant impact in the healthcare industry? Phoenix Medical Supplies is seeking a Senior DataOps Engineer to lead our data … services across the region. As a Senior DataOps Engineer, you will be instrumental in architecting and implementing solutions across the entire data pipeline, from ingestion to visualisation. The role will require you to showcase your skills in data ingestion, storage, modelling, transfer, hosting, visualisation, analysis … and consumption, all while mentoring and training team colleagues. Key Responsibilities: Design and develop robust data pipelines for ingestion, storage, processing, and interface creation. Produce both machine consumable interfaces and human consumable data products, including standard and ad hoc reports. Maintain and optimise data more »

Senior Data Engineer (GCP) My client a financial services business are looking for a Senior Data Engineer (GCP), to spearhead global initiatives to migrate complex data systems to GCP. You will design, implement, and optimise data pipelines, ensuring data integrity … during migration and utilising GCP services for scalable and efficient data transformation. Salary: Up to £65K Hybrid: 2 days a week Key Responsibilities: Lead data engineering projects in Europe, working with partners for GCP transition. Design and deploy data pipelines with automated data … GCP. Essential Qualifications: 5 years of SQL. 3 years of GCP and its different services. Experience with microservice architecture and data processing pipeline design. Desired Qualifications: Certification in GCP. Financial Services experience. Experience coaching and mentoring data engineers. If this role is of interest, please more »

Converted code is causing failures/performance issues. Your responsibilities; As a Spark Scala Engineer you will be working for the GDT (Global Data Technology) Team, you will be responsible for: Designing, building, and maintaining data pipelines using Apache Spark and Scala Working on an Enterprise … Cloud Services in one of the Clouds (GCP). Mandatory Skills; At least 8+ Years of IT Experience with designing, building, and maintaining data pipelines . At least 4+ Years of experience with designing, building, and maintaining data pipelines using Apache Spark and Scala. Programming languages … Proficiency in Scala and Spark is essential. Familiarity with Python and SQL is often a plus. Big Data technologies: Understanding of HDFS, Kafka, Hive, and cloud platforms is valuable. Data engineering concepts: Knowledge of data warehousing, data pipelines, data modeling more »

Overview: We are seeking a highly skilled SOC Engineer to join our team. The ideal candidate will be responsible for developing and maintaining data pipelines, managing cyber defence applications, and collaborating with SOC Engineers and CTI Analysts to enhance network security. This position … requires on-site work at a UK MOD secure site and the candidate must possess a current DV clearance. Key Responsibilities: Data Pipeline Management: Assist with the continued development and maintenance of data pipelines using NiFi, ensuring efficient data flow and integrity. Signature … preferably UK MOD. Location: Based in the West Midlands or willing to commute/relocate for work. Technical Skills: Proficient in data pipeline management using NiFi. Experienced in using Elastic/Kibana for signature updates and threat detection. Strong system administration skills for operating systems including Unix more »

Senior Data Engineer (Remote) - Stealth SaaS Startup Location: Manchester (Remote Friendly) Salary: £65,000 - £75,000 per annum About Us: We are an innovative stealth start-up set to launch a cutting-edge SaaS product next year. We’re looking for a talented Senior Data Engineer to join our team and build robust data pipelines. Responsibilities: Build, and maintain scalable data pipelines. Maintain the pipeline through Databricks and Apache Beam Seamless data integration and quality. Data storage solutions using GCP (will still be considered with … to bring the AI/ML models into production Databricks for daily data processing Qualifications: Experience in data engineering and pipeline construction. GCP and familiarity with other cloud environments. Understanding of SDLC. Experience with Databricks. Programming skills (Python and SQL). What We Offer more »

Job Title: Cloud Data Engineer Location: Northampton,UK Hybrid work: 2-3 days a week Capgemini is seeking a Cloud Data Engineer t o join Capgemini Financial Services I&D Practice. This is a permanent, full-time position in Northampton, UK and this represents a unique … opportunity for someone to enhance their I&D Consulting career. What you’ll do and bring: Proficiency in AWS services relevant to data engineering such as S3, Glue, EMR, Athena, and Lambda. Hands-on experience with Snowflake, Redshift cloud data warehousing solutions. Familiarity with DBT (Data … Glue, Starburst, Snowflake, Redshift, Starburst, Hybrid(on-prem to cloud) for building data pipelines and ETL processes. Experience with data pipeline orchestration tools like Apache Airflow . Knowledge of SQL and ability to write complex queries for data transformation and analysis. Understanding of more »

Job Title: Cloud Data Engineer Location: Knutsford, UK Hybrid work: 2-3 days a week Capgemini is seeking a Cloud Data Engineer t o join Capgemini Financial Services I&D Practice. This is a permanent, full-time position in Knutsford, UK and this represents a unique … opportunity for someone to enhance their I&D Consulting career. What you’ll do and bring: Proficiency in AWS services relevant to data engineering such as S3, Glue, EMR, Athena, and Lambda. Hands-on experience with Snowflake, Redshift cloud data warehousing solutions. Familiarity with DBT (Data … Glue, Starburst, Snowflake, Redshift, Starburst, Hybrid(on-prem to cloud) for building data pipelines and ETL processes. Experience with data pipeline orchestration tools like Apache Airflow . Knowledge of SQL and ability to write complex queries for data transformation and analysis. Understanding of more »

Data Engineer Our client based near Manchester is looking for a skilled Data Engineer to join their agile team. In this role, you'll play a pivotal part in designing, building, and maintaining data pipelines, ensuring the integrity and availability of data for various stakeholders, including BI Teams, Analytics, and Data Scientists. Responsibilities: Design, build, and automate data flow processes and ELTs to provide comprehensive data from business systems. Collaborate with product development teams to ensure best practices and data integrity from … source to destination. Assist in integrating data between systems, including third-party platforms. Write specifications and documentation for data processes. Continuously explore new solutions, tools, and processes to enhance data operations. Requirements: Bachelor's or Master's degree in Computer Science, related field, or more »

Job summary NUH Digital Services are looking for a number of experienced Senior Data Warehouse Analyst/Developer to join our Data Warehouse team for a period of 18 months, to support us during the implementation of a new Patient Administration System. The potential post-holders … need to familiarise yourself with the full job description and person specification documents attached to this advert. Working closely with colleagues in the Data Warehousing and other Digital & Information teams, the post holder will be expected to take a leading role in the analysis, design, and development of … both standard as well as ad-hoc data warehousing packages for the provision of data for reports, dashboards, downstream systems, and for data migration purposes. As an expert in the field of Trust data, data validation, ETL mechanisms, and data more »