1 to 25 of 265 Data Pipeline Jobs in the East of England

Data Engineer will be responsible for, overseeing the design, building and optimization of our data orchestration and data pipeline architecture, as well as optimizing data collection and flow for cross functional teams. The ideal candidate is an experienced technical oriented Lead … Data Engineer who enjoys data orchestration and optimizing data systems and building them from the ground up. This role requires a deep understanding of data management/data engineering principles as they apply to scalability, and design of dynamic flexible … data sources using SQL, Azure Data Factory, Spark/Synapse technologies. Support analytics tools that utilize the data pipeline to provide actionable insights into key business performance metrics. Work with stakeholders including the Executive, Product, Data and Technology teams to assist More ❯

Office Based Position - Cambridge, UK with hybrid working available Your role in our team: At bit.bio, we’re setting our sights on embedding data and AI into our technology stack in new and innovative ways. We are looking for an experienced and passionate Senior Data & AI … highly translational biotech research where you will be directly contributing to cutting edge discoveries. You will support the development of a data pipeline for continuous ingestion of new datasets from external/internal sources into the relevant databases so that it may be integrated into decision making … SQL and NoSQL databases, including Postgres and Cassandra. Experience designing and implementing knowledge graphs for data integration and analysis. Data pipeline and workflow management tools: Luigi, Airflow, etc. AWS cloud services: EC2, S3, Glue, Athena, API Gateway, Redshift. Experience with object-oriented and scripting languages More ❯

Job DescriptionJob Description At EverQuote, we're embarking on an exciting transformation in our approach to data management, moving towards a Data Mesh as our future state. As the Engineering Manager - Data Enablement, you will be responsible for leading a team of engineers focused … on building and scaling the EQ data platform. You will provide technical leadership to drive best practices in data engineering, data governance, and data product development, while also being a people leader—mentoring engineers, fostering career growth, and ensuring effective team collaboration. … integrate cloud infrastructure, CI/CD practices, and observability into data workflows. Collaborate with DS and analytics teams to ensure a seamless pipeline from raw data to production-ready models and insights. Lead and grow a team of high-performing data engineers, fostering More ❯

to hire a mid-level Data Engineer (2+ years' experience) who has worked with Azure and Databricks , and can contribute immediately across pipeline development, categorisation, and optimisation for AI/ML use cases. You'll work with diverse datasets spanning prepaid, financial services, gambling, and payments, supporting … s data architecture and ML-enablement strategies. In the first few months, you would have: Taken ownership of a data pipeline or transformation flow within Databricks and contributed to its optimisation and reliability. Worked across raw and curated datasets to deliver categorised and enriched data … team and be willing to occupy various roles to allow the team to adjust work more effectively. Responsibilities of the role: Data Pipeline Development: Build and maintain batch and streaming pipelines using Azure Data Factory and Azure Databricks. Data Categorisation & Enrichment: Structure unprocessed More ❯

we look to establish and develop a hub for our wide-ranging innovation work across the Trust. We recognise that the Trust's data represents one of our most potent assets to support innovation, as well as research and quality improvement. But we have yet to fully exploit … its potential. Our new Clinical Data Science and Engineering Unit will be the vital engine room for curating, sharing and utilising our data for novel purposes outside of the routine operations of hospital. It will work collaboratively with the Trust's core digital and data … implementation and optimization of data management systems (Databases/Datasets/DataLakes/Lakehouses) Ability to design and implement data pipeline technologies to extract sensitive information from databases/datasets to help understand a process or solve a problem and make informed recommendations based on More ❯

Summary Digital Futures is looking for aspiring data engineering consultants to join our Future Talent Programme. The Future Talent Programme offers a unique entry point for individuals looking to start their career in technology. The programme begins with immersive training at the Digital Futures Academy, where trainees gain … to-one mentoring and group project work. Certification – As part of your graduation from the Digital Futures Academy, you will achieve the AWS Data Engineer Associate certification. The cost of your certification is covered by Digital Futures. Employment – Once your deployment has been secured, we will offer you … with data scientists and analysts to ensure that data is properly formatted and accessible. Troubleshooting and resolving data pipeline and infrastructure issues. Helping to design and maintain data warehouses, data lakes, and other data storage solutions. Developing More ❯

Systems Architecture - AWS data engineer Job Description Location: UK - Will require travel to customer site ](Belfast) Job Summary: We are seeking a skilled and experienced AWS Data Engineer to join our team. The successful candidate will be responsible for implementing, and managing data architecture … solutions for our customers, with a strong emphasis on Cloud technologies (Preferably AWS) and tooling. This role requires understanding of data modelling, database design, and data integration techniques. Key Responsibilities -Data Ingestion and Extraction: Design and implement efficient data ingestion pipelines to … and from databases and file storage services. -Data Transformation and Cleaning: Transform and clean raw data to ensure data quality and consistency. -Data Pipelines: Build, maintain, and optimise data pipelines to automate data flows and enable real-time More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

Social network you want to login/join with: Senior Data Platform Engineer, watford, hertfordshire col-narrow-left Client: easyJet Location: watford, hertfordshire, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 1 Posted: 06.06.2025 Expiry Date: 21.07.2025 col-wide Job Description … big-scale stuff and we’re still growing. Job Purpose With a big investment into Databricks, and with a large amount of interesting data, this is the chance for you to come and be part of an exciting transformation in the way we store, analyse and use data … environment with Spark Streaming, Kafka, DLT or Beam. Understanding of the challenges faced in the design and development of a streaming data pipeline and the different options for processing unbounded data (pubsub, message queues, event streaming etc) Understanding of the most commonly used Data More ❯

Salary: Competitive + Benefits + Bonus If you are an agile, committed, and innovative analytics engineer skilled in defining and shaping long-term data management strategy, then we want to hear from you! Join us to help improve peoples' lives and make healthcare better for everyone! Why Pharmanovia … are seeking a highly skilled and motivated Senior Analytics Engineer to join our team to help define and build Pharmanovia's long-term data management strategy. Pharmanovia is aiming to build a highly autonomous self-service data and analytics platform, which can extract and align data … with container orchestration on AWS (Amazon EKS) Strong understanding of DevOps principles and experience with CI/CD pipelines using tools like AWS Code Pipeline, Jenkins, or GitLab CI/CD Excellent problem-solving skills and ability to troubleshoot complex issues in distributed systems Effective communication skills with the More ❯

willingness to contribute across the full data pipeline.Have the ability and willingness to support other areas of the data science pipeline when required.Have a strong academic background, holding at least a 2:1 degree in Computer Science, Data Engineering, or a related field.Ideally … . Your cover letter should include why you are suitable for the role and answers to the following: Describe a relevant data pipeline or infrastructure project that you've built or maintained and explain how you would approach adopting similar to support a football club.Explain an example … of how you proactively identified and rectified data quality issues in a data pipeline that you were working with.How would you approach balancing the creation of production-ready code while being adaptable and supporting others within a relatively small technical team? Closing date for applications More ❯

big-scale stuff and we’re still growing. Job Purpose With a big investment into Databricks, and with a large amount of interesting data, this is the chance for you to come and be part of an exciting transformation in the way we store, analyse and use data in a fast paced organisation. You will join as a Senior Platform Data Engineer providing technical leadership to the Data Engineering team. You will work closely with our Data Scientists and business stakeholders to ensure value is delivered through our solutions. Job … environment with Spark Streaming, Kafka, DLT or Beam. Understanding of the challenges faced in the design and development of a streaming data pipeline and the different options for processing unbounded data (pubsub, message queues, event streaming etc) Understanding of the most commonly used Data More ❯

Job Description Description: We are actively seeking a Data Engineer responsible for designing, building, and maintaining infrastructure that supports data storage, processing, and retrieval. The role involves working with large data sets and developing data pipelines to transfer data from … source systems to data warehouses, data lakes, and other storage and processing systems. The Data Engineer will collaborate with stakeholders to address data-related technical issues and support their data infrastructure needs during the development, maintenance, and sustainment of the … KR data architecture and data-driven solutions. Note: Due to federal security clearance requirements, applicants must be U.S. citizens or Permanent Residents with the ability to obtain an active Secret clearance. This is a contract-to-hire position. Applicants should be willing to work on a More ❯

National Lottery that delivers more money to good causes. Key Responsibilities: BI Operations & System Monitoring: Ensure day-to-day smooth running of BI data pipelines, reporting systems, and cloud-based ETL/ELT workflows (AWS, Redshift, Glue, Airflow).Monitor system uptime and performance, proactively addressing issues to maintain … followed. Approve or review changes published by Dev/Test teams on Production environments. SLA & Performance Compliance: Maintain SLAs for uptime, latency, and data freshness, using monitoring tools like AWS CloudWatch. Track and optimise system performance, reducing recurring issues. Data Governance & Quality: Oversee data … Expertise: Strong SQL scripting, database optimisation, and troubleshooting. Operations Experience: Incident & Problem Management: Identify root causes, troubleshoot, and implement fixes for data pipeline and platform issues. Manage L2 escalations effectively, ensuring timely resolution and minimizing business impact. Monitoring & Automation for Stability: Track data pipeline More ❯

on the world. This community needs people like you: dedicated, agile and above all audacious so we can truly drive science forward. The Data Engineer will be responsible for ensuring data availability, reliability and security, through the administration of modern data warehousing solutions, as … well as spearheading the design, build, and maintenance of robust data products and pipelines for AI. This position reports to the AI Sales Integration Manager and is part of the Data Science and AI Team at Abcam/Danaher and can either be onsite in Cambridge … a 12-month fixed-term contract with the possibility of extension. In this role, you will have the opportunity to: Build and optimize data products and pipelines from various sources, ensuring data availability, quality, and reliability. Administer and maintain databases (Snowflake, Redshift), ensuring optimal performance and More ❯

Dublin, Ireland Work from the Client office 5 days a week Day rate: GBP 400/day Responsibilities: • Design, develop, and maintain robust data pipelines and ETL processes using Snowflake on AWS. • Implement data warehousing solutions, ensuring efficient storage, retrieval, and transformation of large datasets. • Collaborate … with data analysts, scientists, and other stakeholders to define and fulfil data requirements. • Optimise performance and scalability of Snowflake data warehouse, ensuring high availability and reliability. • Develop and … maintain data integration solutions, ensuring seamless data flow between various sources and Snowflake. • Monitor, troubleshoot, and resolve data pipeline issues, ensuring data quality and integrity. • Stay up-to-date with the latest trends and best practices in data engineering More ❯