151 to 175 of 198 Data Pipeline Jobs in Yorkshire

Timeform Sporting Life is one of the biggest sports editorial businesses in the UK. We consume, analyze and package sports data including horse racing, football and greyhounds. Powering front end products for some of the biggest bookmakers, we reach millions of users. Our skilled team of analysts are globally renowned for accuracy, credibility, and judgement. Working in Technology … this exciting opportunity for a Data Engineer to join our growing team and contribute to the design, construction, enhancement, and maintenance of digital products and platforms. You will have the experience to challenge the existing with newer suitable solutions that will feed into our future success. What you’ll do Build and maintain data pipelines that … move data from various sources (e.g., databases, APIs, files) into data lakes or warehouses where it can be easily accessed and analyzed. Automate data collection, ensuring continuous availability of fresh, accurate data for business and AI algorithms. Configure storage solutions (such as cloud storage, data lakes, and data More ❯

you will collaborate across multiple teams to lead AI & ML model training, validation, experimentation, and monitoring, ensuring reliable performance in production. This full-stack role involves working with Data teams to ensure system compatibility and efficient handling of workloads, as well as with Software Development Teams for POC experiments and deployment. You will design and implement scalable AI … and ML solutions, fostering innovation and experimentation. Your work will significantly impact operational efficiency, safety, and data-driven decision-making, supporting A-SAFE’s mission to prevent workplace injuries and safeguard lives globally. In This Role, You Will Proven experience with Generative AI, LLMs, RAG, transformers, diffusion models, and recommender systems. Developing and managing data pipelines … including data gathering, cleaning, augmentation, labelling, and managing vector databases for RAG workflows. Model deployment, monitoring, versioning, and continuous improvement using frameworks like MLflow and AWS SageMaker. Experience with deep learning frameworks (TensorFlow, PyTorch), AWS SageMaker, Bedrock, Lambda; familiarity with Azure AI Foundry is a plus. Knowledge of software engineering best practices (version control, CI/CD, code More ❯

s redefining real-time analytics and infrastructure at scale. This is a fantastic opportunity to work with a passionate, technically strong team delivering critical infrastructure to support live data pipelines, scalable compute, and a rapidly evolving platform — all in a high-growth, high-performance environment. Role Highlights: Salary: Up to £85,000 Location: Hybrid (Leeds) Tech: AWS, Kubernetes … PowerShell), and GitLab CI Enabling observability across services using Prometheus , OpenTelemetry , and custom tooling Implementing secure deployment practices, performance tuning, and cost optimisation strategies Collaborating with engineers and data scientists to ensure platform reliability and rapid iteration What We’re Looking For: Solid experience as an SRE or DevOps Engineer in cloud-first environments Strong command of AWS More ❯

s redefining real-time analytics and infrastructure at scale. This is a fantastic opportunity to work with a passionate, technically strong team delivering critical infrastructure to support live data pipelines, scalable compute, and a rapidly evolving platform — all in a high-growth, high-performance environment. Role Highlights: Salary: Up to £85,000 Location: Hybrid (Leeds) Tech: AWS, Kubernetes … PowerShell), and GitLab CI Enabling observability across services using Prometheus , OpenTelemetry , and custom tooling Implementing secure deployment practices, performance tuning, and cost optimisation strategies Collaborating with engineers and data scientists to ensure platform reliability and rapid iteration What We’re Looking For: Solid experience as an SRE or DevOps Engineer in cloud-first environments Strong command of AWS More ❯

to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by diversity and inclusion, talented peers, comprehensive benefits and career development opportunities. Come make … We are looking for a Software Engineer with solid Python or Java experience and a passion for cloud/platform technologies skills to join our talented and innovative data engineering team. This role sits within a part of Optum UK responsible for taking large data sets from various sources and converting them into meaningful insights. You … will get to work with Big Data utilising cutting edge Cloud services and technologies. Your work will help healthcare teams to identify opportunities to be more efficient, improve services, and design/optimise healthcare interventions. We are a collaborative team, we work hard and are passionate about what we do, but we ensure we have fun along the More ❯

Senior Data Engineer - Azure - Leeds I'm looking for a Senior Azure Data Engineer for a client near to Leeds. The company are building a greenfield Data Platform in Azure and Snowflake and so looking for someone who has been heavily involved in building end to end data platforms in Azure using … Synapse, Data Factory, Data Lake and SQL. Role & Responsibilities Building end to end data pipelines in Azure (Data Factory and SQL) Building workflows in SQL, Spark and DBT Data and dimensional modelling Skills & Qualifications Azure Data factory, Synapse and SSIS Python/Park/PySpark Ideally Snowflake and More ❯

The Role We are looking for a Solution Architect to lead the design and implementation of smart, scalable data solutions that enhance how we understand and engage with our supporters. Latest Videos [Content appears irrelevant and not related to the job description; consider removing or replacing with relevant content] This role involves shaping data flow across … our ecosystem, primarily using Azure and Databricks, integrated with platforms like Tableau, Cortex, and MS Dynamics. You will leverage GraphQL for real-time data access, ensuring data is well-structured, secure, and accessible for actionable insights. This is a leadership position at the intersection of business needs and technical delivery. You will serve as a key … valuable solutions. You will also lead a growing internal team and manage external partners, establishing best practices, fostering collaboration, and driving transformative change to build a scalable, modern data function. Who we are We are one of the most famous football clubs worldwide, with a rich heritage and history of success. Learn more about our history here . More ❯

Social network you want to login/join with: ZONTAL is revolutionizing data management and digital transformation for the life sciences industry. Our cutting-edge platform enables seamless data integration, standardization, and accessibility—empowering scientists and organizations to drive innovation, enhance compliance, and accelerate breakthroughs in healthcare and biotechnology. ZONTAL has offices in the US, Europe … the job. Your Responsibilities: Be a subject matter expert in the ZONTAL Platform and its customer use-cases. Analyze business requirements for customer-facing projects, with emphasis on data flows and data formats as they feed into & out of the ZONTAL Platform. Create functional and design specifications for the proposed solution, focusing on overall Platform architecture … sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker More ❯

Social network you want to login/join with: ZONTAL is revolutionizing data management and digital transformation for the life sciences industry. Our cutting-edge platform enables seamless data integration, standardization, and accessibility—empowering scientists and organizations to drive innovation, enhance compliance, and accelerate breakthroughs in healthcare and biotechnology. ZONTAL has offices in the US, Europe … the job. Your Responsibilities: Be a subject matter expert in the ZONTAL Platform and its customer use-cases. Analyze business requirements for customer-facing projects, with emphasis on data flows and data formats as they feed into & out of the ZONTAL Platform. Create functional and design specifications for the proposed solution, focusing on overall Platform architecture … sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker More ❯

Social network you want to login/join with: ZONTAL is revolutionizing data management and digital transformation for the life sciences industry. Our cutting-edge platform enables seamless data integration, standardization, and accessibility—empowering scientists and organizations to drive innovation, enhance compliance, and accelerate breakthroughs in healthcare and biotechnology. ZONTAL has offices in the US, Europe … the job. Your Responsibilities: Be a subject matter expert in the ZONTAL Platform and its customer use-cases. Analyze business requirements for customer-facing projects, with emphasis on data flows and data formats as they feed into & out of the ZONTAL Platform. Create functional and design specifications for the proposed solution, focusing on overall Platform architecture … sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker More ❯

Social network you want to login/join with: ZONTAL is revolutionizing data management and digital transformation for the life sciences industry. Our cutting-edge platform enables seamless data integration, standardization, and accessibility—empowering scientists and organizations to drive innovation, enhance compliance, and accelerate breakthroughs in healthcare and biotechnology. ZONTAL has offices in the US, Europe … the job. Your Responsibilities: Be a subject matter expert in the ZONTAL Platform and its customer use-cases. Analyze business requirements for customer-facing projects, with emphasis on data flows and data formats as they feed into & out of the ZONTAL Platform. Create functional and design specifications for the proposed solution, focusing on overall Platform architecture … sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker More ❯

Social network you want to login/join with: ZONTAL is revolutionizing data management and digital transformation for the life sciences industry. Our cutting-edge platform enables seamless data integration, standardization, and accessibility—empowering scientists and organizations to drive innovation, enhance compliance, and accelerate breakthroughs in healthcare and biotechnology. ZONTAL has offices in the US, Europe … the job. Your Responsibilities: Be a subject matter expert in the ZONTAL Platform and its customer use-cases. Analyze business requirements for customer-facing projects, with emphasis on data flows and data formats as they feed into & out of the ZONTAL Platform. Create functional and design specifications for the proposed solution, focusing on overall Platform architecture … sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker More ❯

United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 4 Posted: 10.06.2025 Expiry Date: 25.07.2025 col-wide Job Description: ZONTAL is revolutionizing data management and digital transformation for the life sciences industry. Our cutting-edge platform enables seamless data integration, standardization, and accessibility—empowering scientists and organizations to drive innovation … the job. Your Responsibilities: Be a subject matter expert in the ZONTAL Platform and its customer use-cases. Analyze business requirements for customer-facing projects, with emphasis on data flows and data formats as they feed into & out of the ZONTAL Platform. Create functional and design specifications for the proposed solution, focusing on overall Platform architecture … sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker More ❯

United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 3 Posted: 10.06.2025 Expiry Date: 25.07.2025 col-wide Job Description: ZONTAL is revolutionizing data management and digital transformation for the life sciences industry. Our cutting-edge platform enables seamless data integration, standardization, and accessibility—empowering scientists and organizations to drive innovation … the job. Your Responsibilities: Be a subject matter expert in the ZONTAL Platform and its customer use-cases. Analyze business requirements for customer-facing projects, with emphasis on data flows and data formats as they feed into & out of the ZONTAL Platform. Create functional and design specifications for the proposed solution, focusing on overall Platform architecture … sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker More ❯

a Senior Machine Learning Engineer to take responsibility for developing AI/ML software products. What we are looking for Develop AI/ML software products including large data set and deploy the solution for production usage Design, develop and maintain the large-scale data infrastructure required for the AI/ML projects Leverage on understanding … successfully deployed to production Hands-on experience with cloud technologies for machine learning development in AWS, alongside solid expertise in MLOps practices, encompassing the development of ML pipelines, data pipelines, and the deployment of ML applications to production Proficient working knowledge of various AI/ML techniques and experience with different frameworks, as well as extensive experience in … handling and developing high-volume heterogeneous data (both batch and stream), preferably with hands-on development in DBT, and a solid understanding of cloud data storage technologies and data quality Demonstrated ability to align and analyse requirements, drive design solutions, break down work into stories and tasks, support and mentor team members, and ensure More ❯

help the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world. YOUR ROLE Writing and maintaining reusable Python scripts for data analysis and testing of Real-Time streaming Data Pipelines. Writing and executing SQL queries on BigQuery to extract and manipulate data. Understand requirements to create test plans … test cases, test data and automate using Pytest. Creating Jenkins CI/CD jobs to integrate tests and ensure execution for every code deployment. Create and manage tasks in JIRA to structure work, log efforts and communicate with stakeholders. Collaborating with BAs, ITPMs and Data Architects in understanding business problems and working on several POCs to … needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, cloud and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of €22.5 billion. Get the future you want | www.capgemini.com More ❯

help the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world. YOUR ROLE Writing and maintaining reusable Python scripts for data analysis and testing of Real-Time streaming Data Pipelines. Writing and executing SQL queries on BigQuery to extract and manipulate data. Understand requirements to create test plans … test cases, test data and automate using Pytest. Creating Jenkins CI/CD jobs to integrate tests and ensure execution for every code deployment. Create and manage tasks in JIRA to structure work, log efforts and communicate with stakeholders. Collaborating with BAs, ITPMs and Data Architects in understanding business problems and working on several POCs to … needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, cloud and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of €22.5 billion. Get the future you want | www.capgemini.com More ❯

Architect- Cloudera Location: Leeds (2days/week Onsite) Duration: 06+ Months Job Summary: We are looking for an experienced Kafka Architect to design and implement scalable, high-throughput data streaming solutions using Apache Kafka on Cloudera (on-premises). The ideal candidate will have a strong background in distributed systems, data pipelines, and real-time data … on-premises infrastructure. -Lead the setup, configuration, and optimization of Kafka clusters. -Define standards and best practices for Kafka producers, consumers, and streaming applications. -Integrate Kafka with various data sources, storage systems, and enterprise applications. -Monitor Kafka performance and ensure high availability, fault tolerance, and data security. -Collaborate with DevOps, Data Engineering, and Application … teams to support real-time data needs. -Automate deployment and configuration using tools like Ansible, Terraform, or Cloudera Manager. -Provide technical leadership and mentorship to junior team members. Required Skills: -Strong hands-on experience with Apache Kafka (including Kafka Connect, Kafka Streams). -Experience with Cloudera distribution for Kafka on on-premises environments. -Proficiency in designing high-volume More ❯

Data Quality and Governance Manager - Hybrid - Leeds - £65,000 My client is looking for a Data Quality & Governance Manager to shape how data is trusted and used throughout their organisation. They are looking for someone to lead efforts in ensuring the consistency, reliability, and governance of data. This role will work closely with business … and technical teams to develop and implement frameworks for data quality, governance, and master data management, using Snowflake and Microsoft Azure. Requirements: Experience in data governance and data quality management Strong knowledge of data warehouse architecture and cloud platforms, especially Snowflake and Microsoft Azure. Proven experience designing and managing data pipelines in cloud environments with integrated data governance principles. Hands-on experience with data migrations Strong experience in working with data engineers, business stakeholders, and data stewards. Please Note: This is role for UK residents only. This role does not offer Sponsorship. You must have the right to work in More ❯

Data Engineer - Remote - £65,000 My client, a growing, high-impact consultancy is looking for a hands-on Data Engineer to join its expert team. Working across a mix of exciting, high-profile clients in various different sectors. You will join their expanding team to build robust platforms and create intelligent solutions. If you thrive in … fast-paced environments, enjoy variety in your work, and want to build meaningful data products using Snowflake, dbt, Python, and SQL, this role is for you. Requirements: Experience designing and building scalable data pipelines using Snowflake, dbt, and Python Proven experience as a Data Engineer or Analytics Engineer Great knowledge of Snowflake SQL optimization … and warehouse design experience Strong Python skills for scripting, orchestration, and data manipulation Please Note: This is a permanent role for UK residents only. This role does not offer Sponsorship. You must have the right to work in the UK with no restrictions. Some of our roles may be subject to successful background checks including a DBS and More ❯

Data Engineer - Remote - £65,000 My client, a growing, high-impact consultancy is looking for a hands-on Data Engineer to join its expert team. Working across a mix of exciting, high-profile clients in various different sectors. You will join their expanding team to build robust platforms and create intelligent solutions. If you thrive in … fast-paced environments, enjoy variety in your work, and want to build meaningful data products using Snowflake, dbt, Python, and SQL, this role is for you. Requirements: Experience designing and building scalable data pipelines using Snowflake, dbt, and Python Proven experience as a Data Engineer or Analytics Engineer Great knowledge of Snowflake SQL optimization … and warehouse design experience Strong Python skills for scripting, orchestration, and data manipulation Please Note: This is a permanent role for UK residents only. This role does not offer Sponsorship. You must have the right to work in the UK with no restrictions. Some of our roles may be subject to successful background checks including a DBS and More ❯

now to be part of our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of roles including but not … limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for further analysis and investigation. Working with others is key, you could be working with other engineers, developers More ❯

now to be part of our award-winning team at Sparta Global. About This Role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of roles including but not … limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for further analysis and investigation. Working with others is key, you could be working with other engineers, developers More ❯

now to be part of our award-winning team at Sparta Global. About This Role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of roles including but not … limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for further analysis and investigation. Working with others is key, you could be working with other engineers, developers More ❯

Job title: Solution Architect Location: Leeds, UK (Hybrid- 3 days/week) Type: Contract Client: Wipro Job description As part of the CTO Data Ingestion Service, the incumbent will be required to: Designing and architecting scalable, real-time systems in Kafka, with a focus on on-premise Cloudera open source Kafka and disaster recovery aspects. Configuring, deploying, and … maintaining Kafka clusters to ensure high availability, resiliency, and scalability, including understanding and explaining features like KRAFT. Integrating Kafka with other data processing tools and platforms such as Kafka Streams, Kafka Connect, Spark Streaming, Schema Registry, Flink, and Beam. Collaborating with cross-functional teams to understand data requirements and design solutions that meet business needs. Implementing … security measures to protect Kafka clusters and data streams. Monitoring Kafka performance, troubleshooting issues, and enhancing disaster recovery strategies. Providing technical guidance and support to development operations teams. Staying updated with the latest Kafka features, industry practices, and real-time technologies like Spark. Mandatory keywords: Experience with on-premise/cloudera open source Kafka Focus on disaster recovery More ❯