24 of 24 Big Data Jobs in the North of England

frameworks. About the Role We are looking for an experienced Test Manager to lead quality assurance and testing activities within a large-scale cloud data engineering programme operating across a modern AWS-native technology ecosystem, including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, DynamoDB, and related cloud services. … hands-on technical leadership with test strategy ownership . You will be responsible for defining and driving the overall testing approach across automated pipelines, data workflows, cloud infrastructure, and release delivery while managing risks, improving testing standards, and mentoring engineering teams. The ideal candidate will have strong experience ...

Title: Data Engineer Location: UK Job Type: Full-time About Scrumconnect: Scrumconnect is a leading force in technology consultancy, proudly contributing to over 20% of the UK’s most significant citizen-facing public services. Our award-winning team has made a substantial impact, delivering more than 64 services … collaboration, sharing knowledge, and continuous learning to address and solve complex challenges. Our mission is to combine advanced software engineering, human-focused design, and data-driven insights to deliver unparalleled service to our clients. About the role and responsibilities: We are looking to fill the position of Data ...

more, decide faster, and act with confidence across complex domains. We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … solutions are secure-by-design, built from the ground up to protect sensitive information and ensure resilience in contested environments. We simplify complex heterogeneous data sources into a rich picture of fused intelligence, providing a digital backbone that underpins success. We operate as small teams delivering fast value, blending ...

more, decide faster, and act with confidence across complex domains. We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … solutions are secure-by-design, built from the ground up to protect sensitive information and ensure resilience in contested environments. We simplify complex heterogeneous data sources into a rich picture of fused intelligence, providing a digital backbone that underpins success. We operate as small teams delivering fast value, blending ...

Broster Buchanan are partnered global business within the food industry with a turnover of £8Bn+ who are looking for a Fabric Data Engineer to join as they set up an exciting new project team within the group. This would suit an experienced Azure Data Engineer with good Fabric … bonus is on offer. Responsibilities: Help set up and deliver the future direction of strategy and approach to the management, storage and usage of data Collaborate with their wider technology and business teams to ensure goals and visions are aligned and achieved Help with setting up the data ...

Broster Buchanan are partnered global business within the food industry with a turnover of £8Bn+ who are looking for a Fabric Data Engineer to join as they set up an exciting new project team within the group. This would suit an experienced Azure Data Engineer with good Fabric … bonus is on offer. Responsibilities: Help set up and deliver the future direction of strategy and approach to the management, storage and usage of data Collaborate with their wider technology and business teams to ensure goals and visions are aligned and achieved Help with setting up the data ...

Employment Type: Full-time About the Role We are seeking a highly experienced Databricks Architect to lead the design, development, and optimization of scalable data platforms and advanced analytics solutions. This role requires a deep understanding of Databricks Lakehouse architecture, big data engineering, cloud ecosystems, and enterprise … data strategy. The ideal candidate will partner with business and technology leaders to deliver high-performance data systems that support analytics, AI/ML, and real-time insights. Key Responsibilities Architecture & Solution Design Architect end-to-end Databricks Lakehouse solutions including data ingestion, ETL/ELT pipelines ...

Title: AWS Data Engineer Contract: 6 Months Location: Remote, UK Skills: Advanced knowledge of data management tools including SQL/DBMS, MongoDB, Hadoop and/or other big data technologies. Advanced programming skills in Java, Python, R, C++, C#, etc. Knowledge of statistical and data mining techniques (regression, decision trees, clustering, neural networks, etc.). Experience with data visualization tool is plusExposure to online, mobile, and social data is a plus. Intellectual curiosity, along with excellent problem-solving and quantitative skills, including the ability to disaggregate issues, identify root causes and recommend ...

Title: AWS Data Engineer Contract: 6 Months Location: Remote, UK Skills: Advanced knowledge of data management tools including SQL/DBMS, MongoDB, Hadoop and/or other big data technologies. Advanced programming skills in Java, Python, R, C++, C#, etc. Knowledge of statistical and data mining techniques (regression, decision trees, clustering, neural networks, etc.). Experience with data visualization tool is plusExposure to online, mobile, and social data is a plus. Intellectual curiosity, along with excellent problem-solving and quantitative skills, including the ability to disaggregate issues, identify root causes and recommend ...

Data Engineer Opportunity, DV-cleared only London Manchester Bristol A progressive, leading-edge UK consulting and technology organisation is hiring Data Engineers to deliver mission-critical work across defence and security programmes, building modern data platforms and production-grade pipelines that enable better decisions at pace. Active … clearance is essential, we are seeking DV cleared candidates only. The role You'll design and deploy production-grade data pipelines, from ingestion through to consumption, within a modern big data architecture. Work is delivery focused and delivered using agile engineering practices. Typical responsibilities Build and operate ...

Copello are looking for an experienced Data Engineer to join a growing team delivering data solutions within secure government environments. You'll design and develop scalable data pipelines, integrate data from multiple sources, and build high-performance data platforms that support analytics and operational decision … making. Key Responsibilities: Design and build data pipelines and ingestion processes Develop scalable data integration and processing solutions Optimise data platforms for performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop ...

Location –(Remote) UK Job Description Your role: As a Confluent Consulting Engineer, you will be responsible for designing, developing, and maintaining scalable real-time data pipelines and integrations using Kafka and Confluent components. You will collaborate with data engineers, architects, and DevOps teams to deliver robust streaming solutions. … Kafka (any distribution: open-source, Confluent, Cloudera, AWS MSK, etc.) · Strong proficiency in Java, Python, or Scala · Solid understanding of event-driven architecture and data streaming patterns · Experience deploying Kafka on cloud platforms such as AWS, GCP, or Azure · Familiarity with Docker, Kubernetes, and CI/CD pipelines · Excellent ...

Location (Remote) UK Job Description Your role: As a Confluent Consulting Engineer, you will be responsible for designing, developing, and maintaining scalable real-time data pipelines and integrations using Kafka and Confluent components. You will collaborate with data engineers, architects, and DevOps teams to deliver robust streaming solutions. … Kafka (any distribution: open-source, Confluent, Cloudera, AWS MSK, etc.) Strong proficiency in Java, Python, or Scala Solid understanding of event-driven architecture and data streaming patterns Experience deploying Kafka on cloud platforms such as AWS, GCP, or Azure Familiarity with Docker, Kubernetes, and CI/CD pipelines Excellent ...

Location (Remote) UK Job Description Your role: As a Confluent Consulting Engineer, you will be responsible for designing, developing, and maintaining scalable real-time data pipelines and integrations using Kafka and Confluent components. You will collaborate with data engineers, architects, and DevOps teams to deliver robust streaming solutions. … Kafka (any distribution: open-source, Confluent, Cloudera, AWS MSK, etc.) Strong proficiency in Java, Python, or Scala Solid understanding of event-driven architecture and data streaming patterns Experience deploying Kafka on cloud platforms such as AWS, GCP, or Azure Familiarity with Docker, Kubernetes, and CI/CD pipelines Excellent ...

Hear from the team | Check out our Career Launchpad What you'll be working on: The Analytics Engineering team is responsible for building downstream data models from backend services with the desire to make our Data Warehouse a genuine competitive advantage for Monzo. We want a discipline capable … building an amazing Data Warehouse to support decision making, Business Intelligence, key financial reconciliation processes and best in class analytics and Data Science. You'll enable our data driven approach, and: Support the building of robust data models downstream of backend services (mostly in BigQuery) that ...

Hear from the team | Check out our Career Launchpad What you'll be working on: The Analytics Engineering team is responsible for building downstream data models from backend services with the desire to make our Data Warehouse a genuine competitive advantage for Monzo. We want a discipline capable … building an amazing Data Warehouse to support decision making, Business Intelligence, key financial reconciliation processes and best in class analytics and Data Science. You'll enable our data driven approach, and: Support the building of robust data models downstream of backend services (mostly in BigQuery) that ...

World-Changing Company at Lifebit At Lifebit, we're driven by the mission to make complex data analysis simpler and more accessible. We empower key institutions worldwide to revolutionize their data utilization and technology integration. Lifebit is powering the world s federated data platforms for population-scale … life sciences. Our Federated AI Platform is trusted across national health systems, global pharma, and research biobanks to unlock sensitive biomedical and real-world data securely, compliantly, and at scale. As we scale through our Series B growth phase, we are expanding our product organization to meet the demands ...

goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as a Data Science AI Trainer, you'll have the opportunity to collaborate on these projects. Although every project is unique, you might typically: Design original computational data … problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks). Develop problems requiring non-trivial reasoning chains in data processing, statistical analysis, feature engineering, predictive modeling, and insight extraction. Create deterministic problems with reproducible answers: avoid stochastic elements or require fixed random seeds ...

goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as a Data Science AI Trainer, you ll have the opportunity to collaborate on these projects. Although every project is unique, you might typically: Design original computational data … problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks). Develop problems requiring non-trivial reasoning chains in data processing, statistical analysis, feature engineering, predictive modeling, and insight extraction. Create deterministic problems with reproducible answers: avoid stochastic elements or require fixed random seeds ...

goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as a Data Science AI Trainer, you'll have the opportunity to collaborate on these projects. Although every project is unique, you might typically: Design original computational data … problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks). Develop problems requiring non-trivial reasoning chains in data processing, statistical analysis, feature engineering, predictive modeling, and insight extraction. Create deterministic problems with reproducible answers: avoid stochastic elements or require fixed random seeds ...

dedicated Data Science team is at the forefront of revolutionizing pharma intelligence and how patients gain access to life-saving therapies. Armed with cutting-edge technology and a passion for innovation, we leverage the vast landscape of data to extract actionable insights that drive informed decision making. … unique collaborative approach fosters a dynamic synergy between data science and product development. Our deep expertise in machine learning, artificial intelligence, advanced statistical modelling, and big data, combined with our domain knowledge, enables us to deliver comprehensive solutions that empower our clients to stay ahead ...

Join a leading independent technology and services provider as an OpenSource Data Platform Solution Engineer. Job Overview: Looking for a hands-on Solution Engineer/Architect to design and support large-scale cybersecurity data platforms using open-source technologies. Location: Sheffield Rate: A daily rate, in-scope IR35 … Strategy and Governance, Cyber Sciences and Analytics, Cybersecurity Key Responsibilities: Design end-to-end technical solutions for cybersecurity applications Work on large-scale (petabyte) data platforms Create architecture documents (HLDs/LLDs) Evaluate and standardise open-source technologies Collaborate with engineering and governance teams Ensure solutions meet security, compliance ...

Aker Systems was founded in 2017 by a team of experienced technology professionals who recognised an opportunity to provide highly secure enterprise data platforms to large organisations. We build and operate ground-breaking, ultra-secure, high performance, cloud-based data infrastructure for the enterprise. Our proprietary technology solutions … drive performance and reduce costs while helping our clients to improve the management and sharing of data across their organisations. In 2024, Aker Systems won the Breakthrough Culture Awards highlighting growth companies putting culture first. In 2020 Aker Systems was recognised as a One to Watch on the Sunday ...

Kubernetes, OpenShift, MicroK8s; • Infrastructure as Code (IaC), automation & configuration management; • Ansible (plus Puppet, Saltstack), Terraform, CloudFormation; • NodeJS, REACT/MaterialUI (plus Angular), Python, JavaScript; • Big data processing and analysis, e.g. Apache Hadoop (CDH), Apache Spark; • RedHat Enterprise Linux, CentOS, Debian or Ubuntu. • Java 8, Spring framework (preferably Spring ...