1 to 25 of 108 Big Data Jobs in the UK excluding London

Job Type: 6+ month and possibility of extension Looking for an enthusiastic Spark Scala Engineer who will be responsible for designing, building, and maintaining data pipelines using Apache Spark and Scala. This includes tasks like: Extracting data from various sources (databases, APIs, files) Transforming and cleaning the data Loading the data into data warehouses or data lakes (eg, BigQuery, Amazon Redshift) Automating the data pipeline execution using scheduling tools (eg, Airflow) Data analysis and modelling: While the primary focus might be on data engineering, some JDs might require basic data analysis skills: Writing analytical queries using SQL or Spark SQL to analyse processed data and Building simple data models to understand data relationship. Work with Big Data technologies: You'll likely work with various Big Data technologies alongside Spark, including: Hadoop Distributed more »

Data Scientist/Engineer - Hybrid/Bristol Job Title : Data Scientist/Engineer Location : Hybrid/Bristol Remuneration : £50,000 - £60,000 per annum Contract Type : Permanent Role will be based with the modelling team. 25% Data Submission and Cleansing (Data Engineer), 75% Process Improvement (Data Science). Ideally, seeking someone with strong Data Science exposure but Data Engineering skills. Responsibilities : Play a pivotal role in crafting, enhancing, and managing intricate data ingestion pipelines, ensuring their efficiency, scalability, and dependability. Integrate advanced machine learning elements into data pipelines to elevate data processing and analytical capabilities. Devise and implement data quality metrics and dashboards for monitoring, ensuring the fidelity and precision of ingested data. Collaborate closely with data scientists, engineers, and cross-functional team members in an agile development setting to meet project objectives and deadlines. Harness modern NLP more »

Job Title: Data Engineer Location: London Job Type: Permanent Work Type: Hybrid Salary : £50,000 to £55,000 per annum Are you a sports enthusiast with a passion for working in the sports industry? Whether you're interested in the dynamic world of professional sports, the business of sports … of possibilities for those eager to immerse themselves in this thrilling and fast-paced environment. Databuzz is presently seeking an experienced candidate for a Data Engineer role with one of our clients located in London. About the Client: Our client is a sports tech company that leverages AI and … expert knowledge to offer data and analysis, addressing performance and commercial challenges in the sports and media industries. About the Role: As a Data Engineer you will be pivotal in designing, developing, and maintaining data architecture and infrastructure. The ideal candidate should have a strong foundation in more »

Senior Data Engineer Mobysoft is one of the fastest growing SaaS providers in the UK and has been shortlisted in the "Top 50 fastest growing technology companies in the North" for four successive years. Mobysoft provides predictive analytical software that has helped our customers deliver tens of millions of … year at the Real I.T Awards 2024- one of the many awards we are proud of ! Role overview We are an ambitious, customer-centric Data & Insights team, dedicated to developing a new generation of data products that unlock significant value for the social housing sector. We operate with … a focused product lens, driven by curiosity and a commitment to technical excellence. Data Engineering is foundational to this mission, ensuring synchronized, curated, trusted, dynamic data is available at speed and scale for our clients, data analysts,data scientists, and stakeholders. As such we are looking more »

an experienced AWS Cloud Architect to design AWS reference architecture for our services-led client. Responsibilities: Design and implement scalable, secure, and high-performance data architectures on AWS. Develop data models and database designs that align with business requirements and ensure data integrity. Proficiency in CI/… CI, or AWS CodePipeline. Hands-on experience with Infrastructure as Code (IaC) tools like AWS CloudFormation, Terraform, or similar. Create and maintain ETL processes, data pipelines, and data integration workflows using AWS services. Implement data storage solutions, including data lakes, data warehouses, and operational databases. … Provide technical leadership and guidance on AWS data services and architecture to the team. Requirements: Proven experience as an AWS Data Architect or similar role, with a strong focus on data architecture and management. In-depth knowledge of AWS data services, including but not limited to more »

This Global Pharmaceutical Services provider are looking for a Data Engineer to work within their growing Business Intelligence technical service function. You will be supporting the technical development of their cloud data platforms, offering the options for near real time process reporting solutions. Client Details Global Pharmaceutical Services … provider Description This Global Pharmaceutical Services provider are looking for a Data Engineer to work within their growing Business Intelligence technical service function. You will be supporting the technical development of their cloud data platforms, offering the options for near real time process reporting solutions. You will work … on designing and developing their company data acquisition strategy allowing for near real time process reporting and full support of contractual reporting to customers. Key Responsibilities: Monitor and maintain ETL processes to ensure the continuation of an accurate data reporting platform for the business. Explore solutions for optimising more »

benefits Please note: you need to have the right to work in the UK – no sponsorship, unfortunately About the job Reporting to the Senior Data & Analytics Lead, the BI Developer will be responsible for driving the direction and maintaining the business data model ensuring it aligns with business … activities and processes. The role will work closely with key internal and external stakeholders in organising and creating a data model that makes data and information accessible with appropriate channels of access controls. Fostering a relationship of trust and playing an advisory role in the use of data to improve business performance. The role will require playing a highly collaborative role by overseeing cross-functional data governance while ensuring the adoption and adherence to data quality are understood by stakeholders. Key Responsibilities Responsible for designing effective, efficient data models to support various types of more »

We are currently seeking a talented Senior Full Stack Developer/Data Engineer to join the Human Genomics Team at the EMBL European Bioinformatics Institute (EMBL-EBI). The team provides genome variation and regulation resources as part of the Ensembl project, human variation interpretation services including the DECIPHER … platform, and data portals for partners around the world. With underlying infrastructure for these services based on EMBLs large and long-standing data resources this is an opportunity to rise to the challenges of big data and make contributions that advance the reach and utility of … e.g. Perturb-seq) data. This project is funded through the Open Targets consortium, an innovative public-private partnership that uses human genetics and genomics data for systematic drug target identification and prioritisation. The new resource will help to address a fundamental need in the scientific and drug discovery community more »

we do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers. You will have a proven track record as a Data Engineer and be adept in your ability to implement data … ingestion and transformation pipelines for large scale organisations. This career may align to individuals who like to work with a variety of technologies and data services, that will develop and deliver early proofs of concept and production implementation. On a day to day basis you'll: Design and implement more »

we do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers. You will have a proven track record as a Data Engineer and be adept in your ability to implement data … ingestion and transformation pipelines for large scale organisations. This career may align to individuals who like to work with a variety of technologies and data services, that will develop and deliver early proofs of concept and production implementation. On a day to day basis you'll: Design and implement more »

we do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers. You will have a proven track record as a Data Engineer and be adept in your ability to implement data … ingestion and transformation pipelines for large scale organisations. This career may align to individuals who like to work with a variety of technologies and data services, that will develop and deliver early proofs of concept and production implementation. On a day to day basis you'll: Design and implement more »

we do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers. You will have a proven track record as a Data Engineer and be adept in your ability to implement data … ingestion and transformation pipelines for large scale organisations. This career may align to individuals who like to work with a variety of technologies and data services, that will develop and deliver early proofs of concept and production implementation. On a day to day basis you'll: Design and implement more »

we do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers. You will have a proven track record as a Data Engineer and be adept in your ability to implement data … ingestion and transformation pipelines for large scale organisations. This career may align to individuals who like to work with a variety of technologies and data services, that will develop and deliver early proofs of concept and production implementation. On a day to day basis you'll: Design and implement more »

of our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of … roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and … debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for data more »

of our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of … roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and … debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for data more »

of our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of … roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and … debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for data more »

of our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of … roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and … debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for data more »

About the AWS Data Engineerj ob Stride Recruitment are looking for skilled and motivated AWS Data Engineers to join a consulting firm offering end-to-end data solutions, delivering best-in-class experiences for their external clients. To be successful in this AWS Data Engineer role … you will need to have a strong background in AWS, with experience in data warehousing, data lakes, and data integration. You should also have a good understanding of data modelling, ETL processes, and data governance. Technical Skills: Proficiency in AWS Glue, AWS Redshift, and Python … Experience with ETL processes, data integration, and data warehousing. Strong SQL skills Experience with Big Data technologies such as Hadoop, Spark, and Kafka Familiarity with cloud platforms (AWS, Azure, Google Cloud) Working knowledge of data visualisation tools (PowerBI, Tableau, Qlik Sense) Additional Skills: Client-facing more »

of our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of … roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and … debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for further analysis more »

of our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of … roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and … debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for further analysis more »

Data Analyst Up to £60k Permanent (beneficial to have a shorter notice period) Birmingham (3 days a week) HL58319 Finatal is currently partnered with a rapidly expanding manufacturing business that has recently had Private Equity Investment from a mid-market Fund. They are experiencing heavy transformation through M&A … and are now looking for an Operational Finance Analyst to join the Group to leverage their huge amount of data and translate this into clear and interesting visualisations to optimise business processes and decision-making across the organisation. You will work closely with the CEO and CFO establishing a … strong relationship with the senior leaders of the business, working with the finance team you will gain valuable financial data analytics experience. Role: Reporting to the Finance Operations Manager you will be the go-to person for collecting requirements and analysing key data for data-driven business more »

business. About the Role: As the Lead AI Consultant, you will report directly to the CTO and be responsible for driving AI adoption, developing 'big data' solutions, and defining API strategies. You will collaborate with senior product, technical, and operational managers to enhance the product suite with cutting … edge AI technologies. Key Responsibilities: Develop and execute the technical integration strategy. Create comprehensive 'big data' solutions across products. Define internal and external API strategies. Enhance existing products through AI technologies. Design, develop, and implement AI solutions. Integrate AI with client data, APIs, and production systems. Conduct … data analysis and build machine learning models. Utilize modern ML and AI libraries and frameworks. Deliver reliable solutions collaboratively. Desired Experience: Significant experience as an AI Consultant, Machine Learning Engineer, or similar role. Proven track record in start-up or dynamic environments. Hands-on experience with cloud-based AI more »

Foundry Specialist/Architect to join our team. The ideal candidate will possess extensive knowledge and hands-on experience with Palantir Foundry, including its data integration, analytics, and operational decision-making capabilities. As a Palantir Foundry Specialist/Architect, you will be responsible for designing, developing, and maintaining data-driven solutions that leverage Palantir Foundry's platform to address complex business challenges. Key Responsibilities: *Solution Architecture:* - Design and architect end-to-end data solutions using Palantir Foundry, ensuring alignment with business goals and technical requirements. - Develop scalable and efficient data models, pipelines, and workflows within the … Foundry environment. - Collaborate with stakeholders to understand business needs and translate them into technical requirements and solutions. *Data Integration:* - Implement data integration solutions to ingest, transform, and manage large datasets from various sources into Palantir Foundry. - Ensure data quality, consistency, and security throughout the data life more »

About the Role: I am looking for a talented Data Engineer to join our clients dynamic team on a 6-month contract basis. This is an exciting opportunity for a mid-level professional with 3-5 years of experience to make a significant impact on our data infrastructure … remote/hybrid working options and a competitive day rate of £250-£400, falling inside IR35 regulations. Key Responsibilities: Design, develop, and maintain scalable data pipelines and ETL processes using AWS, Databricks, Python, Spark, and SQL. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions. Optimize and troubleshoot data workflows to ensure efficiency and reliability. Implement data quality and validation checks to ensure data integrity. Contribute to the development and maintenance of our data lake and data warehouse solutions on AWS. more »

Data Product Solutions Lead Salary: Up to £68,000 Our Data Analytics Platform Design team is currently recruiting for a Data Product Solutions Lead to play a key role shaping and delivering a number of exciting data projects. This role has a key function driving how … we manage, prepare, and assure a wide selection of data assets to support a diverse group of business areas. As a Data Product Solutions lead, you will define solutions to support a range of development and transformation projects working with Business Data Product owners to define design … architects, and fellow designers. It’s an exciting time to be a part of our team and business! We are constantly modernising our complex data estate - cloud and data lake are our strategic targets, with business self-service and the concept of Mesh computing also in our sights. more »