101 to 125 of 472 Big Data Jobs in the UK

Data Engineer AWS Pyspark SQL Redshift Lambda London Remote **About the Role** The Data Engineer will play a pivotal role in organization by designing and implementing robust data pipelines that facilitate efficient data flow and management across various platforms. This position is essential for ensuring the integrity, reliability, and accessibility of our data, which supports … critical business decisions and drives insights. **Required Skills** - **Proficiency in PySpark and AWS:** You should have a strong command of both PySpark for data processing and AWS (Amazon Web Services) for cloud-based solutions. - **ETL Pipeline Development:** Demonstrated experience in designing, implementing, and debugging ETL (Extract, Transform, Load) pipelines is crucial. You will be responsible for moving and transforming … data from various sources to data warehouses. - **Programming Expertise:** A solid understanding of Python, PySpark, and SQL is required to manipulate and analyze data efficiently. - **Knowledge of Spark and Airflow:** In-depth knowledge of Apache Spark for big data processing and Apache Airflow for orchestrating complex workflows is essential for managing data pipelines. - **Cloud More ❯

View our cookie policy .Principle Data Architect page is loaded Principle Data Architectlocations: Edinburgh: Not Office Based (UK)time type: Full timeposted on: Posted Todaytime left to apply: End Date: November 11, 2025 (13 days left to apply)job requisition id: R Job Description At Aberdeen, our ambition is to be the UK's leading Wealth & Investments group.Strengthening … of the market as we design digital experiences that supercharge advice firms to delight their clients. About the role We are looking for an exceptionally talented and knowledgeable Principal Data Architect to deliver best practice data architecture for the Adviser business, and to help integrate data with our clients. They will own the technical design and delivery … of data and software architecture, whilst embracing the latest technology. This opportunity is perfect for someone who wants a senior "hands on" technical role with senior stakeholder interaction.This role will play a critical part in driving data-driven decision-making and innovation within the company. It requires someone who can blend a mix of consultancy with advanced technical More ❯

Data Engineer – Cardiff/Hybrid – £40,000 - £50,000 + benefits Yolk Recruitment are excited to be working with a global technology business that’s continuing to expand its data capability and invest in modern cloud solutions. Known for their collaborative culture and commitment to innovation, they’re offering an excellent opportunity for a Data Engineer to … make a real impact. We’re looking for a Data Engineer to help design, build, and maintain scalable data pipelines and systems that power analytics and business intelligence across the organisation. You’ll play a key role in ensuring data is accurate, accessible, and high-quality — driving data-led decision making at every level. What you … ll be doing: Design, build, and maintain scalable data pipelines and ETL processes to support analytics and operations. Develop and optimise data models and storage solutions for performance and reliability. Ensure data quality, integrity, and security throughout the data lifecycle. Collaborate with data scientists, analysts, and engineers to deliver effective data solutions. Implement and More ❯

Data Engineer - Cardiff/Hybrid - £45,000 - £50,000 + benefits Yolk Recruitment are excited to be working with a global technology business that's continuing to expand its data capability and invest in modern cloud solutions. Known for their collaborative culture and commitment to innovation, they're offering an excellent opportunity for a Data Engineer to … make a real impact. We're looking for a Data Engineer to help design, build, and maintain scalable data pipelines and systems that power analytics and business intelligence across the organisation. You'll play a key role in ensuring data is accurate, accessible, and high-quality - driving data-led decision making at every level. What you … ll be doing: Design, build, and maintain scalable data pipelines and ETL processes to support analytics and operations. Develop and optimise data models and storage solutions for performance and reliability. Ensure data quality, integrity, and security throughout the data lifecycle. Collaborate with data scientists, analysts, and engineers to deliver effective data solutions. Implement and More ❯

Senior Solution Architect (Big Data) – Outside IR35 I’m supporting a major UK government programme looking for an experienced Senior Solution Architect with a strong big data and cloud background. You’ll provide the technical leadership needed to ensure data solutions are secure, scalable and aligned with strategic goals - influencing design, governance and delivery across … a complex portfolio of digital services. Key Responsibilities: • Define and own the technical vision and roadmap for data-driven solutions across multiple workstreams. • Design, review, and assure data architectures ensuring they meet performance, governance and security standards. • Act as the technical bridge between engineering teams and business stakeholders, translating complex requirements into clear architectural direction. • Provide mentorship and … guidance to technical teams, promoting best practice and consistency across projects. • Oversee data management and modelling standards, working closely with engineers to enable efficient automation and integration. Details: Outside IR35 £600–£625 per day SC Cleared Remote , with occasional travel if required 6-month initial contract (start within 2 weeks) If you’re an experienced architect who enjoys shaping More ❯

We are seeking a skilled Subject Matter Expert (SME) & Data Engineer with a strong focus on data modelling, mapping, and transformation to join our Big Data Services team within the Hybrid Cloud & Data service line. This role will involve collaborating with cross-functional teams to define, implement, and validate data integration and transformation requirements … across a range of structured and semi-structured data sources. Key Responsibilities Analyse source and target data formats and structures (e.g., databases, XML, JSON, CSV, EDI) to define and document accurate and effective data mappings. Develop and maintain data mapping specifications, transformation logic, and integration rules based on business and technical requirements. Work closely with data architects, domain experts, and business analysts to ensure semantic alignment and consistency between systems. Validate mapping outputs and troubleshoot discrepancies or issues in data transformations. Create and maintain documentation to support data traceability, governance, and reusability. Ensure all data processing activities comply with relevant data quality, security, and privacy standards. Required Skills & Experience UK National More ❯

The Role The Data Practice is the team made up of experts dedicated to Data centric offers (Data platform, Data Analytics, IoT, Big Data, Data compliance) who support our customers in their IT and Data transformations. We help customers choose the right tools and services, build highly scalable secure environments, and help More ❯

The Role The Data Practice is the team made up of experts dedicated to Data centric offers (Data platform, Data Analytics, IoT, Big Data, Data compliance) who support our customers in their IT and Data transformations. We help customers choose the right tools and services, build highly scalable secure environments, and help More ❯

The Role The Data Practice is the team made up of experts dedicated to Data centric offers (Data platform, Data Analytics, IoT, Big Data, Data compliance) who support our customers in their IT and Data transformations. We help customers choose the right tools and services, build highly scalable secure environments, and help More ❯

Senior Data Engineer - Trino/Starburst 🚀 Senior Data Engineer Opportunity Salary: Up to £75k (Based on candidate experience) + Excellent benefits Work commute Model: Mostly remote with just 1 day per week on-site Location: London/Portsmouth Job Type: Permanent Ready to Shape the Future of Data? My client is a leading UK-based IT services … and consulting company seeking an experienced Senior Data Engineer who thrives on building robust, scalable systems that unlock the true potential of enterprise data. This is your opportunity to join their cutting-edge AI & Data team as a Senior Data Engineer . What You'll Be Doing 🔧 Build & Optimise : Take ownership of Starburst (Trino) and Dell Data Lakehouse deployments, turning complex data ecosystems into seamless, high-performance platforms 🔗 Connect Everything : Create unified data experiences by integrating diverse data sources across multiple environments ⚡ Performance Mastery : Fine-tune containerised solutions for maximum scalability and lightning-fast query performance 🛡️ Secure by Design : Implement bulletproof security frameworks and ensure rock-solid compliance across all data touchpoints More ❯

Senior Data Engineer - Trino/Starburst 🚀 Senior Data Engineer Opportunity Salary: Up to £75k (Based on candidate experience) + Excellent benefits Work commute Model: Mostly remote with just 1 day per week on-site Location: London/Portsmouth Job Type: Permanent Ready to Shape the Future of Data? My client is a leading UK-based IT services … and consulting company seeking an experienced Senior Data Engineer who thrives on building robust, scalable systems that unlock the true potential of enterprise data. This is your opportunity to join their cutting-edge AI & Data team as a Senior Data Engineer . What You'll Be Doing 🔧 Build & Optimise : Take ownership of Starburst (Trino) and Dell Data Lakehouse deployments, turning complex data ecosystems into seamless, high-performance platforms 🔗 Connect Everything : Create unified data experiences by integrating diverse data sources across multiple environments ⚡ Performance Mastery : Fine-tune containerised solutions for maximum scalability and lightning-fast query performance 🛡️ Secure by Design : Implement bulletproof security frameworks and ensure rock-solid compliance across all data touchpoints More ❯

of something bigger. MAIN PURPOSE OF THE ROLE & RESPONSIBILITIES IN THE BUSINESS:As a Senior Palantir Engineer (Senior Associate) at Kainos, you will be responsible or designing and developing data processing and data persistence software components for solutions which handle data at scale. Working in agile teams, Senior Data Engineers provide strong development leadership and take … responsibility for significant technical components of data systems. You will work within a multi-skilled agile team to design and develop large scale data processing software to meet user needs in demanding production environments.YOUR RESPONSIBILITIES WILL INCLUDE:• Working to develop data processing software primarily for deployment in Big Data technologies. The role encompasses the full … implications of user stories and user journeys.• Coaching and mentoring team members.MINIMUM (ESSENTIAL) REQUIREMENTS:• Strong software development experience in one of Java, Scala, or Python• Software development experience with data-processing platforms from vendors such as AWS, Azure, GCP, Databricks.• Experience of developing substantial components for large-scale data processing solutions and deploying into a production environment• Proficient More ❯

Qualifications Bachelor's or Master's degree in Computer Science, Mathematics or similar field (PhDs will be given preference) 7+ years of experience working with massive data Strong knowledge of public cloud (AWS) Experience with data acquisition (API calls/FTP downloads), ETL, transformation/normalization Proficiency in ETL process is mandatory Experienced building components for enterprise central … data platforms (data warehouses, Operational Data Stores, Access layers with APIs, file extracts, user queries) Hands-on experience with SQL, Python, Spark, Kafka Excellent communication skills, including a good working proficiency in verbal and written English About this Job Scopeworker Business Intelligence real time processes enterprise customer data. We develop and maintain robust and reliable data pipelines to support 24x7 business operations and enable systems, tools, dashboards and data analysts to surface business insights and opportunities. This actionable business intelligence is provided in real time to our Fortune 100 users from data engineers, to analysts and C-Level stakeholders. As a Data Architect, you'll innovate, design, build and automate, scalable solutions More ❯

achieve enduring results. Working with clients in consumer and manufacturing, defence and security, energy and utilities, financial services, government and public services, health and life sciences, and transport. The Data Engineer will have experience in AWS cloud technologies for ETL pipeline, data warehouse and data lake design/building and data movement. You will join the … following technologies to ensure scalable, high-performance applications: AWS is a significant growth area for the business with a diverse and growing capability, and we are looking for a Data Engineer with experience in AWS cloud technologies for ETL pipeline, data warehouse and data lake design/building and data movement. AWS data and analytics … Essential requirements You thrive in problem-solving and analytical thinking You enjoy collaborating with multiple stakeholders in a fast-paced environment Experience in the design and deployment of production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala, Spark, SQL. Experience performing tasks such as writing scripts, extracting data using More ❯

Data Engineer - Professional Services - £75,000 - Leeds (hybrid) A leading global professional services firm is seeking a Design Data Engineer to join its growing data and analytics team. This role offers the opportunity to play a pivotal part in shaping the firm’s data infrastructure and driving the intelligent use of data across global operations. … The successful candidate will design, develop, and optimise data pipelines, models, and systems that enable smarter insights, improved efficiency, and enhanced decision-making across the business. Key Responsibilities Design, develop, and maintain scalable and efficient data pipelines, integrations, and ETL processes to support analytics and reporting. Architect and optimize data models and database structures, ensuring scalability, consistency … and business alignment. Collaborate with data scientists, analysts, and business stakeholders to translate data requirements into actionable technical solutions. Ensure data quality, integrity, and governance through validation, monitoring, and documentation processes. Implement and oversee data security and compliance measures, protecting sensitive business and client information. Produce clear technical documentation for data architectures, workflows, and processes. More ❯

Data Engineer - Professional Services - £75,000 - Leeds (hybrid) A leading global professional services firm is seeking a Design Data Engineer to join its growing data and analytics team. This role offers the opportunity to play a pivotal part in shaping the firm’s data infrastructure and driving the intelligent use of data across global operations. … The successful candidate will design, develop, and optimise data pipelines, models, and systems that enable smarter insights, improved efficiency, and enhanced decision-making across the business. Key Responsibilities Design, develop, and maintain scalable and efficient data pipelines, integrations, and ETL processes to support analytics and reporting. Architect and optimize data models and database structures, ensuring scalability, consistency … and business alignment. Collaborate with data scientists, analysts, and business stakeholders to translate data requirements into actionable technical solutions. Ensure data quality, integrity, and governance through validation, monitoring, and documentation processes. Implement and oversee data security and compliance measures, protecting sensitive business and client information. Produce clear technical documentation for data architectures, workflows, and processes. More ❯

About Synechron: Synechron is a leading digital transformation consulting firm, delivering innovative solutions across Financial Services, Insurance, and more. We empower organizations to harness the power of data, technology, and strategy to drive growth and efficiency. Role Overview: We are seeking a skilled Data Engineer to join our UK team. In this role, you will be responsible for … designing, developing, and maintaining scalable ETL data pipelines leveraging Python and DataBricks. You will collaborate with cross-functional teams to understand data requirements and ensure the delivery of reliable, high-quality data solutions that support our business objectives. Key Responsibilities: Collaborate with stakeholders and cross-functional teams to gather data requirements and translate them into efficient … ETL processes. Develop, test, and deploy end-to-end ETL pipelines extracting data from various sources, transforming it to meet business needs, and loading it into target systems. Take ownership of the full engineering lifecycle, including data extraction, cleansing, transformation, and loading, maintaining data accuracy and consistency. Manage and monitor data pipelines, implementing proper error handling More ❯

About Synechron: Synechron is a leading digital transformation consulting firm, delivering innovative solutions across Financial Services, Insurance, and more. We empower organizations to harness the power of data, technology, and strategy to drive growth and efficiency. Role Overview: We are seeking a skilled Data Engineer to join our UK team. In this role, you will be responsible for … designing, developing, and maintaining scalable ETL data pipelines leveraging Python and DataBricks. You will collaborate with cross-functional teams to understand data requirements and ensure the delivery of reliable, high-quality data solutions that support our business objectives. Key Responsibilities: Collaborate with stakeholders and cross-functional teams to gather data requirements and translate them into efficient … ETL processes. Develop, test, and deploy end-to-end ETL pipelines extracting data from various sources, transforming it to meet business needs, and loading it into target systems. Take ownership of the full engineering lifecycle, including data extraction, cleansing, transformation, and loading, maintaining data accuracy and consistency. Manage and monitor data pipelines, implementing proper error handling More ❯

About Synechron: Synechron is a leading digital transformation consulting firm, delivering innovative solutions across Financial Services, Insurance, and more. We empower organizations to harness the power of data, technology, and strategy to drive growth and efficiency. Role Overview: We are seeking a skilled Data Engineer to join our UK team. In this role, you will be responsible for … designing, developing, and maintaining scalable ETL data pipelines leveraging Python and DataBricks. You will collaborate with cross-functional teams to understand data requirements and ensure the delivery of reliable, high-quality data solutions that support our business objectives. Key Responsibilities: Collaborate with stakeholders and cross-functional teams to gather data requirements and translate them into efficient … ETL processes. Develop, test, and deploy end-to-end ETL pipelines extracting data from various sources, transforming it to meet business needs, and loading it into target systems. Take ownership of the full engineering lifecycle, including data extraction, cleansing, transformation, and loading, maintaining data accuracy and consistency. Manage and monitor data pipelines, implementing proper error handling More ❯

Lead Data Scientist – Azure Data Stack – Very Attractive Base + Bonus – Manchester City Centre A rapidly growing software house in the heart of Manchester is seeking a Senior Data Scientist to spearhead innovation within their Data & Analytics function. If you're a seasoned data science professional looking for a role where you can have significant … impact, mentor a developing team, and contribute to cutting-edge analytics solutions, this could be the perfect opportunity. Role Overview As a Senior Data Scientist, you'll be a pivotal member of a cross-functional team, leading the enhancement and development of state-of-the-art features for the platform. You'll be responsible for guiding the entire modelling … lifecycle, from data preparation and analysis to the development of robust machine learning models and the creation of impactful, actionable visualisations. You'll also have the opportunity to shape the team's technical direction and mentor junior colleagues. Key Responsibilities Architecting and overseeing data preparation, cleansing, and structuring processes for advanced model development. Leading investigations into patterns, trends More ❯

Lead Data Scientist - Azure Data Stack - Very Attractive Base + Bonus - Manchester City Centre A rapidly growing software house in the heart of Manchester is seeking a Senior Data Scientist to spearhead innovation within their Data & Analytics function. If you're a seasoned data science professional looking for a role where you can have significant … impact, mentor a developing team, and contribute to cutting-edge analytics solutions, this could be the perfect opportunity. Role Overview As a Senior Data Scientist, you'll be a pivotal member of a cross-functional team, leading the enhancement and development of state-of-the-art features for the platform. You'll be responsible for guiding the entire modelling … lifecycle, from data preparation and analysis to the development of robust machine learning models and the creation of impactful, actionable visualisations. You'll also have the opportunity to shape the team's technical direction and mentor junior colleagues. Key Responsibilities Architecting and overseeing data preparation, cleansing, and structuring processes for advanced model development. Leading investigations into patterns, trends More ❯

Lead Data Scientist – Azure Data Stack – Very Attractive Base + Bonus – Manchester City Centre A rapidly growing software house in the heart of Manchester is seeking a Senior Data Scientist to spearhead innovation within their Data & Analytics function. If you're a seasoned data science professional looking for a role where you can have significant … impact, mentor a developing team, and contribute to cutting-edge analytics solutions, this could be the perfect opportunity. Role Overview As a Senior Data Scientist, you'll be a pivotal member of a cross-functional team, leading the enhancement and development of state-of-the-art features for the platform. You'll be responsible for guiding the entire modelling … lifecycle, from data preparation and analysis to the development of robust machine learning models and the creation of impactful, actionable visualisations. You'll also have the opportunity to shape the team's technical direction and mentor junior colleagues. Key Responsibilities Architecting and overseeing data preparation, cleansing, and structuring processes for advanced model development. Leading investigations into patterns, trends More ❯

Lead Data Scientist – Azure Data Stack – Very Attractive Base + Bonus – Manchester City Centre A rapidly growing software house in the heart of Manchester is seeking a Senior Data Scientist to spearhead innovation within their Data & Analytics function. If you're a seasoned data science professional looking for a role where you can have significant … impact, mentor a developing team, and contribute to cutting-edge analytics solutions, this could be the perfect opportunity. Role Overview As a Senior Data Scientist, you'll be a pivotal member of a cross-functional team, leading the enhancement and development of state-of-the-art features for the platform. You'll be responsible for guiding the entire modelling … lifecycle, from data preparation and analysis to the development of robust machine learning models and the creation of impactful, actionable visualisations. You'll also have the opportunity to shape the team's technical direction and mentor junior colleagues. Key Responsibilities Architecting and overseeing data preparation, cleansing, and structuring processes for advanced model development. Leading investigations into patterns, trends More ❯

Data QA Engineer – Remote-first - £55-65,000 Overview: As a Data QA Engineer, you will ensure the reliability, accuracy and performance of our client’s data solutions. Operating remotely, you will work closely with Data Engineers, Architects and Analysts to design and execute automated testing strategies that validate data pipelines, integrations and transformations across … cloud platforms. Your work will underpin data quality and ensure that insights are built on trusted, consistent data. Key Responsibilities: Design and implement automated tests for data pipelines, ETL processes and APIs across AWS and Azure environments. Develop and maintain test frameworks that validate data quality, completeness and integrity. Collaborate with engineering teams to identify data defects, root causes and performance issues. Support continuous integration and deployment by embedding automated testing into CI/CD pipelines. Validate source-to-target data mapping, transformations and business logic. Produce clear test documentation and communicate test results to technical and non-technical stakeholders. Monitor data test coverage and define quality metrics for ongoing improvement. Contribute to More ❯

Senior Data Scientist - AI/ML (Financial Services) Location: Remote/Major Hubs in APAC, Europe, and the UK Team: Advanced Analytics & Artificial Intelligence About the Role We are seeking a highly experienced and innovative Senior Data Scientist to join our global Advanced Analytics team. This is a strategic role focused on designing, developing, and deploying cutting-edge … growth, optimizing risk management, enhancing customer experiences, and ensuring regulatory compliance on a global scale. Key Responsibilities End-to-End Model Development: Lead the entire ML lifecyclefrom problem framing, data exploration, and feature engineering to model training, validation, deployment, and monitoring in production environments. Advanced Algorithm Design: Develop and implement sophisticated machine learning algorithms (e.g., Gradient Boosting, NLP, Deep … detection, credit scoring, algorithmic trading, customer segmentation, and sentiment analysis. Stakeholder Management: Collaborate closely with business leaders (e.g., Risk, Marketing, Trading, Operations) to translate complex business problems into actionable data science projects with clear KPIs and ROI. Technical Leadership: Mentor junior data scientists, establish best practices for code (e.g., Git), model versioning (e.g., MLflow), and reproducible research. Conduct More ❯