101 to 125 of 138 Apache Spark Jobs in the UK excluding London

large language model (LLM) systems, including retrieval augmented generation (RAG) and fine-tuning. Build scalable ETL/ELT pipelines using tools such as Spark or dbt, so data is high quality, time-stamped, and ready for both training and real-time inference. Design and deploy data collection systems ...

large language model (LLM) systems, including retrieval-augmented generation (RAG) and fine-tuning. Build scalable ETL/ELT pipelines using tools such as Spark or dbt, ensuring data is high quality, time-stamped and ready for both training and real-time inference. Design and deploy data collection systems ...

pipelines to collect, transform and deliver datasets for analytical decision making • Work with cloud platforms, big data technologies, and modern programming languages (Python, SQL, Spark etc) to solve complex data challenges • Support data modelling, database design and integrated data sources • Apply data quality, governance and security controls in line ...

demonstrable experience of working in rapidly changing client environments. Needs to be a practitioner and proponent of Agile and Dev Ops Proficiency in Hadoop, Spark, Scala, Python, or a programming language associated with data engineering Expertise in building and deploying production level data solutions maintained by application support teams ...

. Experience with Natural Language Processing (NLP), Large Language Models (LLMs) or Image Processing. Experience in areas of model development, data processing and streaming (Spark, Kafka), microservices in python (Flask or FastAPI), and interactive visualisations and User Interfaces (Streamlit, Plotly, Gradio etc). While we think the above experience ...

management and international regulatory compliance. YOUR SKILLS AND EXPERIENCE ESSENTIAL CRITERIA Strong experience designing, building and maintaining scalable data pipelines and integration solutions using Apache Airflow (or similar orchestration platforms), Python, SQL and bash/linux. Experience developing cloud-native data platforms and solutions on AWS, including working with … platforms such as Amazon Bedrock. Experience developing data-based apps in Streamlit/Shiny Skills in ‘Big Data’ technology tools like Hadoop/Spark Knowledge of Infrastructure as Code tools like CloudFormation/Terraform If you can bring some of these skills and experiences, along with transferable strengths ...

Senior Data Engineer – SQL/Python/AWS Adria Solutions is recruiting a Senior Data Engineer for a growing business developing a new digital services platform. With established backing, the business is building its data ...

. Experience with Natural Language Processing (NLP) and Large Language Models (LLMs). Experience in areas of model development, data processing and streaming (Spark, Kafka), microservices in python (Flask or FastAPI), and interactive visualisations and User Interfaces (Streamlit, Plotly, Gradio etc). These lists are not exhaustive ...

Experience supporting the Intelligence Community or Department of Defense environments Experience with Python and working with databases Experience with big data frameworks such as Spark, PySpark, or Elastic Search Experience with data visualization platforms such as Kibana, Foundry Workshop, or Palantir Knowledge of secure coding practices and development ...

essential: Background in financial services or regulated environments. Exposure to financial crime domains such as fraud, AML or sanctions. Experience with Spark or PySpark. Familiarity with AWS cloud services and data platforms. You will be assessed on key capabilities relevant to the role, including risk and controls, change ...

Envitia is seeking a Senior Data Engineer to design and deliver cutting-edge data solutions for customers in the UK. Using Databricks, Apache Spark and cloud-native technologies, you’ll build scalable Lakehouse architectures and production-grade pipelines that enable organisations to trust and share data. ...

tenant isolation, row- and column-level security, and PII handling. Define the streaming and batch ingestion architecture, building on our investments in NATS JetStream, Apache Flink, and Debezium CDC patterns, and direct the engineering team on execution. Specify the catalog, lineage, and data-quality foundation and own the operational … streaming vs batch). Hands-on experience with: Dremio, Snowflake, or Databricks AWS data services (S3, Glue, Athena, Lake Formation, or equivalent) NATS/Apache Kafka or similar, and CDC patterns (Debezium or equivalent) Stream processing (Apache Flink, Spark Structured Streaming, or equivalent) SQL and query optimisation ...

engineering practices to multi-disciplinary teams. Desirable Skills, Qualifications & Experience: Experience leveraging Kubernetes for application deployment, and familiarity with distributed computing frameworks (e.g., Ray, Spark), or specialised batch schedulers/resource managers (e.g., Slurm, Volcano, Kueue). Demonstratedexpertisewith specialised serving engines (e.g.,vLLM,SGLang) or techniques for deploying models ...

mindset and openness to learning from others Nice to have: Hands on orchestration skills and experience using Terraform Experience with big data tools (e.g. Spark, AWS Glue, or similar) Strong grasp of code review best practices Cloud Migration/AWS knowledge Understanding/experience working with orchestration and data ...

Chester (Hybrid). You will own data lifecycle, building scalable pipelines and ensuring governance in a regulated financial environment. You'll work with Python, Spark, Kafka, NiFi, Hadoop, Hive, and modern data-platform technologies, delivering production-grade controls and observability. #J-18808-Ljbffr ...

lead embedded data science teams, partnering effectively with senior stakeholders across multiple domains Hands‐on technical expertise with tools such as Databricks, Python, Spark, and GCP/BigQuery Engineering mindset, with experience moving teams toward machine learning engineering best practice Credibility to lead long‐tenured individual contributors while providing ...

building and deploying machine learning models in production, alongside strong hands‐on Python skills. You should be comfortable with distributed computing tools such as Spark or PySpark, cloud platforms and modern MLOps practices. Experience managing and developing data scientists is essential, as is the ability to explain technical decisions ...

building and deploying machine learning models in production, alongside strong hands-on Python skills. You should be comfortable with distributed computing tools such as Spark or PySpark, cloud platforms and modern MLOps practices. Experience managing and developing data scientists is essential, as is the ability to explain technical decisions ...

leadership teams. Minimum Qualifications Masters in Statistics, Mathematics, Data Science, Machine Learning, Physics, Engineering, Computer Science or equivalent Proficiency in data querying using SQL, Spark, or equivalent technologies Proficiency in scripting languages for data processing and analysis (ex: Python, R, or Scala) Excellent communication and multi-functional collaboration skills ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming). Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming). Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

seeking a Secure Data Engineer in Newcastle. This role focuses on building scalable and secure data platforms for Defence clients using Azure, Databricks, and Apache Spark. The successful candidate will engage in developing robust data applications, ensuring high security and compliance standards. Opportunities for professional growth and a flexible ...

Hadoop ecosystem, collaborating with data scientists to operationalize ML models using CML. The role spans data ingestion, transformation, and full-stack tooling with Java, Spark, Python, Flask, and React, emphasizing performance tuning and efficient resource use across batch and streaming workloads. #J-18808-Ljbffr ...

year apprenticeship where you will be put onto the Machine Learning standard with a pathway into Software You will virtually attend Cambridge Spark on a 1-day a week basis with check ins and sessions each quarter Permanent employment after apprenticeship in out DevOps Team Apprenticeship Standard Machine learning … engineer (level 6) Training Provider CAMBRIDGE SPARK LIMITED Working Week Monday to Friday, 9.00am to 5.00pm. Hybrid working role where you may be able to work from home for 2 days a week. Expected Duration 4 Years Positions Available 1 Closing Date Thursday, 31st December 2026 Start Date Tuesday ...

include: Contributing to disaster recovery, business continuity and platform resilience solutions for streaming environments. Working with streaming technologies such as Kafka Streams, Flink or Spark Streaming and applying real‐time data processing approaches to business requirements. Working collaboratively across engineering, architecture, security and infrastructure teams, sharing knowledge and contributing ...