76 to 100 of 106 Apache Spark Jobs in the UK

record of reducing fraud actors at scale, with strong type-1 and type-2 error estimates. Experience working with technologies like Athena/Trino, Spark, AML, CI/CD, and Tableau. ...

data science or statistical role. Expertise in advanced analytics techniques including machine learning, optimisation and segmentation. Strong proficiency in Python, SQL and Spark SQL. Experience working with large datasets and distributed computing tools such as Azure Synapse and Snowflake. Familiarity with data transformation tools (Databricks, Azure Data Factory ...

applying emerging ML techniques. Desirable: Experience with cloud platforms (AWS) and containerisation (such as Docker, Podman, Kubernetes). Exposure to big data technologies (Spark, Hadoop) and Apache tools. Knowledge of NLP, computer vision, and deep learning architectures. Familiarity with Agile and DevOps practices. STEM degree or equivalent experience ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

Spring, Spring Boot Experience working with DevOps methodologies and scalable CI/CD pipelines Microservices Architecture Exposure to Big Data technologies such as: Kafka, Spark, Hadoop etc Strong understanding of clean code principles Experience working with AWS Cloud Kotlin experience (preferred ...

with Git, Python, and Ansible Familiarity with containers (Docker, Kubernetes, LXC) Experience with monitoring tools and SQL databases Exposure to big data systems (e.g., Spark, HDFS, Airflow) Summary A great opportunity for someone who enjoys solving complex infrastructure challenges, automating systems, and working closely with engineering teams. Spectrum ...

team Exposure to sectors such as defence, public sector and consumer/retail Familiarity with modern data and AI tooling such as Python, Spark, Databricks, Azure cloud, and Generative AI platforms (experience with specific tools is helpful but not essential) A mix of technical, product and operational experience - comfortable ...

deploy scalable, secure data solutions using Azure Databricks, Data Factory and Data Lake Storage. Develop and optimise advanced data pipelines with Python, SQL, Spark/PySpark and Delta Lake. Champion strong data quality, governance and observability practices. Modernise legacy systems and support large-scale migrations into Azure. Provide technical … Minimum Criteria Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory. Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing. Proven experience in CI/CD practices using industry-standard tools (e.g., GitHub ...

decisions Technical depth 10+ years in data engineering, data warehousing, or a senior data infrastructure role Expert‐level SQL and data manipulation skills using Spark, PySpark, or equivalent big data frameworks, with proven experience processing high‐volume, high‐velocity datasets Deep expertise in data lake and data warehousing technologies … Airflow or similar orchestration frameworks, with solid grounding in software design principles Strong Python proficiency; familiarity with streaming technologies such as Kafka or Apache Flink is a strong plus Hands‐on experience translating complex business requirements into robust data sourcing and modeling solutions Exposure to data mesh or data ...

optimise scalable data pipelines (batch and real-time) Work primarily with Java , with exposure to Python and streaming frameworks such as Kafka, Flink, or Spark Develop event-driven architectures and streaming integrations Integrate data from multiple sources using AWS services (e.g. Kinesis, Lambda, Glue, MSK) Architecture & Data Management Contribute … experience in Data Engineering or large-scale data systems Strong programming skills in Java (preferred) or Python Hands-on experience with Kafka, Flink, or Spark (streaming focus is a strong advantage) Solid understanding of streaming concepts (event time, state, backpressure) Experience building ETL/ELT or real-time data ...

lead embedded data science teams, partnering effectively with senior stakeholders across multiple domains Hands-on technical expertise with tools such as Databricks, Python, Spark, and GCP/BigQuery Engineering mindset, with experience moving teams toward machine learning engineering best practice Credibility to lead long-tenured individual contributors while providing ...

lead embedded data science teams, partnering effectively with senior stakeholders across multiple domains Hands-on technical expertise with tools such as Databricks, Python, Spark, and GCP/BigQuery Engineering mindset, with experience moving teams toward machine learning engineering best practice Credibility to lead long-tenured individual contributors while providing ...

learning or LLM-based systems Exposure to enterprise data platforms or similar tools Frontend development experience (e.g., React) Familiarity with distributed computing frameworks (e.g., Spark) What’s on Offer Competitive compensation package, including equity Comprehensive benefits Hybrid working model Opportunity to build products and systems from the ground ...

Core programming for data analysis NumPy & Pandas – Data preparation and manipulation Matplotlib & Seaborn – Visualisation and exploratory analysis TensorFlow, SciPy, Scikit-learn – Machine learning fundamentals Spark – Scalable data processing Flask & Streamlit – Deploying APIs and interactive data applications Who This Program Is For Graduates or final-year students in STEM, computer ...

inside IR35 & is fully remote with occasional office meetings in Leeds. Must have SC clearance. Key Skills required: Python AWS Terraform Apache Spark Airflow Docker Gitlab Security Scanning Agents (eg Trivy, Trend Micro, Wiz etc) Jupyter Notebooks If the above sounds like you, please apply now for immediate ...

year; 1–3 years; 3–5 years; more than 5 years. Core technologies Akka (Actors, Streams, Persistence, Clustering, Projection, HTTP); ZIO; Play; Spark Libraries Circe; Cats; Cats Effect; Slick; Doobie; HTTP4S; Shapeless; Monix Databases RDBMS; MongoDB; Redis; Cassandra; Elastic Search Messaging Kafka; RabbitMQ Architecture and Cloud Microservices; DDD; Event ...

Transformation You will develop and improve UCLH's data warehouse environments. Being involved in the transition from using an R-based data pipeline to Spark jobs running on a data platform. You will model and transform data from our Epic electronic health record system, coordinating closely with stakeholders … clinical safety and data governance standards. Desirable Experience of using R or Python Experience with modern data engineering tools and frameworks (e.g. dbt, snowflake, spark, etc.) Experience in an NHS informatics role or other healthcare information environment Skills and Abilities Essential In depth specialist knowledge of information analysis ...

About Quantiphi: Quantiphi is an award-winning, AI-First digital engineering and consulting company focused on delivering high-impact Services and Solutions that help organizations solve what truly matters. We partner with enterprises to reimagine ...

maintainable code Participating in code reviews, pairing, and team delivery Tech Stack & Environment Backend: Scala, Java (Python desirable) Frontend: HTML, CSS, JavaScript Frameworks: Play, Spark Data: Postgres, MongoDB, Hudi Cloud: AWS (S3, ECS, Aurora, Lambda) DevOps: Jenkins, Airflow, Terraform Monitoring: Kibana, Grafana Ways of working: Agile (Scrum/Kanban … Agile, multidisciplinary teams Strong analytical and problem-solving skills Clear communicator and collaborative team player Desirable Python, Docker, Airflow Big data tools (e.g. Spark, EMR) Experience with data platforms or risk/fraud systems How You'll Work Deliver value iteratively and early Collaborate closely across disciplines Write simple ...

handling edge cases, improving data quality and solving complex data challenges. The environment is modern and cloud based, with exposure to technologies such as Spark, Snowflake and BigQuery across AWS, GCP and Azure. You will also have the opportunity to support and mentor other engineers through collaboration, code reviews … Looking For Strong Python and SQL expertise gained within large scale production environments Experience delivering complex data transformation projects end to end Experience with Spark, Snowflake, BigQuery or similar technologies Exposure to cloud environments such as AWS, GCP or Azure Comfortable working with large, messy and evolving datasets Experience ...

handling edge cases, improving data quality and solving complex data challenges. The environment is modern and cloud based, with exposure to technologies such as Spark, Snowflake and BigQuery across AWS, GCP and Azure. You will also have the opportunity to support and mentor other engineers through collaboration, code reviews … Looking For Strong Python and SQL expertise gained within large scale production environments Experience delivering complex data transformation projects end to end Experience with Spark, Snowflake, BigQuery or similar technologies Exposure to cloud environments such as AWS, GCP or Azure Comfortable working with large, messy and evolving datasets Experience ...

A leading organisation undergoing major investment in its data and engineering capabilities is looking for a Senior Automation QA Engineer to strengthen quality across complex, distributed systems. This role sits at the intersection of software ...

A leading organisation undergoing major investment in its data and engineering capabilities is looking for a Senior Automation QA Engineer to strengthen quality across complex, distributed systems. This role sits at the intersection of software ...

systems, and modern cloud architectures . What youll be doing Design and build scalable data pipelines (batch & streaming) Work with technologies like Kafka, Flink, Spark, AWS services Develop real-time, event-driven data solutions Contribute to data architecture and modelling best practices Implement CI/CD pipelines and Infrastructure … deliver clean datasets What were looking for Strong programming skills in Java (preferred) or Python Experience with streaming technologies (Kafka/Flink/Spark) Knowledge of distributed systems and large-scale data processing Experience with cloud platforms (AWS preferred) Familiarity with DevOps tools and containerisation (Docker/Kubernetes ...

Developing and maintaining scalable data pipelines using Python - Automating manual Excel/VBA workflows into reliable, production-grade systems - Working with large datasets using Spark and Azure (Databricks/Synapse) - Supporting data modelling and analytics use cases (Crossing over to more Data Science driven work) - Driving best practices including … code reviews - Collaborating with stakeholders to deliver actionable data solutions What they re looking for: - Strong Python and SQL skills - Experience or exposure to Spark, big data tools, or cloud platforms (ideally Azure) What s on offer: - The chance to join a business at an exciting stage of growth ...