26 to 47 of 47 Apache Spark Jobs in London

Docker and Kubernetes is a plus. Experience designing end to end ML data pipelines. Experience with database technologies such as PostgreSQL, MySQL, or MongoDB, Spark, Databricks and stream data processing such as Kafka, RedPanda is a plus. Direct experience in building and applying large-scale (100M+ users) machine learning ...

experience in software engineering, MLOps, and deploying machine learning models at scale.* Proficiency in SQL, Python, and familiarity with big data technologies (e.g., Kafka, Spark) and/or cloud platforms (e.g., GCP, AWS, or Azure).* Industry knowledge: Experience in gaming or digital entertainment is a strong plus. ...

experience in software engineering, MLOps, and deploying machine learning models at scale. Proficiency in SQL, Python, and familiarity with big data technologies (e.g., Kafka, Spark) and/or cloud platforms (e.g., GCP, AWS, or Azure). Industry knowledge: Experience in gaming or digital entertainment is a strong plus. ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

large-scale analytics platforms Expert knowledge of Quantexa, Entity Resolution (ER), Scoring Framework, Contextual Network Generation Expertise in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch Strong stakeholder management and communication skills - capable of interfacing with both business and technical audiences. The work is heavily weighted toward ...

record of reducing fraud actors at scale, with strong type-1 and type-2 error estimates. Experience working with technologies like Athena/Trino, Spark, AML, CI/CD, and Tableau. ...

data science or statistical role. Expertise in advanced analytics techniques including machine learning, optimisation and segmentation. Strong proficiency in Python, SQL and Spark SQL. Experience working with large datasets and distributed computing tools such as Azure Synapse and Snowflake. Familiarity with data transformation tools (Databricks, Azure Data Factory ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

Informatica or DataStage. Knowledge of Unix/Linux scripting. Preferred Qualifications Experience in Teradata migration projects. Knowledge of GCP is a plus. Experience with Apache Spark is a plus. Experience with DataStage or similar ETL tools. ...

Spring, Spring Boot Experience working with DevOps methodologies and scalable CI/CD pipelines Microservices Architecture Exposure to Big Data technologies such as: Kafka, Spark, Hadoop etc Strong understanding of clean code principles Experience working with AWS Cloud Kotlin experience (preferred ...

with Git, Python, and Ansible Familiarity with containers (Docker, Kubernetes, LXC) Experience with monitoring tools and SQL databases Exposure to big data systems (e.g., Spark, HDFS, Airflow) Summary A great opportunity for someone who enjoys solving complex infrastructure challenges, automating systems, and working closely with engineering teams. Spectrum ...

with Git, Python, and Ansible Familiarity with containers (Docker, Kubernetes, LXC) Experience with monitoring tools and SQL databases Exposure to big data systems (e.g., Spark, HDFS, Airflow) Summary A great opportunity for someone who enjoys solving complex infrastructure challenges, automating systems, and working closely with engineering teams. Spectrum ...

team Exposure to sectors such as defence, public sector and consumer/retail Familiarity with modern data and AI tooling such as Python, Spark, Databricks, Azure cloud, and Generative AI platforms (experience with specific tools is helpful but not essential) A mix of technical, product and operational experience - comfortable ...

facing products Experience in ML Ops and deploying machine learning mode at scale. Proficiency in Python, and familiarity with data processing technologies (e.g., Kafka, Spark) and/or cloud platforms (e.g., GCP, AWS, or Azure). Ability to work on a hybrid work basis requiring at least 3 days ...

execute. You have deep experience designing, building, and operating production data pipelines, including owning reliability, monitoring, and failure recovery for orchestration systems such as Apache Airflow (or equivalent). You are fluent in modern data and cloud ecosystems, with hands-on experience (or the ability to ramp quickly … technologies such as Iceberg, Spark, PostgreSQL, and cloud-native infrastructure on AWS (or equivalent). You write clean, readable, and testable Python code, applying sound abstractions, naming, and review discipline to build systems that scale beyond the first version. You bring strong analytical and problem-solving skills ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive ...

learning or LLM-based systems Exposure to enterprise data platforms or similar tools Frontend development experience (e.g., React) Familiarity with distributed computing frameworks (e.g., Spark) What’s on Offer Competitive compensation package, including equity Comprehensive benefits Hybrid working model Opportunity to build products and systems from the ground ...

year; 1–3 years; 3–5 years; more than 5 years. Core technologies Akka (Actors, Streams, Persistence, Clustering, Projection, HTTP); ZIO; Play; Spark Libraries Circe; Cats; Cats Effect; Slick; Doobie; HTTP4S; Shapeless; Monix Databases RDBMS; MongoDB; Redis; Cassandra; Elastic Search Messaging Kafka; RabbitMQ Architecture and Cloud Microservices; DDD; Event ...

Transformation You will develop and improve UCLH's data warehouse environments. Being involved in the transition from using an R-based data pipeline to Spark jobs running on a data platform. You will model and transform data from our Epic electronic health record system, coordinating closely with stakeholders … clinical safety and data governance standards. Desirable Experience of using R or Python Experience with modern data engineering tools and frameworks (e.g. dbt, snowflake, spark, etc.) Experience in an NHS informatics role or other healthcare information environment Skills and Abilities Essential In depth specialist knowledge of information analysis ...

maintainable code Participating in code reviews, pairing, and team delivery Tech Stack & Environment Backend: Scala, Java (Python desirable) Frontend: HTML, CSS, JavaScript Frameworks: Play, Spark Data: Postgres, MongoDB, Hudi Cloud: AWS (S3, ECS, Aurora, Lambda) DevOps: Jenkins, Airflow, Terraform Monitoring: Kibana, Grafana Ways of working: Agile (Scrum/Kanban … Agile, multidisciplinary teams Strong analytical and problem-solving skills Clear communicator and collaborative team player Desirable Python, Docker, Airflow Big data tools (e.g. Spark, EMR) Experience with data platforms or risk/fraud systems How You'll Work Deliver value iteratively and early Collaborate closely across disciplines Write simple ...

handling edge cases, improving data quality and solving complex data challenges. ·The environment is modern and cloud based, with exposure to technologies such as Spark, Snowflake and BigQuery across AWS, GCP and Azure. ·You will also have the opportunity to support and mentor other engineers through collaboration, code reviews … Looking For ·Strong Python and SQL expertise gained within large scale production environments ·Experience delivering complex data transformation projects end to end ·Experience with Spark, Snowflake, BigQuery or similar technologies ·Exposure to cloud environments such as AWS, GCP or Azure ·Comfortable working with large, messy and evolving datasets ·Experience ...

Senior Developer (Python, Gen AI, Vector DB, Spark) Role Overview We are seeking a highly skilled Senior Developer with expertise in Python, Generative AI, Vector Databases, and Spark. The role focuses on building scalable AI-driven applications and data processing solutions. Python Development Develop robust applications and APIs using … performance and scalability. Vector Database Work with vector databases like Pinecone, FAISS, Weaviate. Develop embedding pipelines and semantic search. Optimize indexing and retrieval performance. Spark & Data Processing Build data pipelines using Apache Spark. Handle large-scale data processing. Optimize performance and scalability. CI/CD & DevOps Implement ...