1 to 25 of 275 Apache Jobs in London

services for personalisation· Developing Java/Spring Boot microservices with REST/gRPC APIs· Working with Kafka and event-driven architectures· Building pipelines using Apache Flink and Apache Beam· Integrating ML models and feature stores· Working with NoSQL databases and Redis caching· Using Apache Camel for integration … Familiarity with stream processing concepts· Knowledge of distributed systems· Strong communication and teamwork skills· Interest in data-driven personalisation systemsNice to have· Experience with Apache Flink and Apache Beam· Experience with Kafka· Familiarity with Apache Camel· Experience with NoSQL databases (Cassandra, DynamoDB) and Redis· Cloud experience ...

platform at scale, ideally in a consumer or marketplace environment. Deep understanding of distributed systems and modern data ecosystems — including extensive experience with Databricks, Apache Spark, Apache Kafka, Apache Flink, Apache Beam and DBT. Demonstrated success in designing, building, and operating data platforms at scale, encompassing ...

wider stakeholders. Technical Environment Experience with some of the following technologies would be beneficial: Python SQL Cloud platforms – AWS, Azure or GCP Databricks Apache Spark/PySpark Kafka/event-driven architectures Airflow dbt Git/GitHub CI/CD Infrastructure as Code Modern data platforms and distributed processing ...

using Python, PySpark, and dbtBuild and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observabilityDevelop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformationsModel and transform data for analytics using SQL and dbt to support business intelligence ...

analysis and returning the results to various business systems.We build on a vast variety of technological stacks, from building products on top of Apache Kafka, Apache Spark, and Apache Beam to integrating Neo4j into Google Dataflow Templates and Microsoft Fabric. We also build backend components … ideally have professional experience developing software systems, either in cloud or on-premises, involving either of the following;Distributed event streaming platforms, such as Apache Kafka or similar products,Data processing and analytics platforms, such as Apache Spark, Apache Beam, or their derivatives,Data warehouses ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

code reviews.Strong understanding of modern software engineering and DevOps practices.Experience working within Agile delivery methodologies.Deep expertise in the Databricks Data Intelligence Platform, including Apache Spark, PySpark, Delta Lake, Unity Catalog, Databricks SQL, Lakeflow, Mosaic AI ,AI/BI Genie and Agent Bricks. Strong experience designing and delivering production-grade ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. Understanding ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. Understanding ...

improvements.Experience with modern data architectures, including cloud-based and distributed systems.Strong proficiency in SQL and Python.Experience with data engineering and pipeline technologies such as Apache Airflow and Spark.Experience working with cloud platforms, particularly AWS and Azure.Knowledge of big data technologies and distributed data processing.Understanding of data analytics and visualisation.Awareness ...

React, Node) using Webpack, Typescript, and GitHub.Experience with Docker and/or Kubernetes and containerized deployments (AKS).Experience with workflow orchestration tools such as Apache Airflow or n8n.Experience of code quality, test, CI/CD and package managers such as Jest, Lint, Brigade, and Helm.Experience with Kafka and event ...

expected to be up to 10% but may increase over time as the business evolves.Desired QualificationsExperience working with distributed data processing frameworks, including Apache Spark.Exposure to advanced analytics or AI‐adjacent data use cases, including preparing data for machine learning or intelligent applications.Familiarity with additional Azure services such ...

increase over time as the business evolves.**Desired Qualifications*** Experience working with distributed data processing frameworks, including Apache Spark.* Exposure to advanced analytics or AI‐adjacent data use cases, including preparing data for machine learning or intelligent applications.* Familiarity with additional Azure services such as Azure Functions or Logic ...

expected to be up to 10% but may increase over time as the business evolves. Desired QualificationsExperience working with distributed data processing frameworks, including Apache Spark. Exposure to advanced analytics or AI‐adjacent data use cases, including preparing data for machine learning or intelligent applications. Familiarity with additional Azure ...

pipelines, deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. ...

build, and maintain scalable and robust data pipelines using SQL and Python, and leveraging technologies such as: Databricks, Snowflake, Azure Data Factory, AWS Glue, Apache Airflow and Pyspark. Lead the integration of complex data systems and ensure consistency and accuracy of data across multiple platforms. Implement continuous integration ...

value. Excellent analytical, problem-solving, and communication skills. Proven experience leading data modernisation initiatives across multiple sectors. Technical Expertise Proficiency with big data technologies: Apache Spark, Kafka, Confluent, Databricks, Unity Catalog. Strong knowledge of cloud ecosystems (AWS, Azure, GCP) and infrastructure-as-code (Terraform, CloudFormation). Hands-on expertise ...

Advanced experience operating Delta tables at scale, covering transaction semantics, schema evolution, partitioning, and compaction, alongside highly optimised analytical query execution using DuckDB or Apache Spark.• Production-grade relational database expertise, particularly PostgreSQL (query tuning, indexing, partitioning, replication, and schema design across OLTP and analytics).• Strong KDB experience ...

measurable value.• Excellent analytical, problem-solving, and communication skills.• Proven experience leading data modernisation initiatives across multiple sectors.Technical Expertise• Proficiency with big data technologies: Apache Spark, Kafka, Confluent, Databricks, Unity Catalog.• Strong knowledge of cloud ecosystems (AWS, Azure, GCP) and infrastructure-as-code (Terraform, CloudFormation).• Hands-on expertise ...

principles. Comfortable working within Agile delivery environments. Experience with CI/CD pipelines, ideally using Azure DevOps or similar. Nice to Have Experience with Apache Airflow or dbt. Exposure to BigQuery/GCP. Azure or other cloud platform certifications Knowledge of data governance and data management frameworks Ideal experience ...

principles. Comfortable working within Agile delivery environments. Experience with CI/CD pipelines, ideally using Azure DevOps or similar. Nice to Have Experience with Apache Airflow or dbt. Exposure to BigQuery/GCP. Azure or other cloud platform certifications Knowledge of data governance and data management frameworks Ideal experience ...

testing, and deployment.Deep understanding of Quantexa components such as:Entity Resolution (ER)Scoring FrameworkContextual Network GenerationUI ConfigurationStrong background in Big Data technologies, including:Scala, Apache Spark, Hadoop, ElasticSearchExperience with data modelling and design in the context of large-scale analytics platforms.Strong stakeholder management and communication skills – capable of interfacing ...

maintain scalable and robust data pipelines using SQL and Python, and leveraging any of the following: Databricks, Snowflake, Azure Data Factory, AWS Glue, Apache Airflow and Pyspark.Lead the integration of complex data systems and ensure consistency and accuracy of data across multiple platforms.Implement continuous integration and continuous deployment ...

design approaches.You understand where traditional patterns remain appropriate – and where modern approaches add measurable value.Data Engineering & Platform Understanding:Strong knowledge of distributed processing (e.g. Apache Spark).Familiarity with Hadoop ecosystem components.Working knowledge of SQL and NoSQL technologies.Understanding of ETL/ELT design trade-offs.Awareness of DevOps practices ...