26 to 38 of 38 Apache Jobs in London

orchestrate workloads across CPU and GPU environments. Your work will include: Designing and building distributed services in Rust Developing scalable work queue systems (e.g. Apache Pulsar, RabbitMQ) Building GPU node orchestration services for data movement Creating robust storage lifecycle and indexing services Designing performant APIs using gRPC What ...

optimise performance across distributed systems. Mandatory Skills & Experience Minimum 3+ years of experience in software development. Proficiency in LAMP stack: Excellent understanding of Linux, Apache, MySQL, and PHP, with hands-on experience in developing and maintaining applications using this stack Scripting and programming skills: Strong knowledge of Scripting languages ...

services including Vertex AI, Model Garden, BigQuery, Cloud Run and Google’s ADK. • Experience with document and data processing using tools such as Apache Tika, plus storage or caching technologies like Postgres, Redis, GCS or BigQuery. • Solid DevOps practices including CI/CD, Docker, observability and automated testing. ...

schema). Version Control Practical experience with Git (branching, merging, pull requests). Preferred Qualifications (A Plus) Experience with a distributed computing framework like Apache Spark (using PySpark). Familiarity with cloud data services ( AWS S3/Redshift, Azure Data Lake/Synapse, or Google BigQuery/Cloud Storage … . Exposure to workflow orchestration tools ( Apache Airflow, Prefect, or Dagster ). Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field. ...

architecture to support data consumption for reporting systems, operational applications, and analytics platforms. Implement automation, orchestration, and scalable pipeline frameworks using tools such as Apache Airflow. Collaborate with business stakeholders and technical teams to align the data platform with enterprise data strategy. Support hospitality analytics use cases, including guest … predictive analytics, recommendation engines, and customer behaviour analytics. Ability to design AI-ready data platforms for future analytics use cases. Preferred Skills Experience with Apache Airflow for pipeline orchestration. Knowledge of CI/CD pipelines, DevOps, and Git-based development workflows. Experience with data governance, metadata management, and enterprise ...

models are always running, always accurate, and always improving. Key Responsibilities ML Pipeline Orchestration & Automation Own and extend Circadia’s ML pipeline orchestration using Apache Airflow, including training, evaluation, and deployment workflows. Build and maintain automated pipelines for model retraining, validation, and promotion across development, staging, and production environments. … closely related infrastructure role. Strong proficiency in Python for ML pipeline development, tooling, and automation. Hands-on experience with ML pipeline orchestration tools, particularly Apache Airflow. Experience with model registries and experiment tracking platforms (MLflow preferred). Experience deploying and operating ML workloads on AWS (Batch ...

acceptable). Experience with NoSQL databases. Experience with vector databases used in modern AI systems. Strong programming experience in Python. Frameworks/Infrastructure:- Apache Spark Apache Airflow Kafka Elasticsearch/OpenSearch ...

with NoSQL databases. Experience with vector databases used in modern AI systems. Strong programming experience in Python Experience within the following technologies is desirable - Apache Spark, Apache Airflow, Kafka, Elasticsearch/Open Search, PostgreSQL, MongoDB, Pandas, Polars About the Package £100k - £110k on the base ...

Understanding of embedding dimensionality, fine-tuning, domain adaptation, and cross-encoder re-ranking. ● Production experience with chunking and ingestion pipelines — document parsing (unstructured.io, LlamaParse, Apache Tika), semantic chunking, contextual chunking (prepending document-level context to chunks), parent-child relationships, and metadata enrichment. ● Experience with hybrid search — combining dense vector … reasoning over charts, tables, images, and structured data alongside text. ● Familiarity with GraphRAG (Microsoft) or similar community-based graph retrieval approaches. ● Experience with Apache Kafka or streaming pipelines for near-real-time document ingestion and index updates. ● Knowledge of semantic caching strategies for reducing retrieval latency and LLM costs. ...

capabilities across the organization. Key Responsibilities Design and implement data architectures on the Databricks Lakehouse Platform Build and optimize ETL/ELT pipelines using Apache Spark and Databricks Implement Delta Lake and medallion architecture (Bronze, Silver, Gold) Ensure data governance, security, and performance optimization Integrate Databricks with cloud platforms … stakeholders to deliver scalable data solutions Required Skills & Qualifications Databricks Certification (e.g., Databricks Certified Data Engineer Associate/Professional) Strong experience with Databricks, Apache Spark, and Delta Lake Proficiency in Python, SQL, and data pipeline development Experience with cloud platforms (Azure, AWS, or GCP) Solid understanding of modern data ...

estate (AWS + Airflow + Databricks). Key Responsibilities Design, build and maintain Infrastructure as Code using CloudFormation and AWS services Manage and scale Apache Airflow (MWAA) environments Develop and maintain CI/CD pipelines (Git + Jenkins/GitHub Actions) Build serverless components with Lambda and other … call rotation and incident resolution Essential Skills & Experience 5+ years as a DevOps/Platform Engineer Strong AWS experience + CloudFormation Production experience with Apache Airflow (MWAA) Lambda , Linux + Bash scripting CI/CD expertise (Git, Jenkins, GitHub Actions) IaC and GitOps practices Nice to Have Terraform Databricks ...

Production Hands-On Experience with Deep Learning Frameworks (TensorFlow, PyTorch, XGBoost) Deep Knowledge of Data Warehousing, ETL/ELT Architectures & Heavy Analytic Environments (Apache Beam, Hadoop, Spark, Hive) Responsibilities: Leading Customer-Facing AI Deployments across Europe, from the Initial Consulting Phase right through to the Live Implementation of Production … Systems Architecting & Optimising Scalable Data Pipelines, ETL/ELT Solutions & Distributed ML Environments (using Hadoop, Spark & Apache Beam). Deploying Recommendation Engines & Deep Learning Models (TensorFlow, PyTorch) to solve specific Client Problems. Running Technical Design Sessions, Tailoring ML Ops Architectures using Vertex AI & Gen AI. Writing Automation-Focused Code ...

data sets can be effectively combined and analysed. They can also utilise programming languages such as Python, Java or SQL and tools such as Apache Spark or in this instance LakeFS, for data processing and management. Useful - R, Apache Spark. Very useful - Python, SQL, JSON, LakeFS. Reasonable Adjustments ...