19 of 19 Permanent Apache Spark Jobs in the Thames Valley

will initially be for 6 months. It is also fully remote. Key Responsibilities • Design, develop, and maintain batch and streaming data pipelines using Databricks (Apache Spark) • Build and optimize ETL/ELT workflows for large-scale structured and unstructured data • Implement Delta Lake architectures (Bronze/Silver/… Gold layers) • Integrate data from multiple sources (databases, APIs, event streams, files) • Optimize Spark jobs for performance, scalability, and cost • Manage data quality, validation, and monitoring • Collaborate with analytics and ML teams to support reporting and model development • Implement CI/CD, version control, and automated testing for data ...

schema). Version Control Practical experience with Git (branching, merging, pull requests). Preferred Qualifications (A Plus) Experience with a distributed computing framework like Apache Spark (using PySpark). Familiarity with cloud data services (AWS S3/Redshift, Azure Data Lake/Synapse, or Google BigQuery/Cloud … Storage). Exposure to workflow orchestration tools (Apache Airflow, Prefect, or Dagster). Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field. ...

commodities markets or broader financial markets. Knowledge of quantitative modeling, risk management, or algorithmic trading. Familiarity with big data technologies like Kafka, Hadoop, Spark, or similar. Why Work With Us? Impactful Work: Directly influence the profitability of the business by building technology that drives trading decisions. Innovative Culture ...

TypeScript, JavaScript Machine Learning: TensorFlow, PyTorch, Scikit-learn GenAI/LLMs: OpenAI API, Hugging Face Transformers, LangChain, Langraph, RAG pipelines Data & Analytics: Pandas, NumPy, Spark, Databricks, Hadoop, Vector Databases, Snowflake Model Deployment: MLflow, FastAPI, Flask, Docker, Kubernetes Cloud & Infrastructure: AWS, Azure, GCP CI/CD pipelines (GitLab, Jenkins) Desirable ...

optimise Cypher queries for performance and scalability • Integrate Neo4j Aura with cloud-native applications (REST, GraphQL, event-driven systems) • Manage data ingestion pipelines (Kafka, Spark, ETL tools, APIs) • Monitor, tune, and troubleshoot Neo4j Aura performance and availability • Implement security, access control, and data governance best practices • Collaborate with product ...

leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas: Extensive enterprise experience with Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL ...

Desired Skills (Bonus Points) Experience with MLOps tooling (MLflow, Kubeflow, Weights & Biases, or similar) Building data pipelines (real-time or batch) using tools like Apache Spark, Kafka, Airflow, or dbt Familiarity with recommender systems, transformers, or graph neural networks Exposure to model optimisation techniques (quantisation, distillation, efficient inference ...

Data Analytics Sales Executive - Data Sales, Talend, Snowflake, Apache Spark, Kafka, Dataops, AWS, Azure, Data visualisation, SQL, ETL, Pre-Sales, Account Manager, SaaS Sales, Cloud Solutions, Digital Transformation, Business Development, Sales Consultant, CXO Engagement, Strategic Accounts, Deal Negotiation, Remote Working, UK-wide, Permanent Role, Sales Targets, New Business ...

models, transformers, retrieval models). Hands-on experience with LLMs and GenAI technologies. Strong programming and problem-solving skills with proficiency in Python, SQL, Spark, and Hive. Deep understanding of classical and modern ML techniques, A/B testing methodologies, and experiment design. Solid background in ranking, recommendation ...

extracting, manipulating, and preparing data from relational databases. Experience with NoSQL databases is a plus. Big Data/Cloud: Experience with big data tools (Spark, Hadoop) and cloud computing platforms (AWS, Azure, or GCP) for scalable ML workflows. Visualization: Ability to create clear, compelling data visualizations using tools like ...

/CD, Git-based workflows, and automation Strong problem-solving skills and a collaborative mindset Bonus Terraform or other IaC tools Slurm, Kueue, Ray, Spark, or similar systems GPU tooling (CUDA, Nvidia operators, schedulers) Experience supporting ML training or data science teams ...

containerised workloads (Docker, Kubernetes, Kubeflow, etc.) and integrating with CI/CD tools (Jenkins, GitHub Actions, GitLab CI). Familiarity with distributed computing frameworks (Spark, Ray, TensorFlow Distributed, PyTorch Distributed). Strong understanding of monitoring, logging, and observability for large-scale ML systems. Experience in cost optimisation for compute ...

/multi-region patterns, meeting defined RPO/RTO targets. Hands-On Engineering Write and review production-grade code using: Python, Scala, SQL Spark/AWS Glue AWS Lambda & Step Functions Build infrastructure using Terraform (IaC). Implement CI/CD pipelines (GitLab, Jenkins). Enforce automated testing, performance ...

DevOps, with the freedom to shape how GenAI and ML are used across the platform. Tech stack: Python PyTorch or TensorFlow scikit-learn Spark Kafka Databricks LangChain and RAG-based GenAI systems Cloud-native Azure environment Important Details: Fully remote across ...

Deep expertise in search and recommendation systems (e.g., semantic embeddings, learning-to-rank, personalisation algorithms). Hands-on experience with modern ML toolchains — Python, Spark, and frameworks such as PyTorch or TensorFlow. Strong grounding in experimental design, A/B testing, and the use of offline/online metrics ...

architecture, and network stack optimization. Data Infrastructure: Familiarity with the modern data stack, including time-series databases, object stores, and streaming frameworks like Apache Flink or Spark Streaming. Engineering Rigor: You write clean, testable, and reliable code. You understand that in this environment, a system failure can cost ...

kernels Experience with RLHF pipelines (PPO, DPO, ORPO) Experience training or deploying multimodal or diffusion models Experience in large-scale data processing (Apache Arrow, Spark, Ray) Prior work in a research lab (Google Brain, DeepMind, FAIR, Anthropic, OpenAI ...

kernels Experience with RLHF pipelines (PPO, DPO, ORPO) Experience training or deploying multimodal or diffusion models Experience in large-scale data processing (Apache Arrow, Spark, Ray) Prior work in a research lab (Google Brain, DeepMind, FAIR, Anthropic, OpenAI ...

kernels Experience with RLHF pipelines (PPO, DPO, ORPO) Experience training or deploying multimodal or diffusion models Experience in large-scale data processing (Apache Arrow, Spark, Ray) Prior work in a research lab (Google Brain, DeepMind, FAIR, Anthropic, OpenAI ...