6 of 6 Remote/Hybrid Apache Spark Jobs in the South West

associated Azure technologies. Build scalable ETL/ELT pipelines to ingest, transform, and load data from multiple enterprise systems. Develop data transformations using PySpark, Spark SQL, SQL, and Python. Create and maintain curated datasets that support business intelligence, reporting, and analytics. Implement data quality, reconciliation, validation, and monitoring processes. … engineering or business intelligence solutions. Strong proven commercial experience with Databricks. Strong experience developing ETL/ELT pipelines and data integration solutions. Experience using Apache Spark, PySpark, and Spark SQL. Strong SQL development and optimisation skills. Experience working with large and complex datasets from multiple source systems. ...

Feast, AWS Feature Store) Experience with containerisation (Docker) and orchestration (Kubernetes, ECS) Familiarity with Infrastructure as Code (Terraform, CloudFormation) Experience with data processing frameworks (Spark, Dask) for large-scale workloads Understanding of data governance and compliance frameworks Experience working in regulated industries (finance, healthcare, or similar) Security Clearance ...

large language model (LLM) systems, including retrieval augmented generation (RAG) and fine-tuning. Build scalable ETL/ELT pipelines using tools such as Spark or dbt, so data is high quality, time-stamped, and ready for both training and real-time inference. Design and deploy data collection systems ...

year apprenticeship where you will be put onto the Machine Learning standard with a pathway into Software You will virtually attend Cambridge Spark on a 1-day a week basis with check ins and sessions each quarter Permanent employment after apprenticeship in out DevOps Team Apprenticeship Standard Machine learning … engineer (level 6) Training Provider CAMBRIDGE SPARK LIMITED Working Week Monday to Friday, 9.00am to 5.00pm. Hybrid working role where you may be able to work from home for 2 days a week. Expected Duration 4 Years Positions Available 1 Closing Date Thursday, 31st December 2026 Start Date Tuesday ...

team capability growth and development. What We’re Looking For Experience as a Data Engineer with strong skills in areas such as Python, NumPy, Spark and Apache Pig. Proven experience in designing, developing and deploying scalable systems. Familiarity with CI/CD, containerisation, deployment technologies and cloud platforms ...

iterative delivery—making pragmatic trade‐offs and documenting them clearly. Technology Stack Languages: Predominantly Python and Node.js Distributed/data platforms: HDFS, HBase, Spark, plus increasing use of Kubernetes and cloud services Storage/search: MongoDB, OpenSearch Orchestration: Airflow, Dagster, NiFi Tooling: GitHub, GitHub Actions, Rundeck, Jira, Confluence Deployment … plus) Experience debugging moderately complex systems and improving reliability/performance Strong fundamentals: data structures, testing, version control, Linux basics Nice to have: Spark/PySpark experience Hadoop ecosystem exposure (HDFS/HBase) Workflow orchestration (Airflow/Dagster/NiFi) Search/indexing (OpenSearch, MongoDB) Kubernetes and infrastructure ...