13 of 13 Apache Spark Jobs in the East of England

keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark. Ensure high … data Billing & revenue assurance Build models and pipelines to support network analytics, customer 360, churn prediction, fraud detection , etc. 7. Performance Optimization & Monitoring Tune Spark/Glue jobs for performance and cost. Monitor Redshift/Athena/S3 efficiency and implement best practices. Perform data quality checks and validation ...

delivering large-scale data integration, engineering, and analytics projects Strong expertise in cloud platforms (e.g. Azure, MS Fabric, AWS) and modern data tools (e.g. Apache Spark, Hadoop) Proficiency in programming languages (e.g. Python, SQL, Java) Extensive experience in stakeholder management and collaboration across health and local government ...

solutions.• Pipeline Development: Develop and configure meta-data driven data pipelines using data orchestration tools such as Azure Data factory and engineering tools like Apache Spark to ensure seamless data flow.• Monitoring and Failure Recovery: Implement monitoring procedures to detect failures or unusual data profiles and establish recovery ...

also have Experience developing or using AI applications Experience with NoSQL databases (e.g. MongoDB). Experience with big data processing frameworks (e.g. Apache Spark). Knowledge of Large Language Model (LLM) integration, Retrieval-Augmented Generation (RAG), or Agentic AI frameworks. Experience with high-performance computing (HPC) environments (e.g. ...

governance, and data architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data governance, data sharing across ...

governance, and data architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data governance, data sharing across ...

kubernetes) Experience with Key value stores Experience in git and basic Unix Commands You might also have (Desirable) Experience with large data processing technologies (Apache Spark) Apply now! Benefits and Contract Information Financial incentives: depending on circumstances, monthly family/marriage allowance of £278 monthly child allowance ...

data. You'll work with engineering to move models from idea to production. Tech Stack: Python, scikit-learn, PyTorch, TensorFlow/Keras, LangChain, Pydantic, Spark, Kafka, Databricks, MLflow, Azure, Docker, Kubernetes. Salary: Up to £100k Working Environment: Fully Remote in UK What you'll do Research, prototype, and deploy … summarisation, document processing) Build and tune models using scikit-learn, PyTorch, TensorFlow/Keras, and XGBoost Create data pipelines for batch and streaming with Spark and Kafka Use Databricks and MLflow for experiments, deployment, and monitoring What you'll bring Hands-on Python for data science and ML Practical ...

data. You'll work with engineering to move models from idea to production. Tech Stack: Python, scikit-learn, PyTorch, TensorFlow/Keras, LangChain, Pydantic, Spark, Kafka, Databricks, MLflow, Azure, Docker, Kubernetes. Salary: Up to £100k Working Environment: Fully Remote in UK What you'll do Research, prototype, and deploy … summarisation, document processing) Build and tune models using scikit-learn, PyTorch, TensorFlow/Keras, and XGBoost Create data pipelines for batch and streaming with Spark and Kafka Use Databricks and MLflow for experiments, deployment, and monitoring What you'll bring Hands-on Python for data science and ML Practical ...

data. You'll work with engineering to move models from idea to production. Tech Stack: Python, scikit-learn, PyTorch, TensorFlow/Keras, LangChain, Pydantic, Spark, Kafka, Databricks, MLflow, Azure, Docker, Kubernetes. Salary: Up to £100k Working Environment: Fully Remote in UK What you'll do Research, prototype, and deploy … summarisation, document processing) Build and tune models using scikit-learn, PyTorch, TensorFlow/Keras, and XGBoost Create data pipelines for batch and streaming with Spark and Kafka Use Databricks and MLflow for experiments, deployment, and monitoring What you'll bring Hands-on Python for data science and ML Practical ...

data. You'll work with engineering to move models from idea to production. Tech Stack: Python, scikit-learn, PyTorch, TensorFlow/Keras, LangChain, Pydantic, Spark, Kafka, Databricks, MLflow, Azure, Docker, Kubernetes. Salary: Up to £100k Working Environment: Fully Remote in UK What you'll do Research, prototype, and deploy … summarisation, document processing) Build and tune models using scikit-learn, PyTorch, TensorFlow/Keras, and XGBoost Create data pipelines for batch and streaming with Spark and Kafka Use Databricks and MLflow for experiments, deployment, and monitoring What you'll bring Hands-on Python for data science and ML Practical ...

data. You'll work with engineering to move models from idea to production. Tech Stack: Python, scikit-learn, PyTorch, TensorFlow/Keras, LangChain, Pydantic, Spark, Kafka, Databricks, MLflow, Azure, Docker, Kubernetes. Salary: Up to £100k Working Environment: Fully Remote in UK What you'll do Research, prototype, and deploy … summarisation, document processing) Build and tune models using scikit-learn, PyTorch, TensorFlow/Keras, and XGBoost Create data pipelines for batch and streaming with Spark and Kafka Use Databricks and MLflow for experiments, deployment, and monitoring What you'll bring Hands-on Python for data science and ML Practical ...

data. You'll work with engineering to move models from idea to production. Tech Stack: Python, scikit-learn, PyTorch, TensorFlow/Keras, LangChain, Pydantic, Spark, Kafka, Databricks, MLflow, Azure, Docker, Kubernetes. Salary: Up to £100k Working Environment: Fully Remote in UK What you'll do Research, prototype, and deploy … summarisation, document processing) Build and tune models using scikit-learn, PyTorch, TensorFlow/Keras, and XGBoost Create data pipelines for batch and streaming with Spark and Kafka Use Databricks and MLflow for experiments, deployment, and monitoring What you'll bring Hands-on Python for data science and ML Practical ...