7 of 7 Permanent Apache Spark Jobs in Manchester

Charity Volunteer Days Cycle to work scheme And More.. DATA ENGINEER – ESSTENTIAL SKILLS Proven experience building data pipelines using Databricks . Strong understanding of Apache Spark (PySpark or Scala) and Structured Streaming . Experience working with Kafka (MSK) and handling real-time data . Good knowledge of Delta ...

Strong SQL skills with a solid understanding of NoSQL technologies Hands-on experience with Azure data services, including Synapse and Fabric Experience working with Spark and/or Databricks Strong experience designing and implementing ETL pipelines Experience using Terraform or similar Infrastructure as Code tools Strong problem-solving ability ...

private/on-prem AI platforms (e.g. NVIDIA-based stacks, private AI platforms). • Solid understanding of data engineering concepts (ETL, data lakes, Spark or similar). • Experience designing scalable, distributed systems. • Familiarity with model lifecycle management and MLOps practices. • Proven ability to align AI initiatives to business outcomes ...

containerised workloads (Docker, Kubernetes, Kubeflow, etc.) and integrating with CI/CD tools (Jenkins, GitHub Actions, GitLab CI). Familiarity with distributed computing frameworks (Spark, Ray, TensorFlow Distributed, PyTorch Distributed). Strong understanding of monitoring, logging, and observability for large-scale ML systems. Experience in cost optimisation for compute ...

implementing enterprise data catalogues (eg Collibra, Informatica, Atlan) Knowledge of modern data architectures (Medallion, Lambda, Data Mesh) Experience with Real Time data platforms (Kafka, Spark) and BI tools (Power BI, Tableau) Confident communicator, able to influence senior stakeholders and chair governance forums Experience with complex operational systems is advantageous ...

will be able to demonstrate skills and experience in many of the following: - Expertise in designing production-grade data pipelines using Python, Scala, Spark, and SQL - Deep knowledge of AWS Cloud Platforms (EMR, Glue, Redshift, Kinesis, Lambda, DynamoDB) - Experience with data processing across structured and unstructured sources - Strong scripting ...

Deep expertise in search and recommendation systems (e.g., semantic embeddings, learning-to-rank, personalisation algorithms). Hands-on experience with modern ML toolchains — Python, Spark, and frameworks such as PyTorch or TensorFlow. Strong grounding in experimental design, A/B testing, and the use of offline/online metrics ...