4 of 4 Apache Spark Jobs in South Yorkshire

millions of developers, data practitioners, and AI agents worldwide from high-level data visualization plugins (Tableau, PowerBI, Superset, Metabase) to connectors for data frameworks (Apache Spark, Flink, Kafka Connect, Fivetran), orchestration platforms, and AI tooling. Our work directly shapes how companies process massive datasets: real-time analytics platforms … should build and why. You'll own the full lifecycle of key Python integrations, driving architecture, performance, and feature direction across: Orchestration Platforms: Apache Airflow, Dagster, Prefect Transformation Tools: dbt, SQLMesh AI & LLM Ecosystem: LangChain, LlamaIndex, n8n, and broader AI tooling: embedding pipelines, retrieval-augmented generation with ClickHouse ...

Financial Services, multi-market experience Data Integration & System Engineers Build scalable data pipelines supporting decisioning systems. Key Skills: GCP (BigQuery) & Hadoop/Cloudera (mandatory) Spark, Scala, Python, SQL Data pipelines (batch & near real-time) Data modelling, governance & CI/CD Pega CDH integration experience QA/Test Specialists Ensure ...

collection systems using Scala and Java. Apply sound engineering principles such as test-driven development and modular design. Preferred Background Hands-on experience with Spark and Scala in commercial environments. Familiarity with Java and Python. Exposure to distributed data systems and cloud storage platforms. Experience designing data schemas ...

best practices, DevOps, CI/CD, IaC You have excellent communication skills What's in it for you: As a Data Software Engineer (Python Spark AWS) you will earn a competitive package: Up to £95,000 salary Remote working including flexible working hours Workplace nursery scheme Enhanced maternity package … range of other perks including Perkbox, cycle to work, season ticket loan Apply now to find out more about this Data Software Engineer (Python Spark AWS) opportunity. Ref: PR/100084/A/KS/ ...