3 of 3 Remote/Hybrid Apache Spark Jobs in Renfrewshire

Python, Java or Scala, and SQL. Hands‐on experience building and operating large‐scale data pipelines using big data and distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud‐native data ...

millions of developers, data practitioners, and AI agents worldwide from high-level data visualization plugins (Tableau, PowerBI, Superset, Metabase) to connectors for data frameworks (Apache Spark, Flink, Kafka Connect, Fivetran), orchestration platforms, and AI tooling. Our work directly shapes how companies process massive datasets: real-time analytics platforms … should build and why. You'll own the full lifecycle of key Python integrations, driving architecture, performance, and feature direction across: Orchestration Platforms: Apache Airflow, Dagster, Prefect Transformation Tools: dbt, SQLMesh AI & LLM Ecosystem: LangChain, LlamaIndex, n8n, and broader AI tooling: embedding pipelines, retrieval-augmented generation with ClickHouse ...

Financial Services, multi-market experience Data Integration & System Engineers Build scalable data pipelines supporting decisioning systems. Key Skills: GCP (BigQuery) & Hadoop/Cloudera (mandatory) Spark, Scala, Python, SQL Data pipelines (batch & near real-time) Data modelling, governance & CI/CD Pega CDH integration experience QA/Test Specialists Ensure ...