2 of 2 Remote/Hybrid Apache Spark Jobs in Scotland

code. LLMOps & AI Model Management: Familiarity with tools like MLFlow, LangChain, Hugging Face, Kubeflow, or similar platforms. Data Processing: Working knowledge of Databricks/Spark or comparable large-scale data processing tools. SQL: Solid capabilities in data querying and preparation. Data Architectures: Understanding of modern data infrastructure (lakehouses, data ...

deliver end-to-end data pipelines on cloud platforms (Google Cloud Platform preferred). Building scalable data workflows using distributed technologies such as Spark, Flink, Storm, or similar Develop robust processing and migration pipelines, including support for legacy DataStage decommissioning and modernisation. Work with a variety of database technologies … Java, Python or Scala. Experience with cloud data services (GCP preferred; Azure/AWS acceptable). Experience with distributed data processing frameworks such as Spark (Core/SQL/Streaming), Flink, or Storm. Understanding of designing scalable data models for varied access patterns. Working on large scale big data ...