26 to 50 of 54 Remote Apache Spark Jobs

MLOps/LLMOps principles, including CI/CD for ML, model monitoring, and governance frameworks. Proficiency with data processing and storage technologies (eg SQL, Spark) Strong stakeholder management and communication skills, with proven ability to translate complex AI concepts for diverse audiences. SUSTAINABLE BUSINESS Passion and curiosity to tackle ...

teams to drive outcomes Core skills (indicative) Production pipeline design and deployment experience Strong engineering capability with Python, SQL, plus big data tooling (e.g., Spark, and Java/Scala where relevant AWS, Azure, GCP Working pattern Hybrid working, with the team on client site or in the office ...

translate business needs into technical roadmaps What were looking for Strong experience in data architecture & large-scale systems design Expertise across Python, SQL, Spark, Scala Deep knowledge of AI/ML frameworks and modern data platforms Experience with cloud-native architecture and DevOps (Kubernetes, Terraform, CI/CD) Proven ...

optimization. Strong experience with Time Series Forecasting, Sequence Modelling, Signal Processing or Quantitive Analysis Proficient in Python and SQL. Experience with frameworks like PyTorch, Spark, or Airflow is highly desirable. Familiarity with AWS (SageMaker, Lambda) and CI/CD automation. Strong understanding of MLOps, data versioning, and quality management ...

Platform Engineering Manager Salary: £90,000 - £100,000 + 20% Bonus Location : Central London (Hybrid – 3 days per week) Tech: Python | Java | SQL | Azure | Spark | Flink | Kubernetes | Terraform | CI/CD | Airflow We’re working with one of the UK’s leading enterprise businesses as it continues to invest ...

data engineering within distributed systems Proven ability to build and optimise data pipelines at scale Solid Python skills with exposure to tools such as Spark, PySpark, and Hadoop Strong understanding of data ingestion and batch processing methodologies Experience with text or unstructured data is advantageous but not essential ...

working Required Experience Extensive commercial experience with Java in production environments Strong experience designing and building Kafka‐based streaming platforms Hands‐on experience with Spark, Flink, Kafka Streams , or similar technologies Experience with Spring Boot and/or Python Deep experience working with AWS , including services such as Lambda ...

manage multiple priorities and respond calmly to issues. • Self-motivated with a collaborative approach. Preferred: • Familiarity with AWS data tooling such as S3, Athena, Spark or notebooks. • Experience supporting data products or client-facing datasets. • Understanding of release testing, UAT or production support processes. • Exposure to Jira/Agile ...

manage multiple priorities and respond calmly to issues. • Self-motivated with a collaborative approach. Preferred: • Familiarity with AWS data tooling such as S3, Athena, Spark or notebooks. • Experience supporting data products or client-facing datasets. • Understanding of release testing, UAT or production support processes. • Exposure to Jira/Agile ...

influence within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing … Engineer or Solutions Architect certification (desirable). Databricks Certified Data Engineer or Machine Learning Associate (desirable). Experience with streaming solutions (Kafka, Event Hubs, Spark Streaming) (desirable). Knowledge of machine learning and AI on Databricks (desirable). ...

Proven experience with modern data platforms such as Snowflake (AI Data Cloud) and cloud‐native services. Good understanding of open‐source table formats, specifically Apache Iceberg , to enable interoperability, schema evolution, and high‐performance analytics across multiple engines. Programming: Expert‐level proficiency in Java , Python and SQL . … Data & Orchestration: Hands‐on experience with Spark, Kafka, and orchestration tools like Apache Airflow, Dagster, or dbt. Data Modeling: Deep understanding of data warehousing and modern data lakehouse architecture. Leadership & Soft Skills Mentorship: Proven track record of upskilling junior and senior engineers. Communication: Ability to explain complex technical ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

data science or statistical role. Expertise in advanced analytics techniques including machine learning, optimisation and segmentation. Strong proficiency in Python, SQL and Spark SQL. Experience working with large datasets and distributed computing tools such as Azure Synapse and Snowflake. Familiarity with data transformation tools (Databricks, Azure Data Factory ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

Spring, Spring Boot Experience working with DevOps methodologies and scalable CI/CD pipelines Microservices Architecture Exposure to Big Data technologies such as: Kafka, Spark, Hadoop etc Strong understanding of clean code principles Experience working with AWS Cloud Kotlin experience (preferred ...

with Git, Python, and Ansible Familiarity with containers (Docker, Kubernetes, LXC) Experience with monitoring tools and SQL databases Exposure to big data systems (e.g., Spark, HDFS, Airflow) Summary A great opportunity for someone who enjoys solving complex infrastructure challenges, automating systems, and working closely with engineering teams. Spectrum ...

optimise scalable data pipelines (batch and real-time) Work primarily with Java , with exposure to Python and streaming frameworks such as Kafka, Flink, or Spark Develop event-driven architectures and streaming integrations Integrate data from multiple sources using AWS services (e.g. Kinesis, Lambda, Glue, MSK) Architecture & Data Management Contribute … experience in Data Engineering or large-scale data systems Strong programming skills in Java (preferred) or Python Hands-on experience with Kafka, Flink, or Spark (streaming focus is a strong advantage) Solid understanding of streaming concepts (event time, state, backpressure) Experience building ETL/ELT or real-time data ...

learning or LLM-based systems Exposure to enterprise data platforms or similar tools Frontend development experience (e.g., React) Familiarity with distributed computing frameworks (e.g., Spark) What’s on Offer Competitive compensation package, including equity Comprehensive benefits Hybrid working model Opportunity to build products and systems from the ground ...

across complex engagements. What You’ll Do Lead design and implementation of enterprise-scale data pipelines and lakehouse architectures Provide technical leadership across Databricks, Spark and cloud data platforms Drive delivery of data engineering workstreams within Agile programmes Engage with client stakeholders to translate business objectives into technical solutions … architecture documentation, design artefacts and delivery roadmaps Must-Have Requirements 7+ years experience in data engineering or data platform development Strong experience with Databricks, Spark and distributed data processing Experience with Unity Catalog, Delta Lake and streaming architectures Advanced Python and Microsoft SQL Server development Experience designing enterprise data ...

A leading organisation undergoing major investment in its data and engineering capabilities is looking for a Senior Automation QA Engineer to strengthen quality across complex, distributed systems. This role sits at the intersection of software ...

systems, and modern cloud architectures . What youll be doing Design and build scalable data pipelines (batch & streaming) Work with technologies like Kafka, Flink, Spark, AWS services Develop real-time, event-driven data solutions Contribute to data architecture and modelling best practices Implement CI/CD pipelines and Infrastructure … deliver clean datasets What were looking for Strong programming skills in Java (preferred) or Python Experience with streaming technologies (Kafka/Flink/Spark) Knowledge of distributed systems and large-scale data processing Experience with cloud platforms (AWS preferred) Familiarity with DevOps tools and containerisation (Docker/Kubernetes ...

Developing and maintaining scalable data pipelines using Python - Automating manual Excel/VBA workflows into reliable, production-grade systems - Working with large datasets using Spark and Azure (Databricks/Synapse) - Supporting data modelling and analytics use cases (Crossing over to more Data Science driven work) - Driving best practices including … code reviews - Collaborating with stakeholders to deliver actionable data solutions What they re looking for: - Strong Python and SQL skills - Experience or exposure to Spark, big data tools, or cloud platforms (ideally Azure) What s on offer: - The chance to join a business at an exciting stage of growth ...

Developing and maintaining scalable data pipelines using Python - Automating manual Excel/VBA workflows into reliable, production-grade systems - Working with large datasets using Spark and Azure (Databricks/Synapse) - Supporting data modelling and analytics use cases (Crossing over to more Data Science driven work) - Driving best practices including … code reviews - Collaborating with stakeholders to deliver actionable data solutions What they’re looking for: - Strong Python and SQL skills - Experience or exposure to Spark, big data tools, or cloud platforms (ideally Azure) What’s on offer: - The chance to join a business at an exciting stage of growth ...

Educated to degree level in a relevant IT subject. Expertise in designing and implementing data pipelines using Azure services (Azure Data Factory, Data storage), Spark and Databricks. Expertise in data modelling, database design and designing enterprise data architecture . Azure Data Stack SQL server, NoSQL, Nanobricks, ADT, Spark ...

generative AI inference and agentic applications. Your depth of expertise in technologies including Golang and Python programming languages, popular distributed compute frameworks including Spark/Dask/Ray/Flink, container (e.g., Kubernetes) and serverless (e.g., AWS Lambda) runtime environments, and ML+AI workload patterns will provide an amplifying technical … control and data plane implementations required to realize a highly available, multi-tenant, large scale and a secure machine learning platform Develop Ray and Spark distributed compute engine solutions to accelerate diverse workloads from LLM pre-training and reinforcement learning to large-scale data processing, while maximizing compute unit ...