14 of 14 Remote Apache Spark Jobs in Scotland

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

platform. Key Skills/requirements Lead the migration of existing AWS-based data pipelines to Databricks. Design and implement scalable data engineering solutions using Apache Spark on Databricks. Collaborate with cross-functional teams to understand data requirements and translate them into efficient pipelines. Optimize performance and cost-efficiency … security, and access control within Databricks. Provide technical mentorship and guidance to junior engineers. Must-Have Skills: Strong hands-on experience with Databricks and Apache Spark (preferably PySpark). Proven track record of building and optimizing data pipelines in cloud environments. Experience with AWS services such ...

record in managing large-scale system retrofitting and data migrations. Familiar with ETL/ELT frameworks, and experienced with Big Data Processing Tools (e.g. Spark, Airflow, Hive, etc.) Degree in STEM, or similar field; a Master's is a plus Data engineering certification (e.g IBM Certified Data Engineer ...

record in managing large-scale system retrofitting and data migrations. Familiar with ETL/ELT frameworks, and experienced with Big Data Processing Tools (e.g. Spark, Airflow, Hive, etc.) Degree in STEM, or similar field; a Master's is a plus Data engineering certification (e.g IBM Certified Data Engineer ...

record in managing large-scale system retrofitting and data migrations. Familiar with ETL/ELT frameworks, and experienced with Big Data Processing Tools (e.g. Spark, Airflow, Hive, etc.) Degree in STEM, or similar field; a Master's is a plus Data engineering certification (e.g IBM Certified Data Engineer ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

chunking, reranking techniques, etc. Extra awesome: You have experience working in a B2B SaaS company Experience with orchestration tools (e.g. Airflow) Familiarity with SQL, Spark, or other data processing frameworks Knowledge of Snowflake or other cloud data warehouses. Strong familiarity with agentic frameworks for decision-making systems. *Typeform drives ...

ideal person for this role will have a strong background in data engineering , with experience building modern data solutions using technologies like Kafka , Spark , Databricks , dbt , and Airflow . You'll know your way around cloud platforms (AWS, GCP, or Azure) and be confident coding in Python , Java ...

ideal person for this role will have a strong background in data engineering , with experience building modern data solutions using technologies like Kafka , Spark , Databricks , dbt , and Airflow . You'll know your way around cloud platforms (AWS, GCP, or Azure) and be confident coding in Python , Java ...

collaborative, forward-thinking culture where engineers work on large-scale datasets, distributed systems, and modern cloud platforms. Teams leverage cutting-edge tools such as Spark, Airflow, Kafka, dbt, and Databricks to build resilient, scalable, and high-quality data solutions. Why this role? ** Lead a talented team of data engineers ...

analytics. Familiarity with industry-standard BI, ETL/ELT, and orchestration tools. Hands-on coding experience with SQL, Python, Java, and/or Spark for building and maintaining analytics solutions. Strong experience with large-scale database technologies (Snowflake, Netezza, Exadata, Teradata, Greenplum, etc.). Demonstrated ability to implement data ...

analytics. Familiarity with industry-standard BI, ETL/ELT, and orchestration tools. Hands-on coding experience with SQL, Python, Java, and/or Spark for building and maintaining analytics solutions. Strong experience with large-scale database technologies (Snowflake, Netezza, Exadata, Teradata, Greenplum, etc.). Demonstrated ability to implement data ...

analytics. Familiarity with industry-standard BI, ETL/ELT, and orchestration tools. Hands-on coding experience with SQL, Python, Java, and/or Spark for building and maintaining analytics solutions. Strong experience with large-scale database technologies (Snowflake, Netezza, Exadata, Teradata, Greenplum, etc.). Demonstrated ability to implement data ...

analytics. Familiarity with industry-standard BI, ETL/ELT, and orchestration tools. Hands-on coding experience with SQL, Python, Java, and/or Spark for building and maintaining analytics solutions. Strong experience with large-scale database technologies (Snowflake, Netezza, Exadata, Teradata, Greenplum, etc.). Demonstrated ability to implement data ...