11 of 11 Remote Apache Spark Jobs in Scotland

SQL Cloud platform architecture (AWS, Azure, GCP) Data Warehousing (Snowflake, Redshift, Synapse, BigQuery, etc.) Data Lakes/Lakehouses and analytical tools (Databricks, Azure Fabric/OneLake, AWS Lake Formation, Spark, Athena, etc.) Relational, NoSQL, graph and vector databases Streaming technologies (Kafka, Kinesis, Flink, etc.) Containers and related services (Docker, Kubernetes, container Registries, etc) CI/CD and other DevOps … practices such as IaC Testing Nice to have - Additional experience with the following would be beneficial but not essential: Data modelling approaches (Kimball, Imnon) Orchestration tools - Apache Airflow, Prefect or cloud-native tools Backend software development (Java, APIs, Scalability, Logging and Monitoring etc.) MLFlow and other MLOps/Machine Learning Engineering processes to support advanced analytical use cases LLMs More ❯

of experience in data engineering or a related field, with a focus on building scalable data systems and platforms. Strong expertise with modern data tools and frameworks such as Spark , dbt , Airflow , Kafka , Databricks , and cloud-native services (AWS, GCP, or Azure). Deep understanding of data modeling , distributed systems , streaming architectures , and ETL/ELT pipelines . Proficiency More ❯

experience we are looking for includes: DevOps automation, idempotent deployment testing, and continuous delivery pipelines Networking and security protocols, load balancers, API Gateways ETL tooling and workflow engines (e.g., Spark, Airflow, Dagster, Flyte) Accelerated compute libraries and hardware (e.g. pytorch, NVIDIA GPUs) Data modeling, and strategies for cleaning and validating data at scale Performance tuning on RDBMS or Big More ❯

DataOps methodologies and tools, including experience with CI/CD pipelines, containerisation, and workflow orchestration. Familiar with ETL/ELT frameworks, and experienced with Big Data Processing Tools (e.g. Spark, Airflow, Hive, etc.) Knowledge of programming languages (e.g. Java, Python, SQL) Hands-on experience with SQL/NoSQL database design Degree in STEM, or similar field; a Master's More ❯

learn, Hugging Face Transformers, Keras, etc.) Familiarity with Python for AI/ML development is a significant advantage. Experience with data engineering pipelines or big data technologies (e.g., Kafka, Spark) is a plus. The recruitment process you can expect for this role is an initial call with your dedicated Talent Acquisition Partner who will chat with you about Waracle More ❯

Hugging Face Transformers, Keras, etc.). Familiarity with Python for AI/ML development is a significant advantage. Experience with data engineering pipelines or big data technologies (e.g., Kafka, Spark) is a plus. The recruitment process you can expect for this role is an initial call with your dedicated Talent Acquisition Partner who will chat with you about Waracle More ❯

functional teams. Knowledge of innovative software practices (SCRUM/Agile, microservices, containerization like Docker/Kubernetes). we'd also encourage you to apply if you possess: Experience with Spark/Databricks. Experience deploying ML models via APIs (e.g., Flask, Keras). Startup experience or familiarity with geospatial and financial data. The Interview Process (subject to change): You’ll More ❯

FinTech team! You will develop scalable data pipelines, ensure data quality, and support business decision-making with high-quality datasets. -Work across technology stack: SQL, Python, ETL, Big Query, Spark, Hadoop, Git, Apache Airflow, Data Architecture, Data Warehousing -Design and develop scalable ETL pipelines to automate data processes and optimize delivery -Implement and manage data warehousing solutions, ensuring … data integrity through rigorous testing and validation -Lead, plan and execute workflow migration and data orchestration using Apache Airflow -Focus on data engineering and data analytics Requirements: -5+ years of experience in SQL -5+ years of development in Python -MUST have strong experience in Apache Airflow -Experience with ETL tools, data architecture, and data warehousing solutions More ❯

FinTech team! You will develop scalable data pipelines, ensure data quality, and support business decision-making with high-quality datasets. -Work across technology stack: SQL, Python, ETL, Big Query, Spark, Hadoop, Git, Apache Airflow, Data Architecture, Data Warehousing -Design and develop scalable ETL pipelines to automate data processes and optimize delivery -Implement and manage data warehousing solutions, ensuring … data integrity through rigorous testing and validation -Lead, plan and execute workflow migration and data orchestration using Apache Airflow -Focus on data engineering and data analytics Requirements: -5+ years of experience in SQL -5+ years of development in Python -MUST have strong experience in Apache Airflow -Experience with ETL tools, data architecture, and data warehousing solutions More ❯

Our client is a leading UK-based consultancy seeking a skilled professional to shape data strategies, mentor dynamic teams, and deliver cutting-edge solutions. With hands-on expertise in Spark, SQL, and cloud platforms like Azure, you’ll lead end-to-end projects, drive innovation, and collaborate with clients across industries. What You’ll Do: Lead complex data engineering … multi-project environments experience. Expertise in ETL, data modelling, and Azure Data Services. Experience in designing and implementing data pipelines, data lakes, and data warehouses. Hands-on experience with Apache Spark and bonus points for Microsoft Fabric. Any certifications are a bonus. Benefits: Competitive base salary Hybrid work once a week into their Edinburgh office 25 days holiday More ❯

products includes on-premise cloud solutions such as Openstack, MicroCloud and Ceph, and solutions that could be deployed either on-premises or in public clouds such as Kubernetes, Kubeflow, Spark, PostgreSQL, etc. The team works hands-on with the technologies by deploying, testing and handing over the solution to our support or managed services team at the end of More ❯