21 of 21 Remote/Hybrid Apache Spark Jobs in England

evolution of data architecture and help shape the organisations data engineering capabilities. Key Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark and Python. Develop and maintain robust ELT processes to ingest, transform and integrate data from multiple sources. Contribute to the design and evolution … platform performance. Essential Skills & Experience Strong hands-on experience with Databricks . Proven experience designing, implementing and optimising data pipelines. Strong knowledge of Apache Spark and Python . Understanding of Lakehouse architecture and modern data engineering principles. At least five years experience across ELT, data engineering and data ...

Senior Data Engineer Role (Python/Spark/Databricks) - Hybrid - Contract - Banking Role - Senior Data Engineer Rate - £850 p/d (Inside IR35) Duration - 6 months with very likely extension Location - Hybrid/Liverpool Street (London) - 3 days per week in a Liverpool Street office About the Role Global … delivering critical data platforms and analytics capabilities across the business. This is a genuinely hands-on engineering role, requiring strong expertise in Python, Databricks, Spark/PySpark and SQL. The successful candidate will be actively involved in the design, development, optimisation and support of enterprise-scale data platforms, working ...

engineers where needed. Core Data Engineering Strong programming proficiency in Java (preferred) or Python. Hands-on experience with at least one of: Kafka, Flink, Spark (Flink/Kafka preferred for streaming). Solid understanding of stream processing concepts (e.g., event time, state, backpressure). Understanding of software engineering best ...

programming skills, with experience developing and maintaining data pipelines Familiarity with ETL/ELT technologies and data streaming tools such as Airflow, Azure, SSIS, Spark or Kafka Experience working with databases and data platforms such as PostgreSQL, SQL Server, Oracle, Snowflake or similar technologies Knowledge of data visualisation tools ...

Must-Have Skills & Experience Deep expertise designing and building scalable data pipelines and data products in cloud environments Strong hands-on experience with Databricks, Spark, SQL, and Python Experience developing and optimizing batch and real-time data processing solutions Strong understanding of modern data architecture patterns, including Kimball, Data ...

Must-Have Skills & Experience Deep expertise designing and building scalable data pipelines and data products in cloud environments Strong hands-on experience with Databricks, Spark, SQL, and Python Experience developing and optimizing batch and real-time data processing solutions Strong understanding of modern data architecture patterns, including Kimball, Data ...

Must-Have Skills & Experience Deep expertise designing and building scalable data pipelines and data products in cloud environments Strong hands-on experience with Databricks, Spark, SQL, and Python Experience developing and optimizing batch and real-time data processing solutions Strong understanding of modern data architecture patterns, including Kimball, Data ...

environments Owning the cloud infrastructure underpinning data systems through a DevOps approach Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data ...

into business processes and decision-making. Technical Skills: Hands-on experience in one or more of the following areas: Programming & Analytics – Python, SQL, R, Spark, machine learning and advanced analytics Cloud & Data Platforms – Microsoft Fabric, Databricks, Snowflake, Azure, AWS and GCP AI Technologies – GenAI frameworks, Agentic AI, vector databases ...

Feast, AWS Feature Store) Experience with containerisation (Docker) and orchestration (Kubernetes, ECS) Familiarity with Infrastructure as Code (Terraform, CloudFormation) Experience with data processing frameworks (Spark, Dask) for large-scale workloads Understanding of data governance and compliance frameworks Experience working in regulated industries (finance, healthcare, or similar) Security Clearance ...

Feast, AWS Feature Store) Experience with containerisation (Docker) and orchestration (Kubernetes, ECS) Familiarity with Infrastructure as Code (Terraform, CloudFormation) Experience with data processing frameworks (Spark, Dask) for large-scale workloads Understanding of data governance and compliance frameworks Experience working in regulated industries (finance, healthcare, or similar) Security Clearance ...

Feast, AWS Feature Store) Experience with containerisation (Docker) and orchestration (Kubernetes, ECS) Familiarity with Infrastructure as Code (Terraform, CloudFormation) Experience with data processing frameworks (Spark, Dask) for large-scale workloads Understanding of data governance and compliance frameworks Experience working in regulated industries (finance, healthcare, or similar) Security Clearance ...

effectively with technical teams including Data Scientists and MLOps Engineers. Desirable Skills Experience working with Data Science and/or MLOps environments. Experience with Apache Spark. Experience with Scala. Experience working with S3 and cloud storage technologies. Experience with graph databases. Experience with Elasticsearch. Experience with vector databases/ ...

large language model (LLM) systems, including retrieval augmented generation (RAG) and fine-tuning. Build scalable ETL/ELT pipelines using tools such as Spark or dbt, so data is high quality, time-stamped, and ready for both training and real-time inference. Design and deploy data collection systems ...

Senior Data Engineer – SQL/Python/AWS Adria Solutions is recruiting a Senior Data Engineer for a growing business developing a new digital services platform. With established backing, the business is building its data ...

understanding of statistics, machine learning methodologies, and model evaluation. Experience delivering AI solutions from proof-of-concept through to production. Exposure to Databricks, MLflow, Spark, or similar cloud-based data platforms. Ability to influence senior stakeholders and lead technical strategy Desirable Experience Experience building multi-agent AI systems ...

kubernetes) Experience with Key value stores Experience in git and basic Unix Commands You might also have (Desirable) Experience with large data processing technologies (Apache Spark) Other helpful information Hybrid Working: At EMBL-EBI we are pleased to offer hybrid working options for all our employees. A dedicated ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming). Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

role combines L2/L3 production support and Data Engineering tasks. Databricks experience covering notebooks, scheduled jobs, cluster configuration, driver/executor logs, Spark UI + performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark ...

role combines L2/L3 production support and Data Engineering tasks. Databricks experience covering notebooks, scheduled jobs, cluster configuration, driver/executor logs, Spark UI + performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark ...

Scala programming to develop robust, scalable, and high-performance data processing applications. Build and optimise large-scale data processing and analytics pipelines using Apache Spark. Develop and maintain end-to-end data pipelines for both batch and real-time data processing. Apply solid understanding of data engineering concepts, including … business stakeholders. Skills & Experience: Proven experience as a Software Engineer with a strong focus on Scala development. Extensive hands-on experience with OpenShift and Apache Spark. In-depth understanding of data engineering principles and practises. Experience in developing and maintaining scalable data pipelines for batch and real-time processing. ...