20 of 20 Remote/Hybrid Apache Spark Jobs in London

with structured streaming, Delta Lake and Delta Live TablesProven experience in Lakehouse migration and modernization using open table formats such as Delta Lake or Apache IcebergProficiency with cloud-native services on Azure and knowledge of multi-cloud environments (AWS or GCP)Hands-on experience with CI/… governance, security practices and enterprise integration controlsBackground in financial trading data or related domainsOfficial Databricks certifications such as Certified Data Engineer Professional or Apache Spark DeveloperWe offerEPAM Employee Stock Purchase Plan (ESPP)Protection benefits including life assurance, income protection and critical illness cover Private medical insurance and dental ...

expertise: Platform Experience: Strong, hands-on experience with Databricks or Snowflake on AWS. Programming & Orchestration: Proven Python skills, including deep expertise in PySpark (or Spark) and Airflow. Data Modelling: Advanced SQL skills and extensive experience with end-to-end data modelling. Data Integration: Proven experience building both batch ...

while collaborating with the Data & Analytics Lead, technical colleagues and business stakeholders. Experience in the following areas would also be useful: Materialised Lake Views Apache Spark or Python. Power BI semantic modelling. APIs and cloud business system integration. Data governance, lineage and quality frameworks. Professional services experience. What ...

Senior Data Engineer: Passion for building scalable, resilient cloud data platforms Strong experience with Databricks or Snowflake on AWS Proven Python skills, including Spark and Airflow expertise Advanced SQL skills and end-to-end data modelling experience Experience building batch and real-time data integrations Hands-on CICD skills ...

requiredDo not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, PalantirExperience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLMSearch and ranking experience. For example Elasticsearch or rerankersBackground in time ...

experience with the SQL Server platform and T-SQL querying. Familiarity with cloud data platforms (e.g., AWS, Azure) and big data technologies (e.g., Kafka, Spark, Databricks, Snowflake).Expert-level knowledge of building ETL processes in Python. Expert-level knowledge of performance tuning and optimization in a Python/ ...

Maintain the controls and records needed to support auditing, data traceability and compliance requirements. Performance and operational management Investigate and improve the performance of Spark workloads, Delta tables and SQL compute. Review resource consumption and identify ways to reduce unnecessary platform and cloud expenditure. Set up monitoring and alerting … knowledge sharing and mentoring across the team. What we are looking for Practical experience managing Databricks in a production environment. A strong understanding of Apache Spark, Delta Lake and Unity Catalog. Experience working with one or more major cloud providers, including storage, networking and identity management. Confidence using ...

Feast, AWS Feature Store) Experience with containerisation (Docker) and orchestration (Kubernetes, ECS) Familiarity with Infrastructure as Code (Terraform, CloudFormation) Experience with data processing frameworks (Spark, Dask) for large-scale workloads Understanding of data governance and compliance frameworks Experience working in regulated industries (finance, healthcare, or similar) Security Clearance ...

Feast, AWS Feature Store) Experience with containerisation (Docker) and orchestration (Kubernetes, ECS) Familiarity with Infrastructure as Code (Terraform, CloudFormation) Experience with data processing frameworks (Spark, Dask) for large-scale workloads Understanding of data governance and compliance frameworks Experience working in regulated industries (finance, healthcare, or similar) Security Clearance ...

Feast, AWS Feature Store) Experience with containerisation (Docker) and orchestration (Kubernetes, ECS) Familiarity with Infrastructure as Code (Terraform, CloudFormation) Experience with data processing frameworks (Spark, Dask) for large-scale workloads Understanding of data governance and compliance frameworks Experience working in regulated industries (finance, healthcare, or similar) Security Clearance ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

Event Hubs on Azure, or BigQuery, Dataflow and Pub/Sub on GCP Databricks and Snowflake running on top of any of the three Spark and distributed processing frameworks for large-scale data Streaming technologies such as Kafka or the native cloud equivalents You’re expected to pick ...

Familiarity with monitoring tools such as CloudWatch, Prometheus or Grafana. Useful additional experience Relevant AWS certifications. Exposure to data-processing environments using Databricks or Apache Spark. The person we are looking for You enjoy making systems easier to run and reducing the need for manual intervention. You investigate problems ...

management and international regulatory compliance. YOUR SKILLS AND EXPERIENCE ESSENTIAL CRITERIA Strong experience designing, building and maintaining scalable data pipelines and integration solutions using Apache Airflow (or similar orchestration platforms), Python, SQL and bash/linux. Experience developing cloud-native data platforms and solutions on AWS, including working with … platforms such as Amazon Bedrock. Experience developing data-based apps in Streamlit/Shiny Skills in 'Big Data' technology tools like Hadoop/Spark Knowledge of Infrastructure as Code tools like CloudFormation/Terraform If you can bring some of these skills and experiences, along with transferable strengths ...

understanding of statistics, machine learning methodologies, and model evaluation. Experience delivering AI solutions from proof-of-concept through to production. Exposure to Databricks, MLflow, Spark, or similar cloud-based data platforms. Ability to influence senior stakeholders and lead technical strategy Desirable Experience Experience building multi-agent AI systems ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming).Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming). Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

Hybrid- London, UKProduct Management & Engineering – Product Engineering/Full-time/HybridWho We AreAt Firstup, our mission is to improve the employee experience at every moment that matters, large and small. As the communication pipeline ...

Senior Data Scientist - Python/Machine Learning Permanent Location: UK - Remote Salary: £65,000 - £70,000 (+ benefits) Skills: Python, Machine Learning, Statistical Modelling, AWS, Data Engineering An exciting opportunity has arisen for an experienced ...

Scala programming to develop robust, scalable, and high-performance data processing applications. Build and optimise large-scale data processing and analytics pipelines using Apache Spark. Develop and maintain end-to-end data pipelines for both batch and real-time data processing. Apply solid understanding of data engineering concepts, including … business stakeholders. Skills & Experience: Proven experience as a Software Engineer with a strong focus on Scala development. Extensive hands-on experience with OpenShift and Apache Spark. In-depth understanding of data engineering principles and practises. Experience in developing and maintaining scalable data pipelines for batch and real-time processing. ...