26 to 50 of 96 Apache Spark Jobs in England

deliver intelligent data products, leveraging modern cloud and AI technologies. Key Responsibilities Design and implement robust data pipelines and ML workflows using Python, SQL, Spark, and Databricks. Develop and deploy machine learning models (including NLP, deep learning, and agentic AI) in production environments. Integrate data from diverse sources, including ...

Vault . Hands-on experience exposing Lakehouse solutions to BI tools (e.g. Power BI , Tableau ). Proficiency in Databricks Workflows , Delta Live Tables , and Spark for batch and streaming pipelines. Strong programming capability in Python , Scala , and/or SQL . Practical experience with at least one major cloud ...

Skills & Experience* Proven experience delivering enterprise-scale Databricks solutions end-to-end.* Strong background in Lakehouse Architecture, including structured and unstructured data.* Expertise in Spark, PySpark, Delta Lake, and Databricks workflows.* Experience building scalable ETL/ELT pipelines, including Delta Live Tables.* Strong programming skills in Python, Scala ...

Salesforce data ingestion experience Exposure to NoSQL databases such as MongoDB, Cassandra, or DynamoDB Experience supporting Power BI and Tableau reporting Proficiency in Python, Spark, or similar data engineering tools Understanding of AI and ML trends Strong communication and stakeholder management skills ...

auditable, lineage-aware, idempotent, and resilient in regulated environments. Implement robust data quality controls including testing, validation, monitoring, and alerting. Drive performance optimization across Spark and Snowflake workloads. Partner with Product, Engineering, DevOps, and Regulatory teams to translate requirements into scalable technical designs. Improve engineering standards, processes, and tooling ...

auditable, lineage-aware, idempotent, and resilient in regulated environments. Implement robust data quality controls including testing, validation, monitoring, and alerting. Drive performance optimization across Spark and Snowflake workloads. Partner with Product, Engineering, DevOps, and Regulatory teams to translate requirements into scalable technical designs. Contribute to the continuous improvement ...

Face Transformers to enable capabilities such as intelligent document processing, conversational AI, and semantic search. Knowledge and experience of some of the following tools; Apache Spark, Databricks, Pandas for data processing and feature engineering SQL, MongoDB for structured and unstructured data management SageMaker, Azure ML for model deployment ...

pipelines and platforms. Strong expertise in SQL, data modelling, and database optimisation (relational, vector and NoSQL). Proficiency in distributed data processing frameworks (e.g., Spark, Flink, Hadoop). Deep knowledge of cloud data platforms (AWS, Azure, or GCP) and associated services. Strong programming skills in Python, Java, or Scala … Proven track record with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Strong proficiency in relational and non-relational databases (e.g., PostgreSQL, MongoDB, Cassandra). Deep knowledge ...

Azure or AWS Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

Azure or AWS Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

modelling • Experience building production-grade ML systems (e.g. forecasting, optimisation, CV) • Solid ML Ops and data engineering experience • Proficiency in Python & SQL (Airflow, PyTorch, Spark a plus) • Experience with AWS (SageMaker, Lambda, etc.) • Interest in backend development (Django or Go) • High level STEM Degree from a leading university. Nice ...

paced financial services environment. Key Responsibilities: Design, develop, and maintain applications using Scala, Python, Hadoop and Java . Work with Big Data technologies , including Spark, Hive (nice to have). Collaborate with cross-functional teams to deliver scalable, high-performance solutions. Participate in code reviews, testing, and performance optimization. … years of software development experience. Strong hands-on expertise in Scala (mandatory) , plus Python and Java . Experience with Big Data frameworks ; Apache Spark experience is an advantage. Solid understanding of software engineering principles, data structures, and algorithms. Strong problem-solving skills and ability to work ...

support Workflow orchestration (Airflow or Dagster) AWS data services (Athena, S3, MWAA, Iceberg) Data quality monitoring & incident management Nice to Have: Airbyte DBT Apache Spark Grafana FTP/data distribution platforms Financial data experience ...

explain technical concepts to varied audiences. Nice to Have Cloud experience: AWS/Azure/GCP . Containerisation: Docker, Kubernetes . Distributed processing: Apache Spark/Flink . Please apply within for further details - Matthew Holmes, Harvey Nash Always use these settings ...

engineering across ingestion, transformation and data centralisation. Experience with tools such as Glue, Fivetran, Kafka, Dataflow, ADF or Kinesis. Transformation experience using dbt, Spark or Databricks. Confidence working directly with clients and engaging in technical discussions. Exposure to dashboards, analytics, ML or LLM concepts is beneficial, but not essential. ...

engineering across ingestion, transformation and data centralisation. Experience with tools such as Glue, Fivetran, Kafka, Dataflow, ADF or Kinesis. Transformation experience using dbt, Spark or Databricks. Confidence working directly with clients and engaging in technical discussions. Exposure to dashboards, analytics, ML or LLM concepts is beneficial, but not essential. ...

engineering across ingestion, transformation and data centralisation. Experience with tools such as Glue, Fivetran, Kafka, Dataflow, ADF or Kinesis. Transformation experience using dbt, Spark or Databricks. Confidence working directly with clients and engaging in technical discussions. Exposure to dashboards, analytics, ML or LLM concepts is beneficial, but not essential. ...

Ensure high standards of data quality, governance, security and accessibility across data platforms Develop scalable data transformation logic using Python, SQL and Spark-based frameworks Monitor pipeline performance and troubleshoot data workflows to ensure reliability and efficiency Work closely with data scientists, analysts and engineering teams to support advanced … data services (such as S3, Glue, Redshift, EMR, Athena, Lambda or similar) Strong programming experience with Python and SQL Hands-on experience with Spark or distributed data processing frameworks Proven experience building and maintaining data pipelines, data lakes and data warehouses Strong understanding of data modelling, metadata management ...

mindset Support identity, RBAC and governance controls across Azure services Tech stack Azure Azure Databricks Terraform Azure DevOps/CI/CD Python SQL Apache Spark/PySpark Entra ID/RBAC/governance Networking/platform security/automation What were looking for Strong experience in Azure ...

statistical techniques, including validation. Understanding of data analytics and data visualisation techniques. Ability to process large datasets via batch or stream processing using Apache Spark or similar tools. Exposure to techniques used for acquiring and fusing data. Experience with cloud platforms (preferably AWS) or implementing cloud-based data ...

experience building data pipelines in production environments. Proficiency with Microsoft Fabric, including Data Factory, lakehouses, warehouses, notebooks and semantic models. Strong skills in PySpark, Spark SQL and T-SQL for data transformation. Experience with medallion architecture or equivalent layered data processing patterns. Working knowledge of OneLake, delta tables ...

will have strong experience in data engineering, with advanced proficiency in SQL and Python, alongside exposure to large-scale data processing frameworks such as Apache Spark. You should have a solid understanding of data architecture, data wrangling and data quality practices, as well as experience working within financial markets ...

Kafka Exposure to cloud platforms (AWS, Azure, or GCP) Familiarity with containerisation technologies (Docker, Kubernetes) Experience with distributed data processing tools such as Spark or Flink Additional Information Daily rate: £850 - £900 Hybrid working model in London Initial contract with strong likelihood of extension McGregor Boyall is an equal ...

common ML frameworks such as PyTorch, TensorFlow, or scikit-learn * Exposure to ML/MLOps tooling such as SageMaker, MLflow, or TFServing * Experience with Spark, Databricks, CI/CD tools, and streaming or orchestration systems like Kafka or Airflow How to Apply If you are interested in this Machine ...

databases into ADLS Gen2 and OneLake Develop structured data models aligned to Lakehouse, Medallion Architecture, and Delta Lake patterns Continuously profile, debug, and optimise Spark jobs, SQL queries, and Python scripts for maximum performance and cost-efficiency at scale Champion DevOps best practices: implement infrastructure-as-code (Terraform), automated … Essential Technical Skills Advanced proficiency in Python and PySpark, writing clean, modular, object-oriented code for data transformations Strong command of SQL (T-SQL, Spark SQL) for data exploration, validation, and final-stage modelling Deep hands-on experience with Microsoft Fabric and its tooling such as Azure Data Factory ...