126 to 150 of 413 Apache Spark Jobs in England

driven decision making. What youll be doing Building and optimising scalable data pipelines using Java (primary) and technologies like Kafka, Flink, or Spark Developing real-time streaming solutions and event-driven architectures using AWS services (Kinesis, MSK, Lambda, Glue) Contributing to data architecture, modelling, and best practices across modern ...

modeling and data transformation techniques; Knowledge of maintaining high data quality and integrity; Strong software development skills with Hadoop technologies such as Hive and Spark; Experience with Workday Studio and managing code within the Workday environment; Knowledge of SaaS applications such as Salesforce or other CRM systems; Knowledge ...

cloud platforms (GCP or Azure) and Databricks. Familiarity with deploying ML workflows using MLflow, Vertex AI, or Azure ML. Nice‐to‐have: Experience with Spark, CI/CD pipelines, and orchestration tools. Nice‐to‐have: Knowledge of Elasticsearch or digital/web analytics platforms. Nice‐to‐have: Understanding ...

complex joins, window functions, aggregation, and query optimization; Experience with a major data warehouse or big data technology such as BigQuery, Snowflake, Redshift, Hive, Spark, Vertica, or StarRocks; Experience with ETL/ELT tools or orchestration frameworks such as Airflow, dbt, or internal frameworks; Proficiency in at least ...

concept to production. Strong knowledge of Python and core ML libraries (e.g., spaCy, NumPy, SciPy, Transformers). Understanding of big data and cloud technologies (Spark, Hadoop, Azure, AWS, or GCP). Deep expertise in MLOps principles, scalable model deployment, and model governance. Excellent communication and stakeholder management skills with ...

alerting for ML systems (model drift, data quality, feature coverage, etc.) Design and scale data ingestion and feature transformation flows using batch (e.g., Spark/BigQuery) and streaming (Kafka or equivalent) Contribute to internal Python libraries and platform tooling that accelerate experimentation and deployment for all model teams Ensure ...

modern cloud engineering practices. Wherever you land, you’ll be working with some of the biggest datasets in the UK - using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Cloud-native AI engineering and large-scale data platforms ...

modern cloud engineering practices. Wherever you land, you’ll be working with some of the biggest datasets in the UK — using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. Cloud-native AI engineering and large-scale data platforms, leveraging Google Cloud technologies and modern ...

modern cloud engineering practices. Wherever you land, you'll be working with some of the biggest datasets in the UK - using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Cloud-native AI engineering and large-scale data platforms ...

modern cloud engineering practices. Wherever you land, you'll be working with some of the biggest datasets in the UK - using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Cloud-native AI engineering and large-scale data platforms ...

including migration from legacy data platforms, lakehouse modernisation, performance optimisation, governance, and production‐grade delivery. Deep expertise across Databricks including Delta Lake, Unity Catalog, Spark, Lakehouse architecture, Databricks SQL, Workflows, Delta Live Tables, MLflow, Mosaic AI, Agent Bricks, Vector Search, and broader Data & AI Intelligence capabilities. Strong experience defining ...

embedded into business processes and decision-making.Technical Skills:Hands-on experience in one or more of the following areas:Programming & Analytics – Python, SQL, R, Spark, machine learning and advanced analyticsCloud & Data Platforms – Microsoft Fabric, Databricks, Snowflake, Azure, AWS and GCPAI Technologies – GenAI frameworks, Agentic AI, vector databases, semantic search ...

Data & AI programmes, including planning, mobilisation, resourcing, financial management, delivery governance, risk and dependency management. Deep expertise across Databricks including Delta Lake, Unity Catalog, Spark, Lakehouse architecture, Databricks SQL, Workflows, Delta Live Tables, MLflow, Mosaic AI, Agent Bricks, Vector Search, and broader Data & AI Intelligence capabilities. Strong experience defining ...

deploy scalable batch and streaming data pipelines, AI-ready data products, feature pipeline and analytics solutions on the Databricks Platform using Delta Lake, Spark SQL/PySpark, Unity Catalog and Databricks SQL. Deliver solutions across the full data & AI engineering lifecycle, from requirements analysis and solution design through development … scalable data platforms, data products and analytics solutions using modern lake house or cloud data warehouse technologies Proficiency writing advanced, highly optimised SQL and Spark (PySpark/Scala), with a focus on performance optimisation and maintainability. Designing and building solutions on a cloud data warehouse or lakehouse platform (e.g. ...

modern data engineering best practices You have experience with Databricks, ideally also Azure You have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You're excited to join a start-up in a role that you can shape and influence ...

skills Desirable Experience Experience with any of the following would be advantageous: * Angular, React or Vue * Chart.js, D3.js or other data visualisation tools * Databricks, Apache Spark or distributed data processing * Docker, Kubernetes or Cloud Run * Terraform or other infrastructure-as-code technologies * GPU or accelerated computing technologies such ...

internal knowledge Competencies and Behaviours: 3+ years experience in data platform engineering, cloud engineering, or similar roles Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/ ...

hands-on engineering experience delivering enterprise data platforms or analytics solutions. Strong commercial experience building and delivering Databricks solutions in production. Deep expertise in Apache Spark, including performance tuning and optimisation at scale. Expert-level Python and SQL, with a track record of writing production-quality, well-tested ...

mesh concepts and their practical implementation within large organisations Experience integrating Databricks with AWS services including S3, IAM, VPC, and AWS Glue Familiarity with Apache Spark, dbt, data pipelines, and modern analytics architectures Understanding of access management, audit frameworks, and enterprise governance controls Delivery Experience Experience leading data ...

/Azure, Terraform, Kubernetes, CI/CD, Linux, automation, DevSecOps and SRE Data Engineering - Python, SQL, ETL/ELT, modern cloud data platforms, Spark/Databricks and data architecture AI & Machine Learning - ML Engineering, MLOps, Python, model deployment, evaluation, LLM/GenAI and production AI Technical Leadership - Senior/ ...

/Azure, Terraform, Kubernetes, CI/CD, Linux, automation, DevSecOps and SRE Data Engineering - Python, SQL, ETL/ELT, modern cloud data platforms, Spark/Databricks and data architecture AI & Machine Learning - ML Engineering, MLOps, Python, model deployment, evaluation, LLM/GenAI and production AI Technical Leadership - Senior/ ...

GenAI infrastructure and workflows Skills and experience: Passionfor building scalable, resilient cloud data platforms Strongexperience with Databricks or Snowflake on AWS ProvenPython skills, including Spark and Airflow expertise AdvancedSQL skills and end‐to‐end data modelling experience Experiencebuilding batch and real-time data integrations Hands-onCICD skills with ...

systems Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

systems Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

experience as the Senior Data Engineer: Passionfor building scalable, resilient cloud data platforms Strongexperience with Databricks or Snowflake on AWS ProvenPython skills, including Spark and Airflow expertise AdvancedSQL skills and end-to-end data modelling experience Experiencebuilding batch and real-time data integrations Hands-onCICD skills with Git, Jenkins ...