51 to 75 of 75 Apache Spark Jobs in the UK

maintain operational continuity and quality output Knowledge of data processing frameworks, including ETL pipelines, data integration, and data modeling (experience with Databricks or Spark preferred) Strong analytical and problem-solving skills with the ability to communicate clearly to both technical and non-technical stakeholders Basic understanding of artificial intelligence ...

operations. Demonstrated depth in Big Data engineering, with proficiency in at least one of the following areas: distributed data processing platforms such as Databricks, Spark, or EMR search platforms such as OpenSearch or Elasticsearch graph databases such as Neo4j Strong understanding of Agile and Lean practices, including backlog refinement ...

Limited (DDIL) communications environments, supporting local processing at the edge Building streaming ingestion, event pipelines, and batch/stream processing workloads using technologies like Apache Flink and/or Apache Spark Implementing metadata, lineage, and catalogue capabilities along with policy-based security and governance Job Requirements: Extensive ...

large-scale analytics platforms Expert knowledge of Quantexa, Entity Resolution (ER), Scoring Framework, Contextual Network Generation Expertise in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch Strong stakeholder management and communication skills - capable of interfacing with both business and technical audiences. The work is heavily weighted toward ...

ongoing basis. Contribute to the planning, research, recommendation, and implementation of current and future IT strategies. PERSON SPECIFICATION: Essential Demonstrable experience of Spark (PySpark) and SQL. Demonstrable experience of Fabric components. Demonstrable experience of working on enterprise environments. Experience of designing architectural Fabric processes. Experience of Azure Data Factory. ...

loop, calibration, guardrails, auditability) Experience with hyperparameter tuning at scale and experimentation frameworks Strong technical experience with SQL and Python; significant experience with Spark for large-scale data processing Excellent interpersonal skills with demonstrable collaborative leadership capabilities (coaching, mentoring, guiding, delivery ownership and stakeholder engagement) and strong standards ...

with large-scale and unstructured datasets. Python development experience, including technologies such as PySpark, Pandas and PyArrow. Expertise in distributed data processing frameworks, particularly Apache Spark. Experience building and maintaining ETL pipelines using technologies such as Apache Airflow, AWS Step Functions or Apache NiFi. Cloud platform experience ...

Swindon (we could consider twice per fortnight etc). Required Skills & Experience Strong experience testing data pipelines/ETL processes Hands-on experience with Apache Spark Experience working with Databricks and/or AWS Glue Proven ability to build automation or testing frameworks (Python preferred) Experience with large ...

work on a Big Data Project in the finance sector. Required Experience: Java Big Data experience On-premises cluster experience Databases Kubernetes Kafka Apache Spark Preffered Experience: Finance sector experience ...

work on a Big Data Project in the finance sector. Required Experience: Java Big Data experience On-premises cluster experience Databases Kubernetes Kafka Apache Spark Preffered Experience: Finance sector experience ...

with stakeholders. Preferred qualifications, capabilities, and skills Experience designing/implementing pipelines using DAGs (e.g. Kubeflow, DVC, Ray) Experience of big data technologies (e.g. Spark, Hadoop) Have constructed batch and streaming microservices exposed as REST/gRPC endpoints Familiarity with GraphQL Level of experience Senior (5+ years of experience ...

Analytics Engineer London, hybrid 3 times a week Department: IT Type: 3-month rolling contract Outside IR35 Position: Analytics Engineer The organisation is seeking an experienced Analytics Engineer to join its Data & Analytics team. The ...

engineer first. Deep proficiency in Python/C++, experience with low-latency inference systems, and familiarity with distributed computing frameworks (Ray, Spark, Flink). You write code that survives in production, not just notebooks. The Full ML Lifecycle: Expertise in end-to‐end system design: from feature stores ...

insights that drive the AI product roadmap forward. Tech Stack Includes : AWS: EC2, Athena, S3, Kinesis, Lambda, Step Functions, RDS, QuickSight Languages & Frameworks: Python, Spark, TensorFlow, LLMs & LLM coding tools Databases & Search: MySQL, PostgreSQL, SOLR Requirements Experience: Minimum 3 years of post-degree commercial experience in a data science ...

delivery of secure, scalable and high-performance data solutions within a regulated environment. This role will focus heavily on the Elastic Stack and Apache NiFi, supporting enterprise-scale data ingestion, transformation and analytics capabilities across complex secure systems. The successful candidate will play a key role in architecting resilient … compliance with strict governance and security standards. KEY RESPONSIBILITIES Data Architecture & Engineering - Design and implement scalable, resilient data pipeline architectures using Elastic Stack and Apache NiFi - Develop secure frameworks for data ingestion, transformation and integration - Build and maintain batch and real-time data processing pipelines - Ensure high standards ...

people management experience You have extensive, up‐to‐date knowledge and hands‐on experience with machine learning and data engineering technologies (e.g., Kubernetes, Spark, Databricks, GPU) You have extensive experience in Generative AI technology areas such as search, personalization, sorting, ranking, recommendation, bidding, pricing, etc You have large‐scale ...

platform role Strong hands-on experience with AWS (EC2, S3, RDS, Lambda, Redshift, Glue, Athena) Experience with real-time and batch data processing - Apache Flink, Kafka, Spark, or similar Proficiency in one or more of: Python, Java, Scala Experience with data modelling , ETL processes, and distributed systems Comfortable ...

Deep experience with cloud platforms (Azure, AWS, or GCP) and cloud-native architecture Strong background in data engineering and large-scale data platforms (e.g., Spark, Databricks) Experience implementing MLOps frameworks, including CI/CD and model governance Experience integrating AI into microservices, APIs, and enterprise systems Exposure to advanced … similar programming languages) TensorFlow, PyTorch, or similar ML frameworks Cloud AI services (Azure AI, AWS SageMaker, GCP AI Platform) Data technologies (SQL/NoSQL, Spark, Databricks) API and microservices architecture DevOps/MLOps tooling Desirable Experience Experience with Generative AI solutions (e.g. Azure OpenAI, GPT-based models) Microsoft stack ...

Purpose of position Data sits at the heart of the company. This role ensures that Awin can fully leverage the data available across the group, enabling the development of robust reporting capabilities that support stronger ...

Senior Data Engineer 6 Month contract with extension Full-time Inside IR35 Hybrid 1 day every two weeks in the office Nottingham - NG1 4BX Join our client in embarking on an ambitious data transformation journey ...

skills, including query optimization and schema design. Programming experience in Python or Scala for data processing. Experience with big data processing tools such as Spark or Databricks. Experience with cloud data platforms such as Azure, AWS, or GCP, especially data services. Responsibilities Develop and implement ETL and ELT data … pipelines. Optimize SQL queries and support schema design and data modeling. Use Python or Scala to build and maintain data processing solutions. Work with Spark or Databricks for big data processing. Support data engineering work across cloud data platforms such as Azure, AWS, or GCP. Technologies AWS Azure ...

skills, including query optimization and schema design. Programming experience in Python or Scala for data processing. Experience with big data processing tools such as Spark or Databricks. Experience with cloud data platforms such as Azure, AWS, or GCP, especially data services. Responsibilities Develop and implement ETL and ELT data … pipelines. Optimize SQL queries and support schema design and data modeling. Use Python or Scala to build and maintain data processing solutions. Work with Spark or Databricks for big data processing. Support data engineering work across cloud data platforms such as Azure, AWS, or GCP. Technologies AWS Azure ...

solving complex challenges and improving data quality as the platform scales. The environment is modern and cloud based, with exposure to technologies such as Spark, Snowflake and BigQuery across AWS, GCP and Azure. You will have the opportunity to support and mentor other engineers, contribute to engineering standards … engineers What We Are Looking For Proven Python and SQL experience within production environments Experience delivering data transformation projects end to end Experience with Spark, Snowflake, BigQuery or similar technologies Solid experience working with AWS, GCP or Azure Comfortable working with complex, imperfect datasets Experience owning solutions from design ...

time inference systems with low latency and high throughput Partner with data engineers to integrate ML workflows into wider data platforms and infrastructure, including Spark and Databricks Implement model monitoring, drift detection, alerting, and retraining strategies Optimise models for reliability and cost efficiency in AWS Prototype and evaluate … learning frameworks Experience creating & maintaining reproducible environments and familiarity with tools such as UV/docker Experience with MLflow or equivalent tooling Experience with Spark and distributed data processing Strong understanding of real-time ML systems and production inference patterns A strong engineering mindset, with a focus on reliability ...

Aker Systems was founded in 2017 by a team of experienced technology professionals who recognised an opportunity to provide highly secure enterprise data platforms to large organisations. We build and operate ground-breaking, ultra-secure ...