51 to 63 of 63 Apache Spark Jobs in the UK excluding London

insights that drive the AI product roadmap forward. Tech Stack Includes : AWS: EC2, Athena, S3, Kinesis, Lambda, Step Functions, RDS, QuickSight Languages & Frameworks: Python, Spark, TensorFlow, LLMs & LLM coding tools Databases & Search: MySQL, PostgreSQL, SOLR Requirements Experience: Minimum 3 years of post-degree commercial experience in a data science ...

delivery of secure, scalable and high-performance data solutions within a regulated environment. This role will focus heavily on the Elastic Stack and Apache NiFi, supporting enterprise-scale data ingestion, transformation and analytics capabilities across complex secure systems. The successful candidate will play a key role in architecting resilient … compliance with strict governance and security standards. KEY RESPONSIBILITIES Data Architecture & Engineering - Design and implement scalable, resilient data pipeline architectures using Elastic Stack and Apache NiFi - Develop secure frameworks for data ingestion, transformation and integration - Build and maintain batch and real-time data processing pipelines - Ensure high standards ...

delivery of secure, scalable and high-performance data solutions within a regulated environment. This role will focus heavily on the Elastic Stack and Apache NiFi, supporting enterprise-scale data ingestion, transformation and analytics capabilities across complex secure systems. The successful candidate will play a key role in architecting resilient … compliance with strict governance and security standards. KEY RESPONSIBILITIES Data Architecture & Engineering - Design and implement scalable, resilient data pipeline architectures using Elastic Stack and Apache NiFi - Develop secure frameworks for data ingestion, transformation and integration - Build and maintain batch and real-time data processing pipelines - Ensure high standards ...

degree, or experience in a professional field or military - Experience as technical specialist in design and architecture - Experience in database (eg. SQL, NoSQL, Hadoop, Spark, Kafka, Kinesis) - Experience in consulting, design and implementation of serverless distributed solutions - Experience in cloud based solution (AWS or equivalent), system, network and operating … military settings - Bachelor's degree, or experience as technical specialist in design and architecture - Experience in Database like NoSQL, or experience in Hive/Spark/Hbase/Yarn and experience in any Bigdata architecture - Bachelor's degree in business administration, finance, economics, computer science, data science, engineering ...

platform role Strong hands-on experience with AWS (EC2, S3, RDS, Lambda, Redshift, Glue, Athena) Experience with real-time and batch data processing - Apache Flink, Kafka, Spark, or similar Proficiency in one or more of: Python, Java, Scala Experience with data modelling , ETL processes, and distributed systems Comfortable ...

governed and used across the organisation. Key Responsibilities: Maintain, improve and optimise an existing Databricks data platform. Build and enhance scalable data pipelines using Spark, SQL and Python or Scala. Develop and mature Bronze, Silver and Gold layers to support analytics, reporting and downstream data use. Ingest and integrate … What We're Looking For: Proven experience as a Data Engineer within a cloud-based data environment. Strong hands-on experience with Databricks and Apache Spark. Experience building or improving medallion architecture, including Bronze, Silver and Gold data layers. Strong SQL skills for transformation, validation and data analysis. Python ...

Deep experience with cloud platforms (Azure, AWS, or GCP) and cloud-native architecture Strong background in data engineering and large-scale data platforms (e.g., Spark, Databricks) Experience implementing MLOps frameworks, including CI/CD and model governance Experience integrating AI into microservices, APIs, and enterprise systems Exposure to advanced … similar programming languages) TensorFlow, PyTorch, or similar ML frameworks Cloud AI services (Azure AI, AWS SageMaker, GCP AI Platform) Data technologies (SQL/NoSQL, Spark, Databricks) API and microservices architecture DevOps/MLOps tooling Desirable Experience Experience with Generative AI solutions (e.g. Azure OpenAI, GPT-based models) Microsoft stack ...

need to potentially work in either Edinburgh or Glasgow depending on client expectations. What You’ll Do Design and develop robust pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog Build real-time event-driven solutions with tools such as Kafka and Azure Event Hubs Apply DevOps principles … knowledge of DevOps tools and CI/CD processes Bonus Points For Development experience in Scala or Java Familiarity with Cloudera, Hadoop, HIVE, and Spark ecosystem Understanding of data privacy regulations, including GDPR, and experience working with sensitive data Ability to learn and adapt new technologies quickly to meet ...

that are accurate, maintainable and easy for the business to use. Key Responsibilities: Build, enhance and maintain data pipelines within a Databricks environment. Use Spark, SQL and Python or Scala to support data transformation and automation. Develop structured data layers to support reporting, analytics and wider business use. Extract … What We're Looking For: Experience working as a Data Engineer within a modern cloud data environment. Strong hands-on experience with Databricks and Apache Spark. Experience building structured data layers, ideally within a Bronze, Silver and Gold architecture. Strong SQL skills for transformation, validation and analysis. Python ...

Senior Data Engineer 6 Month contract with extension Full-time Inside IR35 Hybrid 1 day every two weeks in the office Nottingham - NG1 4BX Join our client in embarking on an ambitious data transformation journey ...

teams and setting best practice standards Designing data models, ETL pipelines, warehousing solutions and distributed systems Building and supporting real-time streaming platforms using Apache Flink and related technologies Ensuring compliance with industry standards and government regulations Tracking delivery against agreed designs and managing design deviations Supporting pre-sales … data architecture on complex, secure programmes Strong hands-on AWS experience (EC2, S3, RDS, Lambda, Redshift); Azure or GCP a bonus Proven experience using Apache Flink for real-time, high-throughput data processing Experience leading, mentoring and developing technical teams Confident engaging with senior technical and non-technical stakeholders ...

Aker Systems was founded in 2017 by a team of experienced technology professionals who recognised an opportunity to provide highly secure enterprise data platforms to large organisations. We build and operate ground-breaking, ultra-secure ...

Senior Software Engineer - Real-Time Data (Kafka/Spark) Chester | Hybrid working | 12-month contract | Inside IR35 | Global Investment Bank A leading global investment bank is looking for a hands-on Senior Engineer to build high-performance, real-time data platforms supporting regulatory compliance across global operations. This … processing , ideal for someone who enjoys solving complex data challenges at scale. What You'll Do Design and build Kafka-based streaming pipelines and Spark processing jobs Develop low-latency, high throughput systems handling large-scale datasets Create data matching and reconciliation solutions across multiple sources Work on event ...