51 to 75 of 81 Apache Spark Jobs in the UK

environments Strong experience designing large-scale, cloud-based data platforms Deep expertise in AWS services, including EC2, S3, RDS, Lambda, Redshift Proven experience with Apache Flink and real-time streaming architectures Strong background in: Data modelling ETL/ELT processes Data warehousing and distributed systems Experience leading, mentoring …/security-focused environments Strong understanding of data governance, standards, and architecture frameworks Desirable Skills Experience with Azure or GCP Knowledge of Scala and Apache Spark Familiarity with DevOps practices and cloud-native delivery models Experience working alongside cyber security teams on secure systems Security Requirements Must currently ...

large-scale analytics platforms Expert knowledge of Quantexa, Entity Resolution (ER), Scoring Framework, Contextual Network Generation Expertise in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch Strong stakeholder management and communication skills - capable of interfacing with both business and technical audiences. The work is heavily weighted toward ...

Core Java Spring Boot frameworks Proven experience building distributed backend systems Hands-on experience with Kafka (or similar streaming platforms) Experience with Scala and Apache Spark for data processing Integration withbackend services such as: PostgreSQL Redis S3 (or equivalent object storage) Expertise in modern engineering practices, including: Continuous ...

unstructured datasets. Strong stakeholder management and communication skills. Desirable Insurance or financial services experience, ideally within pricing, underwriting, fraud, or claims. Experience with Spark, Hadoop, NoSQL, or modern big data platforms. Agile delivery experience and collaborative software development practices. Experience leading analytics teams and driving organisational change. Robert Walters ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation Knowledge of statistical methods and experimentation (A/B testing) preferred ...

Experience with: Relational databases (SQL Server, PostgreSQL, Oracle, MySQL) Data visualisation tools (Power BI, Tableau, Spotfire) Cloud platforms, particularly Microsoft Azure Big data technologies (Spark, Elasticsearch, Hadoop) Statistical modelling and advanced analytics Machine learning and pattern recognition techniques Web technologies such as HTML and JavaScript ...

maintain operational continuity and quality output Knowledge of data processing frameworks, including ETL pipelines, data integration, and data modeling (experience with Databricks or Spark preferred) Strong analytical and problem-solving skills with the ability to communicate clearly to both technical and non-technical stakeholders Basic understanding of artificial intelligence ...

operations. Demonstrated depth in Big Data engineering, with proficiency in at least one of the following areas: distributed data processing platforms such as Databricks, Spark, or EMR search platforms such as OpenSearch or Elasticsearch graph databases such as Neo4j Strong understanding of Agile and Lean practices, including backlog refinement ...

Limited (DDIL) communications environments, supporting local processing at the edge Building streaming ingestion, event pipelines, and batch/stream processing workloads using technologies like Apache Flink and/or Apache Spark Implementing metadata, lineage, and catalogue capabilities along with policy-based security and governance Job Requirements: Extensive ...

large-scale analytics platforms Expert knowledge of Quantexa, Entity Resolution (ER), Scoring Framework, Contextual Network Generation Expertise in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch Strong stakeholder management and communication skills - capable of interfacing with both business and technical audiences. The work is heavily weighted toward ...

ongoing basis. Contribute to the planning, research, recommendation, and implementation of current and future IT strategies. PERSON SPECIFICATION: Essential Demonstrable experience of Spark (PySpark) and SQL. Demonstrable experience of Fabric components. Demonstrable experience of working on enterprise environments. Experience of designing architectural Fabric processes. Experience of Azure Data Factory. ...

loop, calibration, guardrails, auditability) Experience with hyperparameter tuning at scale and experimentation frameworks Strong technical experience with SQL and Python; significant experience with Spark for large-scale data processing Excellent interpersonal skills with demonstrable collaborative leadership capabilities (coaching, mentoring, guiding, delivery ownership and stakeholder engagement) and strong standards ...

Swindon (we could consider twice per fortnight etc). Required Skills & Experience Strong experience testing data pipelines/ETL processes Hands-on experience with Apache Spark Experience working with Databricks and/or AWS Glue Proven ability to build automation or testing frameworks (Python preferred) Experience with large ...

work on a Big Data Project in the finance sector. Required Experience: Java Big Data experience On-premises cluster experience Databases Kubernetes Kafka Apache Spark Preffered Experience: Finance sector experience ...

work on a Big Data Project in the finance sector. Required Experience: Java Big Data experience On-premises cluster experience Databases Kubernetes Kafka Apache Spark Preffered Experience: Finance sector experience ...

Analytics Engineer London, hybrid 3 times a week Department: IT Type: 3-month rolling contract Outside IR35 Position: Analytics Engineer The organisation is seeking an experienced Analytics Engineer to join its Data & Analytics team. The ...

tooling and working with large transactional datasets • Understanding of OTC products including CDS, Interest Rate Swaps and Variance Swaps Highly Desirable • Experience with C++, Spark, Kafka or other distributed compute tools • Exposure to position keeping, risk, or PnL systems • Strong debugging, profiling and optimisation capability across data pipelines ...

/automation skills (e.g., Python, Bash, or Go). Data Platform Experience: Prior exposure to data platforms, batch/streaming data pipelines (e.g., Kafka, Spark), and the unique challenges of data observability and workload reliability. Active or recent SC Clearance. £300.00 - £375.00/day Talent International ...

insights that drive the AI product roadmap forward. Tech Stack Includes : AWS: EC2, Athena, S3, Kinesis, Lambda, Step Functions, RDS, QuickSight Languages & Frameworks: Python, Spark, TensorFlow, LLMs & LLM coding tools Databases & Search: MySQL, PostgreSQL, SOLR Requirements Experience: Minimum 3 years of post-degree commercial experience in a data science ...

delivery of secure, scalable and high-performance data solutions within a regulated environment. This role will focus heavily on the Elastic Stack and Apache NiFi, supporting enterprise-scale data ingestion, transformation and analytics capabilities across complex secure systems. The successful candidate will play a key role in architecting resilient … compliance with strict governance and security standards. KEY RESPONSIBILITIES Data Architecture & Engineering - Design and implement scalable, resilient data pipeline architectures using Elastic Stack and Apache NiFi - Develop secure frameworks for data ingestion, transformation and integration - Build and maintain batch and real-time data processing pipelines - Ensure high standards ...

Data Engineer Salary up to £70,000 + discretionary bonus Hybrid working model with collaborative office environment Work with Databricks, Spark Structured Streaming, Kafka (MSK), and AWS Apply online or contact Chelsea Hackett via ABOUT THE CLIENT Due to continued growth, we're seeking a skilled Data Engineer ...

people management experience You have extensive, up‐to‐date knowledge and hands‐on experience with machine learning and data engineering technologies (e.g., Kubernetes, Spark, Databricks, GPU) You have extensive experience in Generative AI technology areas such as search, personalization, sorting, ranking, recommendation, bidding, pricing, etc You have large‐scale ...

platform role Strong hands-on experience with AWS (EC2, S3, RDS, Lambda, Redshift, Glue, Athena) Experience with real-time and batch data processing - Apache Flink, Kafka, Spark, or similar Proficiency in one or more of: Python, Java, Scala Experience with data modelling , ETL processes, and distributed systems Comfortable ...

Deep experience with cloud platforms (Azure, AWS, or GCP) and cloud-native architecture Strong background in data engineering and large-scale data platforms (e.g., Spark, Databricks) Experience implementing MLOps frameworks, including CI/CD and model governance Experience integrating AI into microservices, APIs, and enterprise systems Exposure to advanced … similar programming languages) TensorFlow, PyTorch, or similar ML frameworks Cloud AI services (Azure AI, AWS SageMaker, GCP AI Platform) Data technologies (SQL/NoSQL, Spark, Databricks) API and microservices architecture DevOps/MLOps tooling Desirable Experience Experience with Generative AI solutions (e.g. Azure OpenAI, GPT-based models) Microsoft stack ...

Purpose of position Data sits at the heart of the company. This role ensures that Awin can fully leverage the data available across the group, enabling the development of robust reporting capabilities that support stronger ...