1 to 25 of 28 Apache Jobs in London

however Azure exposure is essential. Data Warehousing: Proven expertise with Snowflake – schema design, performance tuning, data ingestion, and security. Workflow Orchestration: Production experience with Apache Airflow (Prefect, Dagster or similar), including authoring DAGs, scheduling workloads and monitoring pipeline execution. Data Modeling: Strong skills in dbt, including writing modular ...

Solid understanding of data warehousing, lakehouse and datalake architectures Familiarity with modern data engineering patterns (ETL/ELT, medallion architecture, observability) Experience working with Apache Spark and large-scale data processing For a full consultation on this exciting new role, please get in touch with ARC IT Recruitment today. ...

lakehouse and datalake architectures Familiarity with modern data engineering patterns (ETL/ELT, medallion architecture, observability) Excellent Python and SQL skills Experience working with Apache Spark and large-scale data processing For a full consultation on this exciting new role, please get in touch with ARC IT Recruitment today. ...

coding skills in Java (Spring Boot) for data engineering and services. Proven background working within financial services environments. Hands-on experience building and managing Apache Airflow/Spark and Hadoop clusters. Deep experience in big data engineering, including Hadoop, Kafka, and Spark-based data pipelines. Strong SQL/RDBMS ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

platform, features, and best practices with Strong understanding of data modeling principles and techniques A solid understanding of big data technologies such as Apache Spark, and knowledge of Hadoop ecosystems. Knowledge of programming languages such as Python, R, or Java is beneficial. Knowledge of ETL/ELT processes (such ...

Knowledge of machine learning frameworks (e.g., PySpark, PyTorch) and model deployment tools (e.g., MLflow, TensorFlow Serving). Strong experience with data processing frameworks (e.g., Apache Spark, Flink). Expertise in SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB, Cassandra). Hands-on experience with cloud platforms (e.g. ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

data stack (Glue, OpenSearch, RDS) Expert skills within SQL with experience in both transactional RDBMS systems and distributed systems Hands-on with Lakehouse technologies (Apache Iceberg, S3 Tables, StarRocks) Strong grasp of data governance, schema design, and quality frameworks Comfortable leading infrastructure decisions and collaborating across distributed teams This ...

Desired Skills (Bonus Points) Experience with MLOps tooling (MLflow, Kubeflow, Weights & Biases, or similar) Building data pipelines (real-time or batch) using tools like Apache Spark, Kafka, Airflow, or dbt Familiarity with recommender systems, transformers, or graph neural networks Exposure to model optimisation techniques (quantisation, distillation, efficient inference) What ...

Platform Migration projects and delivering secure, scalable cloud solutions Nice to have: Understanding of Synapse Spark Architecture Familiarity with a variety of technologies including Apache Spark, Spark ML, Serverless Spark Pools, T-SQL, ETL Immediate availability Hays Specialist Recruitment Limited acts as an employment agency for permanent recruitment ...

Data Platform Engineer – London (AWS, Apache Spark, AWS Glue, Iceberg, S3, RDS, Redshift, Kafka/MSK, Python, Terraform, Ansible, CI/CD, Jenkins, GitLab, Snowflake, Databricks) Working with an established FinTech client in London who is looking for a Data Platform Engineer to play a key role in defining ...

data stack (Glue, OpenSearch, RDS) Expert skills within SQL with experience in both transactional RDBMS systems and distributed systems Hands-on with Lakehouse technologies (Apache Iceberg, S3 Tables, StarRocks) Strong grasp of data governance, schema design, and quality frameworks Comfortable leading infrastructure decisions and collaborating across distributed teams This ...

applications and high proficiency in SQL (query tuning, complex joins) Hands-on experience designing and deploying ETL/ELT pipelines using Google Cloud Dataflow (Apache Beam) or similar tools Proficiency in data architecture, data modeling, and scalable storage design Solid engineering practices: Git and CI/CD for data ...

architecture, and network stack optimization. Data Infrastructure: Familiarity with the modern data stack, including time-series databases, object stores, and streaming frameworks like Apache Flink or Spark Streaming . Engineering Rigor: You write clean, testable, and reliable code. You understand that in this environment, a system failure can cost ...

NICE FOR THE SENIOR SOFTWARE ENGINEER TO HAVE. Cloud based experience Microservice architecture or server-less architecture Big Data/Messaging technologies such as Apache Nifi/MiNiFi/Kafka TO BE CONSIDERED. Please either apply by clicking online or emailing me directly to For further information please call ...

services that filter and protect content across Who You Are * Expertise: Deep knowledge of Java and Python . The Stack: Experience with Ray , Apache Beam/Scio , dbt , and BigQuery . Mindset: Comfortable leading in ambiguous problem spaces and delivering end-to-end solutions at scale. Why Join? Work ...

Data Analytics Sales Executive - Data Sales, Talend, Snowflake, Apache Spark, Kafka, Dataops, AWS, Azure, Data visualisation, SQL, ETL, Pre-Sales, Account Manager, SaaS Sales, Cloud Solutions, Digital Transformation, Business Development, Sales Consultant, CXO Engagement, Strategic Accounts, Deal Negotiation, Remote Working, UK-wide, Permanent Role, Sales Targets, New Business, Sales ...

environment 2+ years’ experience working with Red Hat Linux (or other Unix variants), including scripting 2+ years’ experience supporting applications such as Java, .NET, Apache or IIS Desirable: Experience within the financial services or trading environment Knowledge of Windows Server Familiarity with ITIL frameworks Exposure to market data ...

/financial services. Strong Payments and Ledger domain experience within UK Finance . Deep expertise in Streaming Data Technologies , including: Confluent Kafka Apache Flink Java ODS/MongoDB Proven experience with high-volume, low-latency payment systems . Excellent senior stakeholder management and communication skills . Experience working ...

experience writing high-quality production code Strong foundation in operating systems and systems programming Proven track record in performance engineering Familiarity with tools like Apache Arrow, Parquet, DataFusion, Clickhouse, or DuckDB is a plus Understanding of cutting-edge ML infrastructure stack (e.g. PyTorch, CUDA) is also a plus Experience ...

Administration including installation and configuration of servers and databases, server log monitoring and web server hardening. Strong knowledge of Red Hat Linux. Proficiency with Apache and PHP installation/upgrades. Familiarity with content management systems (e.g., WordPress). Proven leadership skills and line-management experience. Excellent communication skills with ...

schema). Version Control Practical experience with Git (branching, merging, pull requests). Preferred Qualifications (A Plus) Experience with a distributed computing framework like Apache Spark (using PySpark). Familiarity with cloud data services ( AWS S3/Redshift, Azure Data Lake/Synapse, or Google BigQuery/Cloud Storage … . Exposure to workflow orchestration tools ( Apache Airflow, Prefect, or Dagster ). Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field. ...

/ELT) to support analytics and downstream use cases Developing scalable data solutions using Python , with a focus on maintainability and performance Working with Apache Spark/PySpark to process and transform large datasets Supporting the ingestion, transformation and validation of complex financial data Improving the performance, reliability … That Will Help You Succeed Essential Commercial experience as a Data Engineer at a mid‐level Strong Python development skills Hands‐on experience with Apache Spark/PySpark Solid experience building ETL/ELT pipelines Background within the financial services industry (investment management experience desirable) Comfortable working with production ...