1 to 25 of 49 Apache Jobs in London

possible! Diverse training opportunities and social benefits (e.g. UK pension schema) Strong hands-on experience working with modern Big Data technologies such as Apache Spark, Trino, Apache Kafka, Apache Hadoop, Apache HBase, Apache Nifi, Apache Airflow, Opensearch Proficiency in cloud-native technologies such ...

data engineering tasks. Experience with cloud platforms such as: AWS, Azure, Google Cloud Platform (GCP) Hands-on experience with data technologies such as: Apache Spark, Databricks, Kafka, Airflow, Snowflake, Hadoop ecosystems Experience designing and maintaining data warehouses and data lakes. Knowledge of DevOps and CI/CD tooling (e.g. ...

maintain scalable and robust data pipelines using SQL and Python, and leveraging any of the following: Databricks, Snowflake, Azure Data Factory, AWS Glue, Apache Airflow and Pyspark. Lead the integration of complex data systems and ensure consistency and accuracy of data across multiple platforms. Implement continuous integration and continuous ...

Collaborate with senior developers and quantitative teams on new analytics capabilities Participate in debugging, testing, and production support activities Technology Environment Python Databricks/Apache Spark SQL Pandas/NumPy Git REST APIs Cloud technologies (Azure or AWS) Numerix (or equivalent pricing/risk platform) Jupyter/notebook-based ...

clear, structured data engineering framework Build, enhance, and maintain data pipelines within Snowflake Develop transformations and data models using DBT Orchestrate workflows using Apache Airflow Write clean, modular Python code for data engineering solutions Improve pipeline reliability, performance, and observability Introduce and embed CI/CD, testing, and deployment ...

role Strong hands-on experience with Snowflake data platform Proficiency in SQL and data modeling techniques Experience with ETL/ELT tools (e.g., dbt, Apache Airflow, Talend, Informatica) Experience with cloud platforms (AWS, Azure, or GCP) Familiarity with data warehousing concepts and best practices Understanding of data governance ...

understand where traditional patterns remain appropriate - and where modern approaches add measurable value. Data Engineering & Platform Understanding: Strong knowledge of distributed processing (e.g. Apache Spark). Familiarity with Hadoop ecosystem components. Working knowledge of SQL and NoSQL technologies. Understanding of ETL/ELT design trade-offs. Awareness of DevOps ...

strong focus on scalability, resilience, and performance. Develop and optimise scalable data pipelines , supporting both batch and streaming workloads, using technologies such as Apache Spark . Work extensively with data technologies , leveraging Python and SQL to deliver high-quality analytical and data-driven solutions. Lead the design and delivery ...

networking, filesystem concepts Configuration management experience with Salt, Ansible, Chef, or similar tools Container and orchestration expertise — Docker, Kubernetes, or related technologies Workflow orchestration — Apache Airflow or similar pipeline frameworks Database design skills — schema design and query optimization in PostgreSQL or similar RDBMS Distributed systems knowledge — you think about ...

with a backend or software engineering background Desirable: Experience with Snowflake Orchestration tools such as Dagster or Airflow dbt for data transformation and modelling Apache Iceberg or modern table formats Background in high-growth, consumer-facing products SALARY AND BENEFITS Base salary up to £100,000 Competitive benefits package ...

Skills (Bonus Points) Experience with MLOps tooling (MLflow, Kubeflow, Weights & Biases, or similar). Building data pipelines (real‐time or batch) using tools like Apache Spark, Kafka, Airflow, or dbt. Familiarity with recommender systems, transformers, or graph neural networks. Exposure to model optimisation techniques (quantisation, distillation, efficient inference). ...

Deliver changes through controlled environments in line with enterprise change and release processes Required Technical Experience Core Technologies Strong hands‐on development experience with Apache Spark Proven experience building Kafka‐based event streaming systems Deep familiarity with Hadoop ecosystems , including Hive Advanced SQL for complex, large‐scale datasets Engineering ...

prompt engineering skills and the ability to critically evaluate AI-generated code for correctness, security, and financial accuracy Desirable, but no essential: Experience with Apache Airflow or similar workflow schedulers REST or GraphQL API design experience Knowledge of fixed income, FX, or derivatives products and associated P&L/ ...

prompt engineering skills and the ability to critically evaluate AI-generated code for correctness, security, and financial accuracy Desirable, but no essential: Experience with Apache Airflow or similar workflow schedulers REST or GraphQL API design experience Knowledge of fixed income, FX, or derivatives products and associated P&L/ ...

Experience with data governance frameworks, metadata management, and regulatory compliance It would be a bonus if you have: Familiarity with emerging technologies such as Apache Iceberg, Delta Lake, or data observability tooling Experience leading architecture workshops with both technical and non-technical stakeholders A background providing technical mentorship ...

internal knowledge Competencies and Behaviours: 3+ years experience in data platform engineering, cloud engineering, or similar roles Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/ ...

Azure Key Vault, Azure Monitor, Azure Log Analytics, Azure API Management (APIM), Azure DevOps. Databases : SQL Server, Oracle, PostgreSQL, Delta Lake Big Data : Apache Spark Version Control : GitHub, Git, Azure DevOps Visualization : Power BI & Integration with REST APIs for custom dashboards. Data Integration & Workflow Orchestration : Azure Data Factory, Databricks ...

with Python-based development workflows and tooling Technology Stack: Azure Cloud Services C# (.NET 10) Python React.js and modern JavaScript (ES6+) Snowflake (Data Warehouse) Apache Airflow Terraform dbt SQL Server ...

with Python-based development workflows and tooling Technology Stack: Azure Cloud Services C# (.NET 10) Python React.js and modern JavaScript (ES6+) Snowflake (Data Warehouse) Apache Airflow Terraform dbt SQL Server ...

ideal candidate will have a strong programming background in Python or Java, knowledge of SQL, and experience with data processing frameworks such as Apache Spark. Responsibilities include enhancing data pipelines, ensuring data quality, and collaborating with engineering teams. The position is located in London and offers significant growth opportunities ...

support data pipelines, ensuring they are production-ready and meet data quality standards. You will work with technologies such as Python, SQL, and Apache Spark, helping drive analytics and operational decision-making. Ideal candidates possess strong programming skills, a relevant degree, and a collaborative spirit, as you will partner ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

clouds (AWS, Azure, GCP) and cloud data warehousing solutions like Snowflake and Databricks. Certifications with each are a plus. Understanding of SQL, Python, R, Apache Spark or similar product or language are also a plus.* Strong knowledge of data analytics, data science, ETL processes, AI/ML workflows, automation ...

execute. You have deep experience designing, building, and operating production data pipelines, including owning reliability, monitoring, and failure recovery for orchestration systems such as Apache Airflow (or equivalent). You are fluent in modern data and cloud ecosystems, with hands-on experience (or the ability to ramp quickly … days a week in office Lean principles and Agile development practices Continuous deployment across all microservices Big Data ecosystem and SQL Databases (Apache Airflow, Iceberg, Spark/Thrift, PostgreSQL, Azure Hyperscale) Python as a primary backend language High-level frameworks (Flask, SQLAlchemy, Alembic, Pytest, Socket.IO) Amazon Web Services ...

NICE FOR THE SENIOR SOFTWARE ENGINEER TO HAVE.... Cloud based experience Microservice architecture or server-less architecture Big Data/Messaging technologies such as Apache Nifi/MiNiFi/Kafka TO BE CONSIDERED.... Please either apply by clicking online or emailing me directly to For further information please call ...