101 to 125 of 257 Apache Jobs in London

technical solutions.Tech StackDevelopment: TypeScript, Angular/React, Python, SQL, OpenAPI.Cloud: AWS (Lambda, API Gateway, S3, SQS, SNS, Aurora, and others as required).Databases: PostgreSQL, Apache Iceberg, DynamoDB, VectorDB (e.g. ChromaDB).IaC and CI/CD: Terraform, AWS CLI, GitLab.AI Development: Agentic AI tooling (e.g. GitHub Copilot, Claude Code).WHAT ...

decisions that matter: architecture, system design, and the tradeoffs that determine whether a platform scales gracefullyNice-to-haveLakehouse and streaming technologies — experience with Databricks, Apache Iceberg, Kafka, or similar is a strong advantageWhat we offerDiverse and internationally distributed team: joining our team means becoming part of a large, global ...

environments. Technical Skills: Experience building and operating ETL/ELT pipelines. Hands-on experience with modern data stacks — our tech includes Python, SQL, Snowflake, Apache Iceberg, AWS S3, PostgresDB, Airflow, dbt, and Apache Spark, deployed via AWS, Docker, and Terraform (experience with some of these or similar technologies ...

Linux operating systems, including command-line administration and troubleshooting. Solid experience with SQL or data pipeline technologies, Experience with data processing frameworks such as Apache Airflow, Apache Flink or Google Dataflow, with a practical understanding of ETL pipeline design and the ability to diagnose issues across data transformation …/IP networking principles, including the ability to diagnose connectivity issues and interpret network traffic. Familiarity with event streaming platforms such as Apache Kafka, including an understanding of producer/consumer patterns and how message queues are used to move data reliably between systems at scale. Familiarity with infrastructure ...

stakeholdersWillingness to pick up and learn new technologies and frameworksNice to have:Knowledge of databases, SQLFamiliarity with Boost ASIOFamiliarity with data serialization formats such Apache Arrow/Parquet, Google Protocol Buffers, FlatbuffersExperience with gRPC, http/REST and Websocket protocolsExperience with Google Cloud/AWS and/or containerization ...

Spring Boot, REST APIs and microservices Experience with a relational database like MS SQL Server, PostgreSQL, Sybase ASE Experience with Streaming technologies like Kafka, Apache Flink Experience with cloud technologies, such as Microsoft Azure Experience with Agile process Experience with DevOps and tools like Azure DevOps Our benefits ...

support data pipelines, ensuring they are production-ready and meet data quality standards. You will work with technologies such as Python, SQL, and Apache Spark, helping drive analytics and operational decision-making. Ideal candidates possess strong programming skills, a relevant degree, and a collaborative spirit, as you will partner ...

models. Your Skills & Experience Strong experience with SQL and modern data engineering practices. Hands-on experience with Snowflake and dbt. Strong experience working with Apache Airflow for data orchestration. Experience working with AWS data services and cloud-based data platforms. Strong Python skills for data engineering and pipeline development. ...

MongoDB. And work with PostgreSQL too. The Senior Data Engineer will be well versed in Kafka. The Senior Data Engineer will know all about Apache Spark, Airflow and Iceberg. The Senior Data Engineer will have solid cloud infrastructure knowledge e.g. Google Cloud (GCP). The Senior Data Engineer will ...

preparation. Machine learning and statistical methods, including model validation. Data analytics and visualisation techniques. Processing large datasets using batch or streaming tools such as Apache Spark. Data acquisition and fusion approaches. Cloud platforms (ideally AWS) and cloud-based data science solutions. DataOps principles and practices. Structured and unstructured databases. ...

warehousing architecturesExpopsure to data governance frameworks, metadata management, and regulatory compliancIt would be a bonus if you have:Familiarity with emerging technologies such as Apache Iceberg, Delta Lake, real-time/streaming architectures, or data observability toolingExperience leading architecture workshops with both technical and non-technical stakeholdersA background providing ...

years' experience delivering enterprise data platforms or analytics solutions. Strong commercial experience delivering Databricks solutions. Consulting or customer-facing delivery experience. Strong understanding of Apache Spark, modern Lakehouse architecture and Unity Catalog. Experience with Azure, AWS or GCP. Experience implementing CI/CD and deployment automation. Excellent communication ...

deployment. Expert-level Python and its data ecosystem (Numpy, Pandas), including designing frameworks for data tasks. Deep understanding of distributed data processing engines like Apache Spark. Ability to reason about the full data lifecycle from ingestion through to real-time serving - your system designs will need to meet downstream ...

blocking framework (Vert.x, SpringWebFlux, or equivalent) and asynchronous, event-driven service design. Deep experience with stream processing and event-driven architecture - Kafka Streams, Apache Flink, or equivalent - including stateful streaming, topology design, and managing local state stores (e.g.RocksDB) at scale. Proventrack recordbuilding high-throughput, low-latency systems where tail ...

Catalog, Spark, Structured Streaming, and Lakehouse architectureStrong hands-on experience with Snowflake, including data modelling, performance optimisation, Snowpark, data sharing, and governance capabilitiesExpertise in Apache Spark, Python, SQL, and modern ELT/ETL frameworksProven ability to rapidly evaluate, prototype, and compare emerging technologies and architectural patternsExperience designing large-scale … market microstructureExperience with AI/ML platforms, feature stores, and MLOps frameworksExperience evaluating alternative analytical databases such as DuckDB, ClickHouse, Redshift, BigQuery, KDB+, or Apache Iceberg-based architecturesExperience with data catalogue and governance platformsExposure to modern visualisation and BI tools such as Power BI, Tableau, Sigma, or ThoughtSpotExperience leading ...

services for personalisation · Developing Java/Spring Boot microservices with REST/gRPC APIs · Working with Kafka and event-driven architectures · Building pipelines using Apache Flink and Apache Beam · Integrating ML models and feature stores · Working with NoSQL databases and Redis caching · Using Apache Camel for integration … with stream processing concepts · Knowledge of distributed systems · Strong communication and teamwork skills · Interest in data-driven personalisation systems Nice to have · Experience with Apache Flink and Apache Beam · Experience with Kafka · Familiarity with Apache Camel · Experience with NoSQL databases (Cassandra, DynamoDB) and Redis · Cloud experience ...

Document stores).Hands-on experience building, scaling, and shipping large-scale data platforms in production.Deep practical experience with distributed data processing frameworks (e.g., Apache Spark, Flink, Hadoop).Strong expertise in message brokers and event streaming platforms (e.g., Apache Kafka, Kinesis).End-to-end exposure to data pipeline lifecycle … development, including extensive experience with workflow orchestration tools (e.g., Apache Airflow, Dagster).Hands-on expertise with cloud data warehouses (e.g., Snowflake, BigQuery, Redshift) and data lake architectures (e.g., Databricks, Delta Lake, Apache Iceberg).Advanced SQL skills and proficiency in Python. Strong background in modern software development practices (testing ...

replication tool, and Waggle Dance, a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg, a high-performance open table format for large analytic datasets that underpins many modern Lakehouse architectures.We guarantee you’ll learn ...

replication tool, and Waggle Dance, a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg, a high-performance open table format for large analytic datasets that underpins many modern Lakehouse architectures. We guarantee you’ll learn ...

software engineers and product managers to deliver robust data solutions. The role requires 7+ years in data engineering, including leadership, and strong expertise in Apache Spark and Azure Data Factory, with hands-on cloud experience across Azure/AWS/GCP. #J-18808-Ljbffr ...

Hybrid : London or Manchester - 2 days per week. This role requires the ability to hold an active security clearance. Skills : Python and SQL . Apache Airflow for DAG orchestration and monitoring. AWS data services: Redshift , OpenSearch , Lambda , Glue , Step Functions , Batch . CI/CD pipelines and YAML-based ...

technical teams. Nice to Have Knowledge or experience within financial markets/securities.Experience working with GCP BigQuery.Experience with GCP Cloud SQL.Experience with Terraform.Experience with Apache Airflow.Understanding of security reference data, pricing data or corporate actions. The Project The wider project is focused on creating a multi-asset Security Master ...

consumers Actively uses AI-powered tools to move faster and focus on architecture, system design, and scalability tradeoffs Nice to have: experience with Databricks, Apache Iceberg, Kafka, or similar lakehouse and streaming technologies Условия Hybrid work schedule with office attendance twice a week and flexible working hours Virtual Shares ...

latency software systems Experience with pipeline orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelor or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experience For more information about ...

GitGood understanding of data management and data quality principles, with experience developing data quality checks or reportsPrior experience using the Bloomberg Terminal Experience with Apache Airflow or similar toolsFor more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice ...