14 of 14 Apache Jobs in the City of London

principles. Comfortable working within Agile delivery environments. Experience with CI/CD pipelines, ideally using Azure DevOps or similar. Nice to Have Experience with Apache Airflow or dbt. Exposure to BigQuery/GCP. Azure or other cloud platform certifications Knowledge of data governance and data management frameworks Ideal experience ...

able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop. Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create data ...

skills. Ability to think creatively and propose innovative solutions. To qualify for this role, you must have: Proven experience and knowledge with PySpark and Apache Spark including the fundamentals of how it works. Experience with cloud platforms (e.g., AWS, Azure, GCP) and data lake architectures. Strong experience and programming ...

Skills MUST HAVE: Strong hands‐on experience with Quantexa implementation and development. Proven experience building and supporting big data platforms . Advanced knowledge of Apache Spark/PySpark . Strong programming skills in Scala and/or Python . Expertise in SQL and large-scale data modelling. Strong cloud ...

also be beneficial, particularly where these have been used to automate migration, transformation, validation or data quality processes. Experience with technologies such as Databricks, Apache Spark, PySpark, Azure, AWS, GCP, data warehouses, data lakes, ETL/ELT platforms, dbt, Airflow, Git, CI/CD and APIs would be advantageous ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and "infrastructure as a code" tools ...

blocking framework (Vert.x, SpringWebFlux, or equivalent) and asynchronous, event-driven service design. Deep experience with stream processing and event-driven architecture - Kafka Streams, Apache Flink, or equivalent - including stateful streaming, topology design, and managing local state stores (e.g.RocksDB) at scale. Proventrack recordbuilding high-throughput, low-latency systems where tail ...

latency software systems Experience with pipeline orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelor or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experience For more information about ...

latency software systems Experience with pipeline orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experience For more information about ...

leading Data Management consultancy in London seeks a skilled professional experienced in PySpark and Apache Spark for a hands-on role. You will work closely with clients and play a crucial part in developing innovative solutions using cloud technologies like AWS, Azure, and GCP. Excellent communication and ability ...

Mockito Experience with clean architecture, DI, modular design Comfortable owning CI/CD, code quality, observability Familiarity with Docker, Maven, Jenkins ⭐ Nice to Have Apache Calcite Starburst or federated query engines JVM performance tuning High-throughput service interfaces (REST/gRPC) Data governance or policy enforcement layers 👥 This Role ...

driven user experience. Hybrid role with three on-site days in City of London. Responsibilities include building scalable data pipelines (BigQuery, Dataflow/Apache Beam, Airflow), ensuring data quality and observability (Looker, Monte Carlo), and collaborating with product engineering and data science teams to plan data tracking and ingestion ...

architecture, acting as the design authority to ensure secure, scalable, auditable solutions across classified environments. The position requires extensive experience with AWS data platforms, Apache NiFi, API #J-18808-Ljbffr ...

bespoke integration approaches for source-system onboarding. Define conceptual, logical and physical data models, including data flows, data domains and data ownership considerations. Specify Apache NiFi architecture patterns, flow standards, interface contracts and reusable ingestion components. Define governance, metadata, lineage, auditability, security monitoring and data quality requirements. Produce High … lake architectures on AWS. Strong knowledge of AWS security, IAM, networking, storage and data platform services relevant to secure data engineering. Extensive experience with Apache NiFi architecture, flow design, governance and operational considerations. Experience designing integrations using REST APIs, SOAP services, file transfers, batch ingestion, legacy adapters and bespoke ...