Senior Real-Time Observability Engineer (eBPF / Market Data)

What makes this role unusual:

This team tracks individual trades across our network in real time, using eBPF and kernel-level instrumentation to capture transaction data on the wire and correlate it into a per-trade observability stream. That means measuring the SLOs we promise our clients, hunting down latency outliers and orphaned trades, and — yes, really — up to potentially flagging cases where a trade has been altered in flight by a third party. Few observability roles get to operate this deep in the stack on data that matters this much.

Day to day, you'll be writing the capture and correlation code, extending the telemetry pipelines that feed it, and partnering with service squads to instrument their applications across Dev, Test, Pre-Production, and Production environments.

Role overview:

As a Real-Time Observability Engineer, you will be building the application layer that combines single-application observability data into end-to-end insight for this Real-Time market data platform. The role applies technical knowledge to design, build, and improve the software and infrastructure that makes trading data and feeds measurable across all instruments traded in 34 markets.

Key Responsibilities:

  • Build and maintain the application components that aggregate, correlate, and present observability data from individual services into unified, end-to-end views of the Real-Time platform.
  • Implement customer-centric aggregation and hotspot detection algorithms that reflect timeliness, completeness, accuracy, and stability of market data across the Real-Time estate
  • Develop and extend telemetry pipelines that ingest, transform, and route metrics, traces, and logs from distributed services into a coherent observability layer
  • Design and build custom synthetic monitoring agents deployed across hundreds of global sites to continuously measure customer experience of Real-Time data from the edge
  • Implement GitOps/API-driven workflows for observability assets, ensuring consistent deployment, versioning, and promotion through build pipelines
  • Work with squads to integrate observability instrumentation into applications throughout Dev, Test, Pre-Production (PPE), and Production environments.
  • Develop and extend eBPF / kernel-level instrumentation to capture transaction data and observability signals from the network and OS layer with minimal performance overhead.

Capabilities & Skills:

  1. Hands-on software engineering experience building observability or data-pipeline applications-metrics aggregation, distributed tracing, or real-time streaming.
  2. Proficiency in at least one systems-level language (C++, Go, or Rust) and one scripting/application language (Python, Java, or TypeScript)
  3. Experience with OpenTelemetry SDKs, collectors, and schema conventions for instrumentation and telemetry export.
  4. Familiarity with SLI/SLO concepts, especially for latency-sensitive or market data systems.
  5. Experience building and operating custom synthetic monitoring solutions at scale, including lightweight agents distributed across geographically diverse sites.
  6. [Desired] Skilled in cloud-native, containerised, and Kubernetes environments including deploying and operating services at scale.
  7. Proficiency with API- and GitOps-based workflows, config-as-code, and CI/CD pipelines for infrastructure.
  8. Strong analytical mindset for modelling complex distributed-system behaviours and understanding customer impact.
  9. Effective communicator able to work across squads and simplify system performance concepts for broader stakeholders.

Job Details

Company
Colehouse Group
Location
City of London, London, United Kingdom
Posted