Senior Engineer, Commodities Technology
Location:
LondonDepartment: TechnologyExperience Level:
Experience
ProfessionalsContact: Paul KrugJob ID: REQ8322Role overviewWe are hiring a senior engineer to build and operate the data systems that turn vendor feeds, weather data, and satellite and vessel-tracking signals into decisions our commodities investment teams act on directly.
The role covers the full path from external data sources to end users. You will ingest vendor feeds, define schemas and data models, load data into analytical stores, and build controls that identify incomplete or incorrect data before it reaches the desk. The datasets you'll work with span the full spectrum, from megabyte-sized reference feeds to petabyte-scale alternative-data sources.
You will work directly with traders, portfolio managers, and researchers to understand new data requirements and investigate production issues. These pipelines feed trading decisions directly, so the systems you build, and fix have immediate, visible impact. Alongside Python, SQL, and orchestration work, you will contribute to services and streaming infrastructure where latency, throughput, and memory usage matter. You will split your time between building new services and improving production systems already used by investment teams. As part of this work, you will have the opportunity to help define and evolve novel AI-driven workflows, building on BAM's commitment to being at the forefront of AI adoption.
Required experience- 5+ years in data engineering, including ownership of data platforms, lakehouses, distributed systems, or backend services.- Experience with analytical databases such as Snowflake, Redshift, ClickHouse.- Strong Python and SQL skills.- Production experience with Java or Kotlin. We will also consider strong experience in another systems language.- Experience with Kafka or another streaming or near-real-time data platform.- Experience with Kubernetes, CI/CD, monitoring, logging, and alerting.- The ability to diagnose technical problems and communicate clearly with both engineers and investment professionals.- Commodities or energy market exposure (power, gas, oil, shipping, agriculture, weather) is a strong plus but not essential.
Responsibilities- Integrate external commodities and alternative-data feeds across different formats, delivery methods, update frequencies, and volumes — from megabytes to petabytes.- Profile and improve latency, throughput, memory usage, and query performance.- Build validation, reconciliation, and monitoring to detect missing, delayed, or incorrect data.- Investigate production issues with traders, portfolio managers, and researchers, prioritising work according to business impact.- Design, build, and operate production data pipelines and platform services.- Define schemas and data models that make vendor data consistent and useful to downstream consumers.- Improve the reliability and observability of existing services and pipelines.- Help migrate and strengthen Airflow-based workflows.- Participate in the shared on-call rota for systems used by the trading teams.- Hold engineering standards across the team: review, testing, release.