Senior Data Engineer | Energy Trading | Agnostic - Cloud-native AI Centric Data Platform | Next-Gen Lakehouse | Up to £100K + Bonus + Benefits
Senior Data Engineer | Energy Trading | Agnostic - Cloud-native, AI Centric Data Platform | Next-Gen Lakehouse | Up to £100K + Bonus + Benefits
Senior Data Engineer is required for a leading global Energy & Commodities trading house — one of the world's largest and most diverse general trading companies. Their trading ecosystem spans multiple commodities and markets, creating high-volume, high-velocity data challenges ideally suited to ClickHouse and modern data engineering.
As a Senior Data Engineer, you will play a pivotal role in chossing the right tooling, building and scaling a CLoud-native AI-driven analytical data platform. You'll be a senior member of the Core Engineering team, shaping the architecture and implementing high-performance data solutions used directly by trading, risk, analytics and operational teams.
Whilst the firm have agreed to do an Azure Databricks POC platform in Q1 2026, with so much of their data on-prem, they are still showcasing different Cloud-native platforms, and therefore are looking for someone who is excited to build Proof of Concepts in the right platform for the business requirements.
This is a hands-on position where you'll be working with a high skilled small team of engineers while designing and developing a next-generation lakehouse architecture.
Azure Databricks & ClickHouse will form the backbone of examples of 2 systems, supporting ultra-fast analytics, market-data ingestion, tick-data storage, and real-time insights for traders and quantitative teams.
Role Details
- Salary: Up to £100,000 per annum + Bonus & Benefits
- Location: London (Hybrid — 3 days per week onsite)
What You'll Do
- Design, build, and optimise high-performance ClickHouse / Azure DataBricks / AI Cloud-native Data clusters for large-scale analytical workloads across commodities and energy trading.
- Develop ingestion frameworks for market data, pricing data, tick data, and trade/position datasets, leveraging ClickHouse's columnar storage and real-time querying.
- Architect and maintain a modern lakehouse platform integrating ClickHouse with Databricks, Snowflake, and Azure-based services.
- Define and enforce best practices for data modelling, partitioning, compression, and query optimisation tailored for ClickHouse.
- Build scalable Python-based ETL/ELT pipelines and data services that deliver robust, low-latency data to trading, risk, and analytics teams.
If you are interested in this role, please send your CV to ross@virtuetech.io
Senior Data Engineer | Energy Trading | Agnostic - Cloud-native, AI Centric Data Platform | Next-Gen Lakehouse | Up to £100K + Bonus + Benefits