Senior Data Engineer - Trading

Role Summary

Aubay is seeking a Senior Data Engineer to join our Supermajor client’s trading & supply space and build a robust data infrastructure that supports research and systematic trading strategies. The role involves designing, optimising, and managing real-time and historical market data pipelines in a cloud-native environment, enabling advanced analytics such as TCA and pre/post-trade insights.

Required Skills and Experience

• Minimum 8 years’ experience in Commodity Trading, Capital Markets, or Investment Banking
• Strong understanding of global futures exchanges and the exchange-traded derivatives lifecycle
• 5+ years working with large financial datasets (market data, order/execution data, positions data)
• 5+ years of experience with real-time market data distribution platforms and feed handlers
• Strong programming/scripting skills (Python, PowerShell, C#, SQL or similar)
• Experience with time-series databases (e.g., KDB, OneTick)
• Knowledge of Parquet-based data storage
• Solid experience in Linux environments and scripting (Bash, Perl)
• Experience building and maintaining high-performance data pipelines
• Strong understanding of data validation, monitoring, and error handling
• Exposure to trading analytics (TCA, pre- and post-trade analytics)

Desired skills and experience

• Familiarity with Kubernetes, containerization, and orchestration tools
• Experience with Databricks or Apache Iceberg
• Advanced experience with time-series databases (KDB, OneTick)
• Experience optimising query performance for large-scale datasets

Roles responsibilities

• Design, build, and maintain real-time and historical market data pipelines
• Evaluate, onboard, and integrate complex data products from external vendors
• Collaborate with traders, stakeholders, and engineering teams to define data requirements
• Develop and enhance analytical data services and statistical functions
• Optimise data infrastructure for fast query execution and high performance
• Ensure data integrity through validation, monitoring, and error handling processes
• Provide operational support, troubleshooting, and issue resolution for data pipelines
• Support the development of trading strategies through high-quality data delivery
• Continuously improve data solutions to enhance trading performance and analytics