Market Data Engineer

Market Data Engineer – Quantitative Trading Firm

We are seeking a Market Data Engineer to manage the capture, normalisation, and distribution of petabytes of historical market data from global exchanges. Working closely with research, development, and infrastructure teams, you will ensure that our data pipelines are reliable, efficient, and meet the high-performance needs of our trading and research operations.

Key Responsibilities:

• Manage large-scale, time-sensitive market data capture and normalisation pipelines.
• Maintain and enhance internal data formats and common downstream research datasets.
• Collaborate with infrastructure teams to ensure reliable packet capture processes.
• Implement robust data validation and monitoring for new data sources.
• Provide production support, troubleshoot issues, and ensure timely resolution.

Qualifications:

• Experience building and maintaining extremely large-scale ETL pipelines.
• Proficient in Python and Bash; familiarity with C++.
• Knowledge of computer networking fundamentals.
• Experience with ETL orchestration frameworks (e.g., Airflow, Luigi, Dagster).
• Experience with distributed computing environments (e.g., Slurm, Celery, HTCondor).

Preferred:

• Experience with binary market data specifications (e.g., ITCH, MDP3).
• Understanding of high-performance filesystems and columnar storage formats.

This is a high-impact role where your work directly supports cutting-edge quantitative research

and trading strategies, handling some of the most challenging market data at scale.