Databricks Consultant
Straive is a global leader in enterprise-grade data analytics and AI solutions, committed to empowering businesses across various industries with cutting-edge technology and expert insights. Backed by EQT, a top private equity firm, we are uniquely positioned to drive innovation through significant investments and an entrepreneurial spirit.
Our core focus is on delivering advanced Data Analytics & AI Solutions. By combining sophisticated technology with subject matter expertise, we deliver material impact on our clients' topline and streamline their operations. We specialize in providing tailored solutions across financial services, CPG, legal, pharma, life sciences, retail and logistics, helping them build robust data analytics and AI capabilities.
With a client base spanning 30 countries, Straive's strategically located teams operate from eight countries and is headquartered in Singapore. This global presence enables us to offer localized expertise with a worldwide perspective.
Join Straive to be part of a dynamic team at the forefront of data analytics and AI innovation. Here, you'll have the opportunity to contribute to transformative projects, supported by significant investments and an entrepreneurial drive fueled by our partnership with EQT.
Website: https://www.straive.com/
Job Title: Databricks Consultant
Type: Contract
Location: London, England, UK
We are seeking a seasoned Databricks Consultant with deep expertise in Energy & Commodities Trading to design, accelerate, and optimize our trading data platform. Acting as the bridge between trading desks, quants, and engineering teams, you will leverage the Databricks Lakehouse to deliver scalable, real-time pipelines powering trading decisions, risk management, and market analytics.
Key Responsibilities
- Data Architecture: Build and optimize ETL/ELT pipelines using Databricks, Spark (PySpark/Scala), and Delta Lake for structured/unstructured energy market data.
- Trading Analytics: Partner with Quants, Risk Managers, and Traders to implement models, curve generation, position tracking, and risk metrics (e.g., VaR).
- Performance Optimization: Tune clusters and SQL warehouses for low-latency, time-sensitive trading environments.
- Data Governance: Enforce governance, access controls, and compliance with UK/EU financial and energy regulations via Unity Catalog.
- Consultative Leadership: Provide architectural best practices, mentor engineers, and advise stakeholders on maximizing Databricks ROI.
Required Skills & Experience
- Energy Trading Expertise: Proven background in energy trading firms, banks, utilities, or oil/gas majors. Strong knowledge of physical/financial markets (Power, Gas, LNG, Crude, Carbon/Renewables) and trading data types (Reuters, Bloomberg, Trayport, weather, ETRM).
- Databricks Mastery: Hands-on experience with production workloads on Databricks (Azure/AWS), PySpark, Spark SQL, Delta Lake, Unity Catalog, Python, SQL, orchestration tools (Airflow, Workflows, ADF), and streaming (Kafka/Event Hubs).
- Professional Skills: Strong consultative mindset, excellent communication, and ability to interface with traders and senior stakeholders.
This job description is not intended to cover or contain a comprehensive listing of all responsibilities, duties, or activities that are required. Responsibilities, duties, and/or activities may change, or new ones may be added at any time with or without notice.
If you are a motivated professional with a passion for delivering impactful solutions, we’d love to hear from you. Apply today to be part of a dynamic and forward-thinking team at Straive.
“Straive is an Equal Opportunity Employer. Our policy is clear: there shall be no discrimination based on age, disability, sex, race, religion or belief, gender reassignment, marriage/civil partnership, pregnancy/maternity, or sexual orientation.
We are an inclusive organization and actively promote equality of opportunity for all with the right mix of talent, skills and potential. We welcome all applications from a wide range of candidates. Selection for roles will be based on individual merit alone.”