Data Scientist

Senior Data Scientist – AI & Conversational Systems (RAG / LLMs) - £800 per day Outside IR35

Location: London / Hybrid

About the Role

A leading financial media and data business is building a next-generation AI-powered conversational assistant designed to surface insights across both proprietary datasets and premium editorial content.

The organisation is seeking a Senior Data Scientist to help evolve this platform into a high-quality, production-grade system. The role focuses on improving answer quality, retrieval performance, and real-time responsiveness - translating cutting-edge LLM capabilities into reliable, user-facing products.

Responsibilities

• Develop and optimise RAG (Retrieval-Augmented Generation) pipelines
• Design approaches that combine:
• Unstructured data (articles, reports, documents)
• Structured data (databases, SQL systems)
• Build and refine text-to-SQL capabilities
• Evaluate and improve LLM outputs:
• Accuracy, grounding, and hallucination reduction
• Retrieval relevance and ranking performance
• Design and implement evaluation frameworks (offline and online)
• Collaborate with engineering teams to deploy models into production environments
• Identify and solve real-world implementation challenges
• Contribute to shaping and executing a focused AI roadmap

Requirements

Essential:

• Proven experience delivering production-grade RAG systems
• Deep expertise in working with LLMs for high-quality answer generation
• Databricks experience
• Strong experience with both:
• Unstructured text data
• Structured data / SQL systems
• Experience in model evaluation, experimentation, and optimisation
• Ability to translate complex problems into scalable, practical solutions