Quality Engineer

Quality Engineer | £750/day Inside IR35 | 6-month initial contract | London

Industry: Technology

Location: London

Job Type: Contract - 6-month initial

This programme delivers a production-grade enterprise agentic AI platform, with MCP acting as the extensibility layer.

Responsibilities

• Design and implement evaluation frameworks for MCP tools and Skills, covering correctness, safety, reliability, and regression performance.
• Build automated testing pipelines for agentic behaviours, including multi-step workflows and tool-calling interactions.
• Identify and mitigate AI failure modes such as hallucination, incorrect tool usage, invalid inputs, and latency amplification.
• Produce and maintain formal testing evidence required for internal release approvals (build and operate gates).
• Collaborate with ML Engineers to integrate test hooks and observability directly into MCP servers and Skills.
• Contribute to defining long-term quality assurance standards, testing practices, and governance models for a federated MCP ecosystem.

Skills & Experience Required

• Strong Python experience for building test frameworks, automation tooling, and validation pipelines.
• Experience with LLM evaluation approaches, RAG evaluation frameworks, or custom model assessment systems.
• Hands-on experience in test automation across unit, integration, and regression testing.
• Understanding of agentic AI systems and their failure modes, particularly in non-deterministic environments.
• Ability to assess systems against governance, security, and audit/compliance requirements.
• Experience working within regulated or highly controlled engineering environments.