frameworks Strong, practical knowledge of agent evaluation (“evals”):task-success and trajectory testing, grounding and retrieval quality assessment, hallucination and drift detection, robustness testing, benchmarking, bias and fairness assessment, alignment and misalignment analysis, adversarial testing, and red-teaming Strong LLM/GenAI engineering fluency: prompting strategies, retrieval-augmented generation, context … identifying, assessing, and mitigating risks associated with GenAI and agentic AI Design and deliver complex validation analyses, including model replication, challenger development, sensitivity testing, benchmarking, and ad-hoc quantitative investigations Deliver validation outputs, from test plan design through clear, concise validation reporting and effective challenge of model developers Maintain clear ...
