Read a sample: 1. What an Evaluation Plan Can and Cannot Prove
A workplace assistant answers a question about an employee’s travel reimbursement. Its response is fluent, cites a company document, and tells the employee what to do next. The answer looks useful. But did the cited document actually support the advice? Was it the current policy? Did the assistant overlook an exception? And if the employee included information about a colleague, did the application handle that information appropriately?
Those questions cannot be settled by reading one convincing response. They call for an evaluation: a planned way to collect evidence about how the application behaves under specified conditions. An evaluation plan identifies what matters, how it will be examined, who will make judgments, and how the findings will inform a decision.
The plan does not make the application safe by declaration. It does not establish that every future answer will be correct, that every harmful response has been found, or that a favorable average score applies to every user. It can, however, give a team a more disciplined basis for deciding what to improve, what to investigate, and whether a proposed use is supported by the available evidence.
This chapter introduces the fictional application used throughout the book and the practical artifacts you will build around it. The aim is not to create paperwork for its own sake. It is to make the relationship between a claim, the evidence behind it, and the limits of that evidence visible to the people who must act on it.
Imagine a fictional company called Northstar Services. Its employees routinely consult internal documents about routine workplace processes: submitting expenses, requesting equipment, finding onboarding information, and locating the right support team. The documents are spread across several maintained collections, and employees sometimes struggle to find the relevant guidance.
Northstar’s technology team proposes an assistant called Northstar Guide. An employee can ask a question in ordinary language, and the application retrieves potentially relevant passages from an approved document collection. It then uses an AI model to draft an answer, with references intended to help the employee inspect the source material.
A typical question might be:
> How do I request a replacement laptop, and where can I check the status of my request?
A useful answer would explain the documented process, identify the appropriate channel, and avoid inventing a status-checking feature if the source material does not describe one. If the available documents do not answer the question, the assistant should say so and direct the employee toward an appropriate source of help when one is known.
Northstar Guide is intended to help employees find and understand information. It is not intended to approve expenses, change access permissions, make employment decisions, or replace an authorized person’s judgment. Those boundaries matter because an answer can be useful as guidance while still being unsuitable as the final word in a consequential decision.
Several groups have a stake in the application. Employees need answers they can understand and verify. Document owners need their guidance represented accurately. Support staff may receive questions the assistant cannot resolve. The application team needs actionable feedback on retrieval, prompts, presentation, and other components. Reviewers and decision-makers need evidence that is sufficiently clear to support a release recommendation without being presented as a guarantee.
The scenario is deliberately ordinary. A system does not need to be a fully autonomous agent to create meaningful risk. An inaccurate answer may waste time; an incorrect policy explanation may affect an employee’s choices; an inappropriate disclosure may expose information that should not have appeared. A polished response can make these failures harder to notice.
When a team says it is evaluating an “AI assistant,” it is easy to focus only on the model’s output. For Northstar Guide, that would leave out important causes of success and failure.
The answer depends on the employee’s question, the documents available to the retrieval process, how those documents are selected and presented to the model, the instructions given to the model, and the application’s response formatting. It also depends on the access controls and operational processes around those components. A reviewer may observe an unsupported claim, but the cause could be an irrelevant retrieved passage, an outdated source, an instruction that encourages overconfident answers, or the model’s treatment of otherwise suitable evidence.
The evaluation should therefore describe the application configuration being tested, not merely name a model. If the document collection changes, a later run may produce different answers even when the model and prompt remain unchanged. If the application changes how it retrieves passages, an improvement on one set of questions may come with a regression on another.
Azure AI capabilities can support parts of an evaluation workflow, but a platform feature does not decide what Northstar should value or what risks its stakeholders will accept. Those remain planning and governance questions. Later chapters connect the durable work of defining cases and measures to a repeatable Azure-based workflow, while recognizing that interfaces and available capabilities can change.
An evaluation plan is a structured account of the questions a team needs answered before it can make or revisit a decision. For Northstar Guide, an initial question might be: When employees ask ordinary process questions covered by the approved documents, does the assistant provide relevant answers supported by those documents?
That question can be turned into test cases, scoring instructions, and a review procedure. Other questions may concern missing information, conflicting sources, attempts to obtain restricted…