Plan & manage an Azure AI solution · 25–30%
Responsible AI
Responsible AI is the practice of building AI systems that are fair, safe, private, and accountable — and proving it with process. The exam tests Microsoft’s version of that process.
Microsoft’s 6 principles
Fairness · Reliability & safety · Privacy & security · Inclusiveness · Transparency · Accountability
Memory hook: “FRPITA” — or just remember pairs: fair+reliable, private+inclusive, transparent+accountable.
The 4-stage process (maps to NIST AI RMF)
- Map — identify potential harms, prioritize them by likelihood × impact, red-team test them, then document and share.
- Measure — build a prompt suite likely to elicit each harm, run it, categorize the output against strict criteria, and record the result as your baseline. Start manual, then automate.
- Mitigate — layered (next section) + transparent communication.
- Manage — pre-release reviews (legal, privacy, security, accessibility), phased delivery, incident-response + rollback plans, block harmful responses/users, user feedback channel, telemetry.
The 4 mitigation layers (from the model up to the user)
| Layer | Mitigations |
|---|---|
| 1. Model | Pick appropriate (smaller if sufficient) model; fine-tune |
| 2. Safety system | Guardrails: content filters, prompt shields, abuse detection |
| 3. System message & grounding | Prompt engineering, RAG on trusted data |
| 4. User experience | Constrain inputs/outputs in UI, transparent docs |
Content filters — exact numbers
Content filters classify every prompt and response into harm categories and block anything above a severity threshold.
- 4 severity levels: safe, low, medium, high.
- 5 harm categories: hate & fairness, sexual, violence, self-harm, task-adherence.
- Prompt shields — detect jailbreak / system-prompt subversion (direct + indirect attacks).
- Defect rate in evaluations = % of responses above severity threshold (typically Medium).
“User tries to override the system prompt” means prompt shields. “Block gory images” means a content filter, category violence. “Model invents facts” is not a filter problem — that’s groundedness (fix with RAG and evaluators).
Instrumentation & governance (agents too)
- Evaluators — groundedness, relevance, coherence, fluency + safety evaluators; run pre-deploy and continuously.
- Auditing — trace logging, provenance metadata, approval workflows.
- Agent governance — oversight modes (human-in-the-loop approvals), constraints in instructions, tool-access controls (limit which tools an agent may call).