Computer vision solutions · 10–15%
Responsible AI for visual content
Filter unsafe visual content
- Content filters classify image input and generated output in the same harm categories as text (hate, sexual, violence, self-harm) with severity levels.
- Content Understanding integrates Azure AI Content Safety — harmful material blocked during analysis.
- Generation APIs refuse harmful prompts (no image/video returned).
Indirect prompt injection via images
Indirect prompt injection is an attack where instructions arrive hidden inside content the model processes — here, embedded as text inside an image (“ignore previous instructions…”) that a multimodal model reads and obeys.
Mitigations:
- Prompt shields cover indirect attacks (content injected via documents/images).
- Treat OCR’d/extracted text as data, not instructions — state this in the system message.
- Validate and constrain model output after processing untrusted images.
Visual policy enforcement
| Requirement | Approach |
|---|---|
| Mark AI-generated media | Watermarks / provenance metadata (C2AF-style credentials) |
| Prohibited symbols in images | Classification filter + block/flag |
| Brand usage rules | Custom classifier or CU classify field against brand guidelines |
| Inappropriate content detection | Content Safety image APIs, severity thresholds |
| Faces / biometric data | Requires notice + consent; face description gated features |
“User uploads an image containing hidden text that changes agent behavior” — that is indirect prompt injection, and the fix is prompt shields. “Prove a video is AI-generated” means watermark/provenance.