Plan & manage an Azure AI solution · 25–30%
Manage, monitor & secure
Quotas, scaling, cost
A quota is the ceiling on how much model traffic your subscription can push.
- Quota is per region + model, measured in TPM (tokens per minute); rate limits also apply as RPM (requests per minute).
- You split regional TPM quota across deployments; raise via quota request.
- Hitting the limit returns HTTP 429; handle it with retry + exponential backoff.
- Cost levers: pick smaller model (SLM), shorten prompts, cache, Batch deployments (50% off), PTU reservations for steady load,
max_tokenscaps.
Monitoring — what signal answers what question
| Question | Watch |
|---|---|
| Model quality dropping over time? | Drift (gradual quality decay as real-world data shifts) — re-evaluate against a baseline |
| Fabrications? | Groundedness evaluators |
| Harmful output happening? | Safety events / content-filter logs |
| Search feeding good context? | Index health, relevance metrics, ingestion quality |
| Cost/latency? | Token usage analytics, latency breakdowns (TTFT, tokens/sec) |
- Azure Monitor + portal metrics cover requests, tokens, latency, errors.
- Tracing (OpenTelemetry-based in Foundry) records each step of app/agent flows — see the observability page in GenAI area.
Security checklist
- Managed identity — an identity Azure creates and rotates for your app, so it authenticates to services with no secrets in code. Keyless = Entra ID auth (
DefaultAzureCredential), recommended for production. - RBAC roles scope what identities can do (e.g., Azure AI User to call models; Owner/Contributor to manage). Grant least privilege.
- Private networking — private endpoints + disable public network access so traffic stays on your VNet.
- Keys: if you must use them, store in Azure Key Vault, rotate regularly.
“Keyless credentials” in the study guide = Entra ID / managed identity auth instead of API keys. When a question offers both, keyless wins for production.