Plan & manage an Azure AI solution · 25–30%
Choose the right Foundry service & model
The platform in one picture
- Microsoft Foundry is Microsoft’s platform for building AI apps and agents on Azure — a web portal plus SDKs. Old names you’ll still see: Azure AI Foundry, AI Studio.
- A Foundry resource holds one or more projects (one is the default). Projects hold models, agents, tools, knowledge.
- Foundry Tools are prebuilt, task-specific AI APIs (language, speech, vision, documents) you call instead of prompting a model (formerly Azure AI Services, before that Cognitive Services).
- Foundry IQ = one central MCP-based knowledge connection for agents — grounding without managing your own search infra.
Service picker
| Task | Use |
|---|---|
| Chat, content generation, reasoning | Foundry Models (LLM deployment) |
| Agents (LLM + instructions + tools) | Foundry Agent Service |
| Text analytics: entities, sentiment, summarization, PII | Azure Language |
| Speech to text / text to speech / live voice | Azure Speech |
| Text translation | Azure Translator |
| Fields from invoices, receipts, forms | Azure Document Intelligence |
| Multimodal extraction: docs + images + video + audio | Azure Content Understanding |
| Vector / hybrid search, RAG retrieval | Azure AI Search |
| Central knowledge for agents | Foundry IQ |
If the question says “cost-effective, predictable, common task”, pick a Foundry Tool, not a generative model. If it says “generate original content / reason”, pick an LLM.
Model type picker
| Need | Model type | Examples |
|---|---|---|
| Deep reasoning, complex generation | LLM | GPT-5, Mistral Large, Llama 70B |
| Speed + low cost + edge devices | SLM | Phi-4, Llama 8B |
| Math / code / multi-step logic (built-in reasoning) | Reasoning model | o-series, Claude Opus |
| Semantic search, RAG vectors | Embedding model | Ada, Cohere embed |
| Image from text | Image gen | GPT-image-1 |
| Video from text | Video gen | Sora 2 |
| Prompts that include images | Multimodal | GPT-4.1, GPT-4o |
| Audio ↔ text | TTS / STT | GPT-4o-tts, GPT-4o-transcribe |
Chain-of-thought prompting = technique for non-reasoning models. Reasoning models do stepwise logic internally — don’t prompt them to “think step by step”.
Benchmarks (model selection)
Benchmarks are standardized test scores shown on each model card so you can compare models before deploying one.
- Quality index — averaged accuracy over datasets (MMLU-Pro knowledge, HumanEval+ code, MATH, GPQA, IFEval instruction-following). 0–1, higher = better.
- Safety — HarmBench measures Attack Success Rate (ASR), lower = safer; ToxiGen (F1, higher = better detection); WMDP.
- Cost — $/1M input + output tokens; estimated cost assumes 3:1 input:output.
- Performance — latency (mean, P50–P99), TTFT (time to first token), tokens/sec.
- Portal tools: model leaderboard, scenario leaderboards, trade-off charts (quality vs cost), side-by-side compare (max 3 models).
Retrieval & indexing choice
| Search type | How it matches |
|---|---|
| Keyword | Exact terms |
| Vector | Embedding similarity (cosine) — meaning, not words |
| Semantic | Semantic models re-rank meaning |
| Hybrid | Keyword + vector + semantic — recommended for gen-AI apps |