Generative AI & agentic solutions · 30–35%

Optimize & operationalize

~2 min read

The 3 strategies — pick by symptom

Order of attack: start with prompt engineering; add RAG when accuracy is the problem; fine-tune when consistency is. Combine freely (fine-tune = style, RAG = facts, prompt = per-session rules).

Prompt engineering patterns

Prompt engineering is designing and refining the text you send the model — instructions, examples, structure — to improve its output without touching the model itself.

• Persona — “You are a seasoned marketing pro…”
• Format template — show the output structure wanted.
• Chain-of-thought — “step by step” (non-reasoning models only).
• Few-shot — example pairs (zero-shot = none, one-shot = 1).
• Delimiters — --- / headings / XML tags separate instructions from content.
• Recency bias — models weight prompt end more; repeat key instruction at the end.
• System message checklist, in order: state the role, set boundaries, specify the output format, and add a “when unsure” policy.

Parameters

• Temperature ↑ = creative, ↓ = deterministic (0.2 factual / 0.7 creative).
• Top_p — probability-mass cutoff. Adjust temperature OR top_p, not both.
• max_output_tokens caps length (and cost).

Fine-tuning facts

Fine-tuning is additional training of a pretrained model on your own examples, so the desired style or format gets baked into its weights.

• Uses LoRA (Low-Rank Adaptation) — updates small subset of weights; cheaper than full retraining.
• Data: JSONL of message conversations, same system message throughout (blank = worse accuracy; reuse it at inference), hundreds+ examples.
• Types: SFT (labeled pairs), RFT (grader rewards, complex reasoning), DPO (preferred vs non-preferred pairs, lighter than RL). Combinable (SFT then DPO).
• Also good for: distillation (use a big model’s outputs to train a small one), shortening prompts, better tool selection.
• Costs: training + hourly hosting + retrain on base-model updates. Always baseline first.

Reflection & self-critique

• Model reflection / self-critique loops — model reviews its own output against criteria, revises.
• Chain-of-thought evaluations — evaluate reasoning steps, not only the final answer.
• Hybrid orchestration — route: rules engine for deterministic parts, LLM for open-ended; or a small model triages requests and passes the hard ones to a big model.

Observability

Observability means you can see what your AI system did and why — every step, token, and delay.

Implement tracing (OpenTelemetry via Foundry SDK) to capture: each step of a flow, token analytics (usage per call), safety signals (filter hits), latency breakdowns (TTFT, per-step). Enable tracing on the project client; view traces in Foundry portal.