What is Foundry Observability?
Foundry Observability is the monitoring and logging component of the Microsoft Foundry platform. The service captures all interactions with AI models: prompts, responses, token consumption, latency, and errors. This data enables performance analysis, quality monitoring, and cost optimization of LLM applications in production.
Observability features integrate seamlessly with Azure Monitor and Application Insights. Dashboards show real-time metrics, alerts notify on anomalies, and Log Analytics enables deep analysis. For quality assurance, GPT-based evaluators can automatically assess groundedness, relevance, and coherence of responses.
Core Features
- End-to-end tracing from prompt to response
- Token tracking for cost analysis per use case
- Latency monitoring with percentile metrics
- Integration with Azure Monitor and Application Insights
- GPT-based quality metrics in production
Typical Use Cases
Engineering teams monitor LLM applications in production. They track latency spikes, identify slow prompts, and optimize token usage. Alerts warn of unusual error rates or latency increases.
FinOps teams analyze token costs per feature, customer, or application area. Reports show trends, identify cost-intensive use cases, and validate ROI of AI investments.
ML teams monitor the quality of AI outputs over time. Automated evaluations detect drift in response quality, e.g., when groundedness scores drop and the model begins to hallucinate.
Benefits
- Full transparency over AI applications in production
- Early detection of performance and quality issues
- Data-driven optimization of prompts and model selection
- Compliance logging for regulated industries
Integration with innFactory
As a Microsoft Solutions Partner, innFactory supports you with Foundry Observability: monitoring setup, dashboard design, alert configuration, and quality management.
Typical Use Cases
Frequently Asked Questions
What is logged?
Prompts, responses, token counts, latency, model version, and custom metadata. Sensitive data can be masked before logging.
How do I integrate Observability into my application?
Through the Azure AI SDK, traces are automatically captured. For custom instrumentation, use OpenTelemetry-compatible APIs.
Can I set alerts for quality issues?
Yes, define alerts for latency thresholds, error rates, or GPT-based quality metrics like low groundedness scores.
How high is the overhead?
Minimal. Logging is asynchronous and does not affect response time. Storage costs depend on retention and data volume.
