Analytics & Observability
Monitor AI usage, costs, performance, and system health
Girard provides comprehensive analytics and observability tools to monitor your AI usage, control costs, track performance, and maintain system health. Get real-time insights into every aspect of your AI operations.
Overview
The Analytics Platform provides:
- AI Usage Dashboard - Track model usage and costs
- Performance Metrics - Response times and throughput
- Cost Management - Budget controls and optimization
- Log Aggregation - Centralized logging with search
- Alerting - Proactive notifications
- Model Optimization - Intelligent model selection
Getting Started
Access analytics from the dashboard:
- Analytics Dashboard -
/analytics - Usage -
/usage - Monitoring -
/monitoring - Logs -
/admin/audit-logs
AI Usage Dashboard
Usage Metrics
Track AI consumption in real-time:
| Metric | Description |
|---|---|
| API Calls | Total requests to AI models |
| Tokens Used | Input + output tokens |
| Cost | Total spend in dollars |
| Response Time | Average latency |
| Error Rate | Failed requests % |
| Cache Hit Rate | Cached response % |
Model Breakdown
See usage by model:
model_usage:
period: "last_30_days"
models:
claude-sonnet-4:
requests: 45000
tokens: 12500000
cost: 450.00
avg_latency: 1.2s
claude-opus-4-5:
requests: 5000
tokens: 2000000
cost: 300.00
avg_latency: 2.8s
gpt-4o:
requests: 20000
tokens: 8000000
cost: 200.00
avg_latency: 1.5s
gemini-2.0-flash:
requests: 80000
tokens: 25000000
cost: 125.00
avg_latency: 0.8s
Time Series Analysis
View trends over time:
- Hourly, daily, weekly, monthly views
- Compare periods
- Identify usage patterns
- Forecast future usage
Cost Management
Budget Controls
Set spending limits:
budget:
monthly_limit: 5000.00
warning_threshold: 80% # Alert at $4000
hard_limit: true # Stop at limit
per_model:
claude-opus-4-5:
daily_limit: 100.00
monthly_limit: 2000.00
per_feature:
workflows: 2000.00
agents: 1500.00
chat: 1000.00
Cost Alerts
Configure cost notifications:
alerts:
- type: daily_spend
threshold: 200.00
channels: [email, slack]
- type: monthly_budget
threshold: 80%
channels: [email, slack, pagerduty]
- type: unusual_spike
threshold: 3x_average
channels: [slack]
Cost Optimization
Automatic cost reduction strategies:
Dynamic Model Selection:
- Route simple tasks to cheaper models
- Use expensive models only when needed
- Automatic fallback on errors
Caching:
- Cache identical requests
- Semantic caching for similar queries
- TTL-based invalidation
Token Optimization:
- Prompt compression
- Response truncation
- Context window management
Cost Analysis
Breakdown costs by dimension:
cost_analysis:
by_feature:
workflows: $1,200
agents: $800
chat: $400
studio: $600
by_user:
heavy_users: 15%
typical_users: 75%
light_users: 10%
by_model:
opus: 35%
sonnet: 40%
haiku: 15%
gpt4: 10%
Performance Monitoring
Latency Metrics
Track response times:
| Percentile | Target | Current |
|---|---|---|
| p50 | < 500ms | 380ms |
| p95 | < 2s | 1.4s |
| p99 | < 5s | 3.2s |
Throughput
Monitor request volume:
- Requests per second
- Peak load times
- Queue depth
- Concurrent requests
Error Tracking
Monitor and categorize errors:
error_breakdown:
rate_limits: 2.1%
timeouts: 0.8%
model_errors: 0.3%
validation_errors: 0.5%
other: 0.2%
total_error_rate: 3.9%
AI Model Optimization
Dynamic Model Selection
Girard automatically selects the optimal model based on task complexity:
model_optimizer:
strategies:
- task: simple_classification
preferred: claude-3-haiku
fallback: claude-sonnet-4
- task: complex_analysis
preferred: claude-opus-4-5
fallback: gpt-4o
- task: code_generation
preferred: claude-sonnet-4
fallback: gemini-2.0-flash
constraints:
max_cost_per_request: 0.10
max_latency: 5s
quality_threshold: 0.85
Task Complexity Analysis
AI analyzes each request to determine:
- Token complexity
- Domain expertise needed
- Reasoning depth required
- Quality requirements
Model Performance Tracking
Compare model performance:
model_comparison:
task_type: "customer_support"
metrics:
claude-sonnet-4:
quality_score: 0.92
avg_cost: 0.02
avg_latency: 1.1s
recommendation: "Best balance"
claude-opus-4-5:
quality_score: 0.98
avg_cost: 0.15
avg_latency: 2.5s
recommendation: "Complex cases only"
gemini-2.0-flash:
quality_score: 0.85
avg_cost: 0.005
avg_latency: 0.6s
recommendation: "Simple queries"
Log Aggregation
Centralized Logging
All logs collected in one place:
- Application logs
- AI request/response logs
- Workflow execution logs
- Audit logs
- Error logs
Log Search
Powerful search capabilities:
search:
query: "workflow failed"
filters:
level: [error, warn]
domain: [workflows, agents]
time_range: "last_24h"
organization: "org_123"
results:
total: 47
logs:
- timestamp: "2026-02-08T10:23:45Z"
level: error
message: "Workflow execution failed: timeout"
workflow_id: "wf_456"
step: 3
error: "AI model timeout after 30s"
Log Analytics
Aggregate insights from logs:
log_analytics:
period: "last_7_days"
total_logs: 2450000
by_level:
debug: 1800000
info: 500000
warn: 100000
error: 50000
error_rate: 2.0%
top_errors:
- message: "Rate limit exceeded"
count: 15000
- message: "Connection timeout"
count: 8000
- message: "Invalid API key"
count: 5000
logs_per_hour:
- hour: "2026-02-08T00:00"
count: 85000
errors: 1200
Log Export
Export logs in multiple formats:
- JSON
- CSV
- NDJSON (newline-delimited JSON)
Retention Policies
Configure log retention:
retention:
default: 30 # days
by_level:
error: 90 # Keep errors longer
warn: 60
info: 30
debug: 7
by_organization:
enterprise: 365 # 1 year for enterprise
pro: 90
free: 14
Alerting
Alert Configuration
Set up proactive alerts:
alerts:
- name: "High Error Rate"
condition: "error_rate > 5%"
window: 5m
severity: critical
channels: [pagerduty, slack]
- name: "Latency Spike"
condition: "p95_latency > 5s"
window: 10m
severity: warning
channels: [slack]
- name: "Budget Warning"
condition: "monthly_spend > budget * 0.8"
severity: warning
channels: [email, slack]
- name: "Model Degradation"
condition: "quality_score < 0.8"
window: 1h
severity: warning
channels: [slack]
Alert Channels
Send alerts to:
- Slack
- PagerDuty
- Webhooks
- SMS
Alert History
Track alert history:
- When alerts fired
- Resolution time
- Acknowledgment status
- Related incidents
Dashboards
Pre-built Dashboards
Ready-to-use dashboards:
- AI Usage Overview
- Cost Analysis
- Performance Metrics
- Error Tracking
- Workflow Analytics
- Agent Performance
Custom Dashboards
Build custom dashboards:
dashboard:
name: "Operations Overview"
refresh: 30s
widgets:
- type: metric
title: "API Calls Today"
query: "sum(api_calls) where time > today()"
position: [0, 0, 3, 1]
- type: line_chart
title: "Requests Over Time"
query: "count(requests) by hour"
position: [3, 0, 6, 2]
- type: pie_chart
title: "Cost by Model"
query: "sum(cost) group by model"
position: [0, 1, 3, 2]
- type: table
title: "Top Errors"
query: "top 10 errors by count"
position: [6, 0, 3, 2]
API Reference
Get Usage Metrics
GET /api/v1/analytics/usage
?start=2026-02-01
&end=2026-02-08
&granularity=day
&models=claude-sonnet-4,gpt-4o
Response:
{
"data": [
{
"date": "2026-02-01",
"requests": 15000,
"tokens": 4500000,
"cost": 150.00,
"by_model": {...}
}
]
}
Search Logs
POST /api/v1/logs/search
{
"query": "workflow failed",
"level": ["error"],
"startDate": "2026-02-07",
"endDate": "2026-02-08",
"limit": 100
}
Get Cost Breakdown
GET /api/v1/analytics/costs
?period=month
&groupBy=feature
Response:
{
"total": 2500.00,
"breakdown": {
"workflows": 1200.00,
"agents": 800.00,
"chat": 300.00,
"studio": 200.00
}
}
Next: Lead Generation | Previous: Social Media