Application Insights Metrics
CritterWatch can read throughput, failure-rate, baseline, and dead-letter rate from the Log Analytics workspace behind an Application Insights resource, instead of Prometheus, VictoriaMetrics, Datadog — or its own Postgres store. The alert evaluator, the per-service summary, the Dashboard's 24h throughput trend, and the trend visualizations all work transparently against whichever metrics backend the operator has configured.
This is the bring-your-own-observability posture: your Wolverine services keep exporting OTel metrics to Azure Monitor exactly as they already do, CritterWatch queries the workspace on demand with KQL, and — combined with SystemDiagnosticsMeter mode and live-only persistence below — CritterWatch stores no metrics data at all.
What CritterWatch needs from Azure
- The Log Analytics workspace id (a GUID) backing your Application Insights resource. This is the workspace id — not the instrumentation key, not the Application Insights app id. Workspace-based Application Insights resources (the default since 2021) all have one.
- An identity with the Log Analytics Reader role on that workspace. Auth uses
DefaultAzureCredential: a managed identity when the console runs in Azure, your developer sign-in locally. No keys or secrets are configured, stored, or persisted anywhere — consistent with the rule that bindings persist source names only. - Your Wolverine services already exporting via the Azure Monitor OpenTelemetry distro (
Azure.Monitor.OpenTelemetry.AspNetCore). The distro lands Wolverine's instruments in the workspaceAppMetricstable with their dimensions (message.type,message.destination,tenant.id,exception.type) intact.
How to wire it up
In your CritterWatch host startup:
services.AddCritterWatchMetricsDataSource<AppInsightsMetricsDataSource, AppInsightsMetricsDataSourceOptions>(
"appinsights",
opts =>
{
// The Log Analytics WORKSPACE id backing the Application Insights resource —
// not the instrumentation key, not the Application Insights app id.
opts.WorkspaceId = builder.Configuration["AppInsights:WorkspaceId"]!;
// Auth defaults to DefaultAzureCredential (managed identity in Azure,
// developer sign-in locally). Assign opts.Credential to override.
// AppInsights ingest lag is routinely 1-3 minutes; every query window ends
// this far behind "now" so the un-ingested tail never reads as starvation.
opts.IngestLagAllowance = TimeSpan.FromMinutes(3);
});Per-service routing
// In the monitored service's UseWolverine — point it at the named source the console
// registered, and stop pushing metrics to CritterWatch at all: AppInsights is the
// system of record on both the write and the read side.
opts.AddCritterWatchMonitoring(
critterWatchUri, systemControlUri,
metricsMode: WolverineMetricsMode.SystemDiagnosticsMeter)
.MetricsDataSource("appinsights");WolverineMetricsMode.SystemDiagnosticsMeter stops the service pushing metrics to CritterWatch entirely — Azure Monitor is the only metrics pipeline. The operator can override the source binding per-service via Settings → Metrics Data Sources.
A service that is both SystemDiagnosticsMeter and bound to an external source defaults to live-only persistence — CritterWatch writes no MetricsSample rows for it (see Metrics storage and sizing). An explicit Persist same as internal override in Settings restores console-side persistence per service.
Which metrics CritterWatch reads
The workspace AppMetrics table, under the instrument names Wolverine emits:
| What | Wolverine instrument | KQL shape |
|---|---|---|
| Executions | wolverine-execution-time | sum(ItemCount) |
| Sent and received | wolverine-messages-sent, wolverine-messages-received | sum(Sum) |
| Failures | wolverine-execution-failure | sum(Sum) |
| Mean execution time | wolverine-execution-time | sum(Sum) / sum(ItemCount) |
| Mean effective time | wolverine-effective-time | sum(Sum) / sum(ItemCount) |
| Dead letters | wolverine-dead-letter-queue | sum(Sum), grouped by message.type / exception.type |
Executions are the observations of the execution-time histogram, the same count every other source reads. Before 1.2 this source counted received messages as executions, so throughput, its baseline, the failure rate's denominator and the Dashboard's 24h trend can all read differently after upgrading, chiefly for a service that retries or handles messages it did not receive from a transport.
Every interface call is one batched KQL query (a single summarize ... by Name), not one query per metric — the Log Analytics query API is rate-limited and the alert evaluator polls per service.
Cloud role mapping: CritterWatch filters on AppRoleName == '<service name>'. If your services report under a different cloud role name than their CritterWatch service name, map them with opts.ServiceLabelMapping["ServiceName"] = "cloud-role-name".
Ingest lag
Application Insights ingestion routinely runs 1–3 minutes behind. Every query window this source issues ends IngestLagAllowance (default 3 minutes) behind now, so the not-yet-ingested tail never reads as a phantom throughput drop. Consequences to expect:
- Metrics views trail live traffic by roughly the allowance plus the actual ingest lag.
- Short-window alert checks evaluate a window that ended a few minutes ago — a real incident is detected a few minutes later than it would be with pushed metrics. That is inherent to the posture, not a configuration problem.
- If your workspace's observed lag exceeds 3 minutes, raise
IngestLagAllowancerather than chasing false low-throughput alerts.
Measured on a live South Central US workspace during an end-to-end verification (a Wolverine service exporting via the Azure Monitor exporter at default 60-second intervals): average lag 39.7 s, p95 2 m 34 s (ingestion_time() - TimeGenerated over the wolverine-* instruments). The 3-minute default allowance covered the p95 with margin; the source, message.type, and message.destination dimensions all arrived intact in AppMetrics.Properties.
Baselines
Unlike the other external sources, the AppInsights source implements the oldest-sample probe (min(TimeGenerated) over a bounded lookback), so observed baselines mature normally once the workspace holds enough history — deviation-style alerts work without declaring baselines by hand. Absolute-threshold alerts work from day one either way.
Per-tenant alerting
On a multi-tenant service, per-tenant metrics alerts evaluate each tenant against its own numbers. Every per-tenant read narrows its KQL to the tenant's measurements by Wolverine's tenant.id dimension in Properties, and the baseline probe is per tenant too, so a tenant onboarded yesterday is not treated as having the service's week of history. A measurement with no tenant carries no tenant.id, so the service-wide reads still sum every tenant, as before.
Tenants do not multiply the query count. The evaluator asks about each tenant in turn, but each read is answered from one batched summarize … by Tenant per service, reused for PerTenantBatchLifetime (20 seconds by default, under the evaluator's 30-second cadence). A pass costs the same handful of queries for a service with one tenant or thousands, so per-tenant alerting cannot exhaust the workspace's query rate limit — which would otherwise make the service-wide reads fail too, and read as starvation.
When a metric isn't available
Query failures, missing tables, and empty results all degrade to zero-populated snapshots — blank cells in the affected widgets, never error pages. Same contract as every other external source. Verify your instruments are actually landing with a workspace query like:
AppMetrics | where Name startswith "wolverine-" | summarize count() by NameIf that returns nothing, the exporter side is the problem — confirm the Azure Monitor OTel distro is registered in the monitored service and that custom metrics aren't being filtered.
