The default observability architecture is centralisation: ship everything to one place and query it there. It is simpler to run, cheaper, and produces the single pane everyone wants.
It also requires that all of your telemetry may legitimately sit in one place, and across a portfolio of independently regulated companies that is precisely the thing you cannot assume.
Why the obvious compromises do not hold
Two approaches get proposed early and both have problems.
Centralise anyway with access control: one store, permissions restricting who sees what. This fails because the data has already moved across the boundary. Access control governs who reads it, not where it resides, and residence is usually the requirement.
Fully separate stacks with no shared view: compliant, and it makes cross-cutting incidents miserable. An engineer chasing something that spans tenants opens several consoles and correlates timestamps by hand, at the exact moment when speed matters.
What federation does instead
Telemetry is collected and stored within each tenant boundary. A query layer above reads across those stores at query time, without relocating the underlying data.
The data stays where it must. The question travels instead.
Practically that meant per-tenant collection and storage, with a query layer that fanned a single query out to each store the requester was entitled to read, merged results, and returned one view. Dashboards were defined once and rendered against whichever tenants the viewer had access to.
Standardise the names or none of it works
Federation is worth very little if the same concept is labelled differently in each tenant. A query across ten stores where service is called service in some and app in others returns ten disconnected answers.
The unglamorous prerequisite is an agreed set of labels applied everywhere: service, environment, team, tenant, version. Enforcing that at collection is far easier than reconciling it at query time, and OpenTelemetry helps here mainly by giving you one place to enforce it regardless of language or runtime.
What it costs
Honestly, more than centralising.
- You operate a storage stack per tenant instead of one, so upgrades and capacity planning multiply.
- Query performance is bounded by the slowest store in the fan-out, and one unhealthy tenant degrades queries for everyone.
- Retention and cardinality decisions are made per tenant, so cost control is a per-tenant conversation.
- There is a new component, the query layer, whose availability determines whether anyone can see anything.
None of that is a reason not to do it when separation is required. It is a reason not to do it when separation is not required, and teams do sometimes federate for aesthetic reasons and inherit the cost for no benefit.
Where to start
Labels first, before any of the storage architecture. Consistent labelling is valuable even in a single stack, it is the hardest thing to retrofit, and every federation decision downstream depends on it.
Then metrics, which federate most cleanly. Logs and traces are heavier and higher cardinality, and are worth doing once the metric layer has proved the model and the naming holds up under real queries.
