The problem classic monitoring cannot see
A platform can be completely in the green —normal CPU, services responding, zero errors in the logs— and still not be delivering service: payments that don't go out, files that don't arrive, a business flow at a standstill. Classic infrastructure signals measure whether components are alive, not whether the business flows. Enterprise observability closes that gap: it connects metrics, logs and traces with business-flow signals.
The right place for AI
The fashionable proposal is to hand diagnosis to a model. The correct proposal is more boring and safer: AI interprets the evidence; it does not produce it. Deterministic tools —queries, checks, extractors— generate verifiable facts with their own identifiers; the model reads them, correlates them and writes a hypothesis any engineer can audit fact by fact. If the model produces the evidence, the diagnosis stops being verifiable, and an unverifiable diagnosis on a critical platform is a risk, not a help.
This also changes how it is consumed: the operations team doesn't write prompts. It consumes versioned, authorized, auditable capabilities — "verify flow X", "investigate delay Y" — with permissions and logging, like any other operation against production.
Where the real investment is
The hard part is not the model; it can be replaced with one line of configuration. The hard part is everything around it:
- Reliable deterministic tools, with clear contracts and owners.
- A single point of governance for AI consumption (models, costs, permissions, audit).
- Evidence normalization, so different systems speak a common format.
- Full traceability: what was asked, what evidence was used, what was concluded.
There is also a silent trap: semantic retrieval of documentation is probabilistic too — it can bring back the similar-but-obsolete runbook. That layer needs the same versioning rigor as the rest.
Costs: tokens as an architecture decision
AI consumption is governed like any other resource: by design, not by surprise. A model strategy by complexity (not every incident needs the most expensive model), budgets per capability, and a business metric: cost per correctly diagnosed incident, compared against the hourly cost of a team investigating by hand — and against the cost of the downtime avoided.
Where to start (and when not to)
If I had to build it today, I would start with the deterministic tools and their catalog: they create immediate value —faster manual diagnosis, consistent evidence— even if the AI never arrives. AI is added once the base exists. And if your operation is small, with few incidents and simple flows, this full architecture is over-engineering: a good runbook and well-placed alerts yield more.
