A platform in the green that was not delivering service
I operated an enterprise platform that processed transfers and moved information between different systems. It was not an isolated server: it was a chain of applications, scheduled jobs, queues, files and external dependencies.
One day the technical signals said everything was fine. Servers available, main processes running, dashboard clear of critical errors. Infrastructure in the green.
But the operations were not arriving.
One of the components responsible for consuming them had stopped working correctly. The process showed as started, but the queue kept growing and operations were not reaching their destination within the expected time. The operational window was missed. The system was technically available and functionally stopped.
It took us several hours to identify it. Not for lack of metrics, but because the ones we had were focused on infrastructure health rather than on business-flow compliance. The user area noticed before we did that operations were not arriving: we saw available components; the business saw a service that was not fulfilling its purpose.
What surfaced it was a question from the user area. Then came the manual checks on queues, pending files, processing times and last recorded movements. Someone who knew the platform's normal behavior immediately spotted that the number of pending items did not match the hour or the expected volume. The knowledge existed, but it lived in a few people's experience and not inside the monitoring system.
The technical details of this case have been simplified to preserve confidentiality. The operational pattern and the lesson are real.
What I changed afterwards was not adding another dashboard. It was changing the question.
Before Is the component running?
After Is the service processing what it should,
at the expected rate, and within the
committed window?
Out of that change came controls that did not exist before: queue depth, age of the oldest pending item, ingress rate versus consumption rate, time since the last completed operation, processing-window compliance, comparison against normal behavior, alerts for running processes with no real movement, and end-to-end validation through synthetic operations.
A runbook also appeared, to tell apart three conditions that until then had looked identical:
- Process stopped.
- Process running, but not consuming.
- Process consuming below the required capacity.
Observability must not prove that components are switched on. It must prove that the business purpose is being met.
Short on time? Read the executive version (6 min).
From isolated metrics to enterprise observability
The four golden signals Google popularized for monitoring services —latency, traffic, errors and saturation— are a useful baseline, but on their own they do not describe whether a business process is fulfilling its purpose. A mature strategy connects several levels:
| Level | Question it must answer | Examples |
|---|---|---|
| Infrastructure | Are the base resources available? | CPU, memory, disk, network, processes, operating system |
| Platform | Can the internal components operate? | Queues, pools, threads, sessions, certificates, storage |
| Service | Does technical capacity meet the expected level? | Availability, latency, errors, saturation, SLOs |
| Business flow | Is the process moving end to end? | Completed operations, files processed, reconciliations, pending items |
| Experience | Does the business receive the expected result? | End-to-end time, rejections, abandonment, retries |
| Automation and AI | Is the diagnostic layer reliable and efficient? | Tokens, latency, tools executed, evidence, cost per analysis |
The common mistake is stopping at the first level. A server can look healthy while a queue is not being consumed. A database can answer queries while an application exhausts its connection pool. A service can stay up while hundreds of operations pile up outside their processing window.
Enterprise observability appears when these signals are correlated with the system's purpose. Knowing that a process is running is not enough: you need to know whether it is processing, at what rate, with what error rate, which dependencies it uses, and what impact its degradation would have.
The right place for AI
AI should not compute disk utilization, verify that a process exists, check certificate validity, or determine whether a transfer completed. Deterministic software solves that: scripts, SQL queries, APIs, rules, thresholds, synthetic tests and monitoring platforms.
AI adds value in three specific functions.
1. Assisted selection
In a predictable flow, the orchestrator runs an explicit sequence of controls defined by the catalog: AI does not freely decide what to do. In an open investigation it may propose an additional check — reviewing a queue's consumption rate, comparing latency before and after a deployment — but that proposal goes through the policy engine, which validates parameters, permissions, scope and risk before any tool runs.
2. Interpretation
It analyzes structured results and relates them to current documentation, prior incidents, dependencies and authorized procedures.
3. Communication
It turns evidence into a clear, prioritized, actionable explanation for different audiences: operator, specialist, service owner or user area.
Current models can use function calling to interact with external tools through defined contracts. The model requests execution, but it is the application that validates the parameters, enforces permissions and actually performs the action. The correct division of responsibility is this:
AI proposes a check when the flow allows it.
The policy engine decides whether it is authorized.
The orchestrator runs the flow.
Code obtains the evidence.
Rules compute verifiable states.
Retrieval supplies versioned context.
AI interprets and communicates.
The validator checks the response.
The specialist approves critical decisions.
Back to the opening incident. With AI wired to reliable controls, the system could have run the health triages, checked queue depth, compared ingress and consumption rates, queried recent errors and retrieved the applicable runbook, to conclude something like this:
Main processes are running, but the queue keeps growing and no
completed operations have been recorded in the last 40 minutes.
The evidence confirms functional degradation, though it does not
yet allow the root cause to be determined.
That would have significantly cut the time spent gathering information. But if the AI had received only the surface-level dashboard, it would have produced a convincing and wrong explanation, faster: "the platform is operational because servers and processes report a healthy state".
An AI wired to incomplete signals only interprets an incomplete view of the system faster.
From the isolated prompt to an enterprise capability
One of the most frequent mistakes when introducing AI into operations is handing users a text box and expecting them to know how to write the right prompt. That approach makes everyone describe the incident differently, leaves the model with incomplete information, makes permissions ambiguous and makes token consumption depend on how much text the user types. It also makes results hard to compare, quality hard to measure and auditing hard to keep consistent.
In an enterprise environment the user should not need to know the internal prompt, the model, the observability tool, the specific script or the format each integration expects. They should consume a previously designed operational capability: diagnose service degradation, analyze queue growth, verify a platform's functional health, compare behavior before and after a deployment, review certificate expirations, evaluate processing-window compliance or prepare the evidence for an incident.
The interface asks for controlled parameters:
Capability : Processing delay diagnosis
System : Transfer platform
Environment : Production
Period : Last hour
Flow : Outbound transfers
Context : Optional field
The user may add context, but that text does not replace the structured parameters. The interaction stops being "write a good prompt and hope the AI understands the problem" and becomes "select a capability that runs a governed, measurable, repeatable process".
The user should be an expert in their business or their platform. Prompt engineering belongs inside the solution.
Reference technical architecture
The solution can be organized into seven operational layers plus one transversal control for model access:
Reference architecture — 7 layers + transversal control
Two zones of uncertainty
A nuance that often gets missed: uncertainty does not live only inside the LLM. Semantic retrieval is probabilistic too. A vector search can return a runbook that looks similar but belongs to another environment, an earlier version, or an architecture that is no longer current. So it pays to separate the two zones explicitly:
Zones of the system
Identity and permissions · tool execution · metrics and queries · rules and thresholds · auditing · schema validation.
Semantic classification · document retrieval · hypothesis prioritization · interpretation and drafting.
The Model Gateway: governing AI consumption
In an enterprise architecture, applications should not connect directly to each model provider. The Model Gateway — or AI control plane — establishes a central boundary between internal systems and the models, and concentrates the decisions each team would otherwise reimplement on its own:
- Select provider and model by request type, and apply fallback when one is unavailable.
- Define the maximum token budget and manage caching of stable context.
- Mask or redact sensitive data before it leaves the perimeter.
- Validate the authorized model and region.
- Apply timeouts, retries and circuit breakers.
- Record model, version and parameters used on every call.
- Block unauthorized tools or instructions.
- Run quality evaluations and compute cost per diagnostic.
Its deeper benefit is that it lets you swap models without redesigning the operational capability. The model becomes replaceable; the contracts and the evidence remain.
The capability catalog
The catalog holds previously designed operational functions. Each capability declares its scope, version, tools, permissions, budget and approval policy — and, where relevant, how many additional checks the AI may propose:
capability:
id: diagnose_service_degradation
name: Diagnose service degradation
version: 2.2
allowed_environments:
- development
- testing
- production
required_roles:
- operator
- platform_engineer
deterministic_flow:
- get_service_metrics
- get_error_summary
- get_dependency_status
- get_recent_changes
- run_synthetic_test
optional_ai_proposals:
enabled: true
maximum_additional_checks: 2
require_policy_validation: true
ai_profile:
model_tier: medium
rag_enabled: true
maximum_context_tokens: 12000
approval:
read_operations: automatic
write_operations: human_required
This turns AI usage into a versioned, governed capability. The user does not freely choose which commands to run: they select an authorized function and the system determines the corresponding flow. The identity layer validates on every request the role, the authorized systems and environments, the permitted tools, and whether the case requires human approval. AI should never decide on its own what permissions a user has.
The deterministic tooling layer
Tools are narrow-scope functions that perform one concrete task:
get_cpu_metrics() get_queue_depth()
get_memory_metrics() get_consumer_rate()
get_disk_growth() get_database_connections()
get_process_status() get_recent_deployments()
get_certificate_expiration() get_error_summary()
run_connectivity_test() run_synthetic_transaction()
Each tool must define its allowed parameters, data types, timeout, authorized systems, risk level, the information it returns, the data it must mask, its retry policy, its audit evidence, its owner, its version and its maintenance criteria.
A tool should not accept an arbitrary command generated by the model. Instead of exposing
execute_command("whatever the AI generated"), you expose closed contracts:
get_disk_usage(server_id, filesystem)
restart_service(service_id, environment, change_ticket)
The first is read-only. The second needs additional permissions, operational-window validation and human approval or a previously authorized automatic policy.
The result should not be free text either:
{
"check": "queue_health",
"target": "service-a",
"status": "degraded",
"value": 1842,
"threshold": 500,
"timestamp": "2026-07-19T15:30:00-05:00",
"evidence": "queue-main",
"execution_status": "success",
"tool_version": "2.3.1"
}
The execution_status field is the easiest to forget and the most important. Without it you
cannot distinguish between four situations that demand different responses: the system is failing, the
control could not run, the source did not answer, or the result was partial. A diagnosis that confuses
"it is broken" with "I could not measure it" is worse than no diagnosis.
Normalizing the evidence
Tools return different formats. Before calling the model, a normalization layer must unify source, value, baseline, threshold, status and validity — and assign each piece a stable identifier that conclusions can cite:
{
"diagnostic_id": "DGN-2026-0719-0042",
"system": "transfer-platform",
"environment": "production",
"time_window": {
"start": "2026-07-19T15:00:00-05:00",
"end": "2026-07-19T15:30:00-05:00"
},
"evidence": [
{
"evidence_id": "metric.queue_depth.01",
"source": "metrics_backend",
"type": "metric",
"name": "queue_depth",
"value": 1842,
"baseline": 320,
"threshold": 500,
"status": "critical",
"execution_status": "success"
},
{
"evidence_id": "check.consumer_process.02",
"source": "application_script",
"type": "process_check",
"name": "consumer_process",
"value": "running",
"status": "healthy",
"execution_status": "success"
},
{
"evidence_id": "metric.consumer_rate.03",
"source": "metrics_backend",
"type": "metric",
"name": "consumer_rate_per_minute",
"value": 68,
"baseline": 120,
"threshold": 90,
"status": "critical",
"execution_status": "success"
},
{
"evidence_id": "change.deployment.04",
"source": "deployment_api",
"type": "change",
"name": "last_deployment",
"value": "release-2026.07.19.2",
"timestamp": "2026-07-19T14:47:00-05:00",
"execution_status": "success"
}
]
}
This layer reduces volume and strips noise before spending tokens. The AI does not need thousands of log lines when code can first produce the error count by code, the first and last occurrence, the deviation from the previous period, the affected components, the correlation IDs, representative samples and identifiers to fetch detail.
The flow: consumption, analysis and result
From request to result
-
User → Channel
Selects capability, system, environment and period. Writes no prompt.
-
Channel → API gateway
Sends structured parameters, not free text.
-
Gateway → Policy engine
Validates identity, role and environment. Returns the authorized scope.
-
Orchestrator → Tools
Runs the deterministic flow declared in the catalog, with bounded timeouts and retries.
-
Tools → Data
Query metrics, logs, traces and states. Return normalized evidence with its
execution_status. -
Orchestrator → Rules
Computes states and confirmed conditions: thresholds and baseline comparison.
-
Orchestrator → Retrieval
Brings only relevant, current documents, with source, version and score.
-
Model Gateway → AI
Selects the authorized model, caps tokens, redacts sensitive data and records the call.
-
Validator
Checks evidence, schema and authorizations. Approves or rejects before anything is shown.
-
Channel → User
Delivers the structured result: diagnosis, cited evidence and next steps.
Example: a processing delay
A user notices transfers are taking longer than expected. They do not type "analyze why the system is slow": they select the capability, the system, the environment, the period and the flow.
1. Validation
The platform confirms the user has read access to that system and environment.
2. Deterministic execution
The orchestrator queries process status, queue depth, ingress rate, consumption rate, disk space, database latency, recent errors, external dependencies, deployed changes and a synthetic test.
3. Rules
{
"overall_status": "degraded",
"confirmed_conditions": [
"queue_depth_above_threshold",
"consumer_rate_below_baseline"
],
"healthy_conditions": [
"process_running",
"database_available",
"disk_capacity_available"
]
}
The degraded state is not invented by the model. It comes from configured rules.
4. Knowledge retrieval
The engine retrieves the current runbook for queue growth, similar incidents, the flow architecture, the consumer's dependencies and the changes applied during the period. Each fragment must carry version, owner, environment and validity date.
5. Interpretation
The AI receives only normalized evidence, rule results, retrieved context, the request scope and the expected schema. It receives no full logs and no isolated screenshots.
6. Validation
The system checks that every finding cites a valid evidence_id, that the metrics exist,
that timestamps belong to the period, that recommendations are permitted, that hypotheses are not
presented as facts, that retrieved documents are current and from the right environment, and that the
response satisfies the schema.
7. Result
{
"status": "degraded",
"executive_summary": "The flow remains available, but its consumption capacity is below baseline.",
"confirmed_findings": [
{
"finding": "The queue grew from 320 to 1842 items.",
"evidence_id": "metric.queue_depth.01"
},
{
"finding": "Consumer rate dropped 43% versus baseline.",
"evidence_id": "metric.consumer_rate.03"
},
{
"finding": "The consumer process remains running.",
"evidence_id": "check.consumer_process.02"
}
],
"hypotheses": [
{
"hypothesis": "The consumer may be processing each operation with higher latency.",
"confidence": "medium",
"required_validation": "Compare internal per-stage duration."
}
],
"missing_evidence": [
"Internal per-stage latency before and after the last deployment."
],
"recommended_next_steps": [
"Analyze per-operation duration over the last 60 minutes.",
"Compare behavior before and after the deployment.",
"Review calls to the external validation dependency."
],
"automatic_actions_executed": [],
"human_approval_required": false
}
The values are illustrative, not measured. What matters is that the AI did not arbitrarily determine the state: it interpreted conditions computed through verifiable controls, and explicitly declared what evidence it was missing.
The prompt as an internal component
The prompt still exists, but it stops being the user's responsibility. The system builds a versioned instruction:
Analyze exclusively the evidence provided.
Rules:
1. Do not invent metrics, systems, dates or dependencies.
2. Distinguish confirmed facts, correlations and hypotheses.
3. Every finding must include a valid evidence_id.
4. Do not declare a root cause when the evidence only shows correlation.
5. Explicitly declare the missing evidence.
6. Propose only checks permitted by the catalog.
7. Do not propose write actions without approval.
8. Return the response using the defined schema.
This instruction can be versioned, tested, compared across models and audited. Quality stops depending on each operator's prompt-writing skill.
Model strategy by complexity
Not every request should consume the largest model. A cheap classifier picks the route before you spend advanced capacity:
Routing by complexity
Process status, available space, certificates nearing expiration, alert classification.
Metric correlation, period comparison, runbook selection, explaining a degradation with sufficient evidence.
Distributed incident, multiple services and dependencies, correlation of changes and traces, prioritized hypotheses.
Rollback, production restart, blocking an integration, capacity change, isolating a component.
The enterprise rule should be: do not use autonomy where an explicit sequence of steps can solve the problem.
Which kind of AI should you use?
Not every problem requires an autonomous agent or the largest model available.
Small models or classifiers
Suitable for classifying alerts, recognizing incident categories, prioritizing requests, extracting fields and selecting a known procedure. Fast, cheap and sufficient when the decision space is well bounded.
Embeddings and semantic retrieval
Useful for locating runbooks, documentation, configurations, postmortems and similar incidents. Here the primary function is not writing but finding relevant evidence inside authorized knowledge — remembering that this layer lives in the probabilistic zone.
LLM with RAG
Appropriate when you need to interpret technical evidence and contrast it with enterprise documentation. RAG reduces reliance on the model's internal knowledge, but does not eliminate the risks: OWASP warns that RAG and fine-tuning can improve relevance and still not fully resolve vulnerabilities such as prompt injection.
LLM with tool calling
This is the right fit for the triage described above. The model can select a function, fill in parameters and process the response. The function remains conventional code subject to authentication, authorization, validation and auditing.
Agentic AI
Reserve it for open-ended investigations requiring multiple steps, several sources and intermediate decisions. When the flow is predictable, a non-agentic solution is usually simpler, more controllable and cheaper. Google Cloud's architectural guidance distinguishes agents —suited to open-ended problems and complex flows— from assistive, RAG or single-call solutions, which are enough for predictable tasks.
How to contain hallucination risk
There is no instruction that guarantees a model will never be wrong. OWASP defines LLM-generated misinformation as false or misleading information that can appear credible, and recommends verified sources, automated validation and human oversight in critical contexts.
The architectural goal is not to promise the absence of errors. It is to prevent an incorrect response from becoming an untraceable conclusion or an unauthorized operational action.
Mandatory evidence
Every conclusion must cite concrete metrics, tools, periods and documents. When the information is not enough, the correct answer is: this cannot be determined with the available evidence.
Structured output
The response should conform to a defined schema, with correlations as their own category — distinct from facts and from hypotheses:
{
"observed_status": "degraded",
"confirmed_findings": [],
"correlations": [],
"hypotheses": [],
"missing_evidence": [],
"recommended_next_checks": [],
"confidence": "medium"
}
Structured output mechanisms let you require fields, types and enumerations. A schema does not guarantee the content is true, but it reduces ambiguity and makes automated validation easier.
Separate levels of assertion
- Fact: the queue holds 1,842 items.
- Correlation: it started growing after a deployment.
- Hypothesis: the change may have reduced the consumption rate.
- Recommended action: compare throughput and latency before and after.
A hypothesis cannot be presented as a root cause.
Restricted tooling
The model can only request functions included in the catalog, with read and write kept separate. Destructive actions require human approval, authorized windows and change control.
Versioned documentation
Runbooks used by semantic retrieval must declare version, owner, environment, validity and last-updated date. Retrieving an obsolete document with high similarity is still retrieving wrong information.
Post-hoc validation in code
Critical claims can be checked: that the cited metric exists, that the timestamp belongs to the period, that the system is within scope, that the document matches the environment, that the recommendation uses an authorized tool, and that every proposed cause has associated evidence.
Evaluation before production
The solution must be tested against historical incidents, synthetic scenarios, downed sources, partial results, cases with insufficient evidence, obsolete or contradictory documents, and attempts to request unauthorized actions. Measuring whether the answer sounds convincing is not enough. The NIST AI RMF proposes managing AI risk across the whole lifecycle, building trustworthiness criteria into design, development, use and evaluation.
Tokens are an architectural decision
Token consumption is not only a financial problem. Sending too much context increases latency, noise, contradictions, exposure of unnecessary data, the surface for malicious instructions, and the difficulty of reconstructing which evidence the model used.
I would not send gigabytes of logs to an LLM and ask what happened. Deterministic software must filter, aggregate and structure first:
- Filter before sending: patterns, counts and samples, not full logs.
- Retrieve selectively: relevant fragments, not every runbook.
- Bound the time window: only the incident's period.
- Avoid open-ended conversations: every diagnostic with a scope and a step limit.
- Reuse stable context: instructions, schemas and tool definitions can benefit from caching.
- Route by complexity: small models for simple tasks.
- Remove duplication: the same metric should not repeat across several sections of the context.
As a provider example, OpenAI documents reductions of up to 80% in time to first token and up to 90% in input token cost for requests compatible with its prompt caching. Real results depend on the model, the provider and how effectively context repeats.
Efficiency should not be measured as cost per million tokens, but as AI cost per correctly diagnosed incident.
AI must be observable too
Adding AI to operations creates another system that needs metrics, logs and traces. At minimum I would measure total analysis latency and each tool's duration, the number of model calls, input, output and cached tokens, cost per diagnostic, the percentage of responses with sufficient evidence, tools proposed, approved and rejected, errors and partial results, human escalations, hypotheses later confirmed, retrieved documents and their version, actions blocked by policy, and cases resolved without further intervention.
Modern platforms are already extending logs, metrics and traces to agent behavior: OpenTelemetry's semantic conventions for generative AI normalize the agent invocation span, tool execution spans, and token and duration metrics.
This makes it possible to answer real questions: is AI reducing triage time? On which incidents does it add value? Which tools fail most? What does each useful case cost? Which hypotheses tend to be confirmed? When should it abstain? Is it reducing work, or only generating more text?
Without that information, an AI observability solution becomes, paradoxically, one more unobservable component.
Where to start (and when not to)
Everything above describes the destination, not the first step. If I had to build this tomorrow, I would not start with a chatbot, with RAG, or by picking a model. I would start at layer 04: the deterministic tooling.
Evidence first, AI second
That layer has an advantage no other one has: it pays off even if the AI never arrives. A triage script, a reliable query or a synthetic test already cut operational time, standardize checks, remove manual steps, formalize tribal knowledge and produce auditable evidence.
But I would not build isolated scripts that print different text depending on who wrote them. My order would be:
- Define the operational questions I need to answer.
- Define a common evidence contract.
- Build the deterministic tools.
- Apply rules, thresholds and baselines.
- Integrate the flow into an orchestrator.
- Add retrieval of current knowledge.
- Introduce AI for interpretation and communication.
- Enable assisted selection only where a real need exists.
AI should not be the foundation. It should be an acceleration layer over a foundation that can stand on its own.
What is realistic in a quarter, and what at a year
Three months is enough for a first production capability if the scope is controlled: one critical platform and one or two recurring incident types. In that window you can deliver five to ten deterministic controls with a common result contract, integration with existing metrics and APIs, a triage runnable from a simple interface, rules to classify states, retrieval of one or two current runbooks, interpretation by a model, response validation, and auditing of executions, tokens and results — all restricted to reads.
What I would not attempt yet: full autonomy, automatic restarts, rollbacks, coverage of the whole organization, unbounded investigation, integration with all corporate knowledge, or automatic root-cause declarations.
The first quarter should prove exactly one thing: that triage time can be reduced using reliable evidence and a repeatable flow.
At twelve months a more mature platform is within reach: more capabilities and systems, permissions per environment and action, versioned knowledge, baseline comparison, continuous evaluation, model routing and integration with the ticketing tool. Only then does it make sense to propose low-risk, reversible, previously authorized remediations — retry a synthetic test, escalate an incident, open a ticket with the evidence attached. The boundary does not change: autonomy is not granted because the technology can do it, but because the organization can govern it.
The trap layer
The trap layer is not the model. It is the combination of orchestration, normalization, tool governance and lifecycle management of the controls. On the diagram it looks simple: a function runs, returns a result and the AI explains it. In production the hard questions show up:
- What happens if the script answers partially, or if the source is down?
- How do I tell a system failure from a measurement failure?
- Which version of the control ran, and who may query production?
- Which data must be masked? What timeout and how many retries are valid?
- Which source wins when two controls contradict each other?
- Who maintains the contract when the platform changes, and who owns each tool?
The model is relatively interchangeable. Reliable integration with real systems is not. A catalog of twenty scripts does not amount to an enterprise capability either: the capability exists when each control has an owner, a version, a contract, permissions, a timeout, evidence, success criteria, error handling, auditing and maintenance.
The hard part is not getting the AI to call a function. It is guaranteeing that the function is still reliable six months later.
When it is not worth it
Not every team needs this. It can be over-engineering when several conditions coincide: few systems, a simple architecture, infrequent and easily reproducible incidents, a small stable team, low dependence on specific specialists, short up-to-date runbooks, limited operational impact and an acceptable resolution time. In that scenario you do not need AI: you need better controls, scripts and documentation.
Nor should the decision rest on "the company needs to use AI". That is a bad architectural reason. I would use this relation as the criterion:
Potential value = frequency × complexity × impact × repetitive manual work
Criticality can break any frequency threshold: a system can have few incidents a year and still justify advanced controls if each interruption compromises critical operations or regulatory compliance. I would also look at technical hours consumed, how many people get pulled in, how long it takes to identify the affected component, how often the same checks are repeated, and the risk of a wrong diagnosis. When only one person knows what to look at, there is already an opportunity to formalize — but that does not mean the first solution should be AI.
The one capability I would build
If I could build only one: automated end-to-end degradation triage. I would not start with postmortem generation, document lookup or alert summarization: those are useful, but they happen afterwards or work on information someone already gathered.
I would pick it not because it replaces analysis, but because it removes the repetitive part: opening different tools, running the same queries, lining up timestamps, comparing metrics and rebuilding context. My time should go to interpreting anomalous behavior and deciding, not to collecting by hand what a system can gather consistently.
The expected answer would not be "the system may be experiencing degradation", but a bounded conclusion: what is available, what degraded and by how much, what changed nearby in time, what is not yet proven, and what the next check is. The AI would not have resolved the incident for me. It would have removed a significant part of the collection work and handed me the problem properly bounded.
Conclusion
The most solid enterprise use of artificial intelligence is not replacing scripts, metrics, rules or procedures. It is connecting them without moving the operational core outside what is verifiable, and leaving approval of critical decisions where it always belonged: with the specialist.
AI delivers speed precisely because it does not need to replace what already works. Its value disappears the moment it is allowed to invent system state, execute actions without limits, or replace checks that conventional software performs more precisely.
The question, then, is not whether an organization already uses AI in operations. It is whether that AI is connected to verifiable evidence, or only to a more sophisticated version of the same incomplete dashboard.
Automate how evidence is obtained. Accelerate how it is interpreted. Never delegate the truth.
Frequently asked questions
Should AI determine whether a system is healthy?
No. State must be computed with deterministic tools, rules, thresholds and verifiable evidence. AI interprets and communicates those results.
Can AI decide which tool to run?
In known flows it does not need to: the orchestrator follows an explicit sequence. In open investigations it may propose additional checks, but the policy engine validates permissions, parameters and risk before they run.
Does RAG eliminate hallucination risk?
No. It reduces reliance on the model's internal knowledge, but it can also retrieve incorrect, outdated or wrong-environment documentation. Sources must be versioned and validated.
When is agentic AI appropriate?
In open-ended investigations requiring several steps and intermediate decisions. When the flow is known and repeatable, a deterministic sequence is simpler, more controllable and cheaper.
Why are tokens an architectural decision?
Because unnecessary context increases cost, latency, noise, information exposure and auditing difficulty. Evidence must be filtered and structured before calling the model.
References
- Google SRE. Monitoring Distributed Systems.
- OpenTelemetry. What is OpenTelemetry?
- OpenTelemetry. Semantic conventions for generative AI.
- OWASP. LLM01: Prompt Injection.
- OWASP. LLM09: Misinformation.
- NIST. AI Risk Management Framework.
- Google Cloud. Choose a design pattern for your agentic AI system.
- OpenAI. Function calling.
- OpenAI. Structured outputs.
- OpenAI. Prompt caching.