The Automation Confidence Gap: When Should AI Be Allowed to Change Production?

Getting an AI to suggest an infrastructure change is easy now. Deciding whether to let it run in production is another matter. A model can read logs, spot an unhealthy cluster, and produce a runbook that looks perfectly reasonable. It can also pick the wrong cluster, overlook a dependency, or recommend a change outside an approved limit.

I’ve spent enough years working with middleware in regulated environments to be wary of the phrase ‘the recommendation looks right.’ That is a good reason to investigate. It is not permission to execute.

We tend to talk about automation as though there are only two settings: A person does the work, or a machine does. Production isn’t that simple. Some jobs are routine enough to run unattended. Others need a second pair of eyes even when the steps are familiar. The distinction should depend on the evidence available for this particular change, not on how many times a runbook succeeded last year.

A Sensible Change Can Still be the Wrong Change

Suppose an AI recommends increasing a production cluster from six instances to ten. Maybe that’s exactly what the application needs. Before allowing it, though, ask: Did traffic actually increase? Is this the right environment? Who authorized the change? Is ten within the approved limit? What else shares that cluster? And can we undo it if the result is worse?

Passing syntax checks won’t answer those questions. Neither will an impressive explanation from the model. What matters is whether the platform has checked the facts.

Seven Things To Check

1. Intent certainty. Does the system know the exact target, action, and expected outcome? “Add more production servers” is not precise enough.
2. Policy compliance. Are the requester, environment, change window, and resource limits authorized? A policy failure stops the change, regardless of any score.
3. Evidence quality. Do latency, traffic, CPU, and queue depth tell the same story? When telemetry disagrees with itself, that is a reason to stop, not to average.
4. Runbook maturity. Has this workflow succeeded under comparable conditions? A runbook edited yesterday has not earned the same trust as one with a solid record.
5. Blast radius. Restarting one stateless instance is different from touching a shared platform that supports dozens of applications.
6. Rollback readiness. A recovery page in Confluence is not a rollback test. For a change that needs recovery, the procedure must be usable and fast enough to matter.
7. Change novelty. The same automation aimed at a new environment or given unfamiliar parameters is not really the same operation.

The Score Should Decide What Can Run

Turn those checks into a confidence index from 0 to 100. Not a prediction that a change has, say, an 85% chance of success. Just a practical way to assign execution authority using evidence that can be verified. The example bands below are starting points, not universal rules.

One nonnegotiable rule: A high score cannot override a failed gate. Missing authorization, a violated policy, or an unsafe recovery plan can block a change outright. And the AI should not permit itself. A separate policy service has to enforce those limits; the model can propose and explain, but it doesn’t get to approve its own work.

Figure 1. Seven checks inform the confidence index. Mandatory policy gates can still block execution.

What This Looks Like for Certificate Renewal

Take a certificate serving 18 application endpoints that expires in 21 days. When the alert first arrives, the platform may not know who owns the endpoints or what depends on them. I’d give it little authority at that point: investigate and recommend, don’t deploy.

Then it confirms ownership, maps the endpoints, checks the replacement certificate, tests the deployment outside production, and verifies that recovery will work. Those checks may justify preparing a rollout plan, but a person still approves the first production step. The automation updates one endpoint as a canary, checks TLS and application health, and only then moves to controlled batches. If a health check fails or an unexpected dependency appears, the rollout stops.

The scores in that example would be illustrative. The important part is that permission changes when the evidence changes.

Keep the Reason, not Just the Command

A job log tells you what ran. After an incident, someone is going to ask why the platform was allowed to run it. Capture the trigger, the evidence used, the affected systems, the policy checks, who approved what, and what happened during validation and rollback. Otherwise, the most important part of the decision disappears.

Confidence also has an expiration date. A workflow that was safe before a dependency change may not be safe afterward. New parameters, stale rollback tests, worsening failure rates, or missing telemetry should reduce its authority on the next run. Yesterday’s success is useful evidence. It is not permanent permission.

The Point is Not to Remove People

I don’t measure a good automation program by how many changes it makes without an engineer. I care whether the routine work moves quickly and the risky work stops in the right place. Let AI help diagnose and plan. Let deterministic tooling enforce the rules and run the approved steps. Put evidence between the two.

That is the level of autonomy worth pursuing: not maximum automation, but the most an operation can safely justify right now.