Software teams sometimes talk about reliability as if it were a capability you add near the end. Build the product, add tests, turn on monitoring, and call it reliable.
That approach can produce a lot of green checks without producing much confidence.
Reliability is the result of a system. Product boundaries, architecture, evaluation, deployment, observability, authority, and recovery all have to support one another. A weakness in any one of them can invalidate the signal from the rest.
Start with the boundary
Before testing a system, define what it is responsible for.
What may it change? Which inputs can it trust? Which decision belongs to a person? What happens when a dependency is slow, unavailable, or inconsistent? Which parts of the workflow are safe to retry?
These are product questions and architecture questions at the same time. If the boundary is vague, the tests will be vague. If authority exists only in a prompt or a meeting note, it is not a dependable control.
A good boundary turns "be safe" into behavior that can be evaluated.
Builder is not tester
The person or agent that built a system knows how it is supposed to work. That knowledge is useful during development and dangerous during evaluation.
Builders tend to exercise the paths they designed. They know the correct language, the expected sequence, and the assumptions that make the happy path succeed. Even with good intent, they can reproduce their own blind spots.
Independent evaluation changes the posture. The evaluator asks how the system could be misunderstood, pressured, interrupted, or used outside the path the builder imagined.
For an AI agent, that might mean conflicting instructions, missing authority, partial tool failure, misleading context, or a request that should stop at a human review gate.
For a conventional application, it might mean duplicate submissions, stale data, an integration timeout, a failed deployment, or a restore performed against newer records.
The principle is the same: test the boundary, not the builder's confidence.
Evidence makes a result reviewable
A score is a summary. It is not evidence.
When a test matters, preserve enough context to understand:
- What behavior was expected
- Which conditions were present
- What the system did
- Which rule produced the result
- What changed between runs
This does not mean logging everything. Indiscriminate logs create noise and privacy risk. Evidence should be intentional, scoped, and attached to the decision it supports.
The goal is reproducibility. Another person should be able to inspect the result without relying on the memory of whoever ran the test.
Validate the deployed system
A successful build proves that an artifact was produced. It does not prove that the intended artifact is serving real traffic, that assets load, that configuration is correct, or that an old process did not survive the release.
Deployment validation should check the operating system from the outside:
- Is the expected build present?
- Does the public route reach the intended origin?
- Are important assets available?
- Does the health signal represent useful behavior?
- Can the service restart cleanly?
- Did the change leave unrelated systems untouched?
This is where release engineering becomes part of product reliability. The deployment is not finished when the command exits. It is finished when the system is verified in its real path.
Recovery is a feature
Backups are not recovery. A backup becomes useful only when it can be restored safely and the team knows what the restore will do.
Recovery design should answer:
- What state can be recreated?
- What state must be preserved?
- Can a restore overwrite newer work?
- How will we know the recovered system is correct?
- What is the path back if the attempted recovery makes things worse?
These questions belong in product and architecture discussions before an incident.
The same applies to rollback. A rollback plan that says "undo the change" is not a plan. A useful rollback identifies the exact state to restore, the scope that may change, and the checks that confirm unrelated systems remained intact.
Reliability needs an owner
Tests, dashboards, and backups cannot own an outcome. Someone still needs to decide what the signal means and what happens next.
That ownership does not have to be centralized forever, but it must be explicit. A warning with no response path is decoration. A failed evaluation with no release gate is trivia. An incident log with no recovery authority is history, not control.
Reliable systems make responsibility visible.
A practical reliability review
Before calling a system dependable, I ask:
- Are its responsibilities and authority clear?
- Have we tested failure conditions, not only expected use?
- Is evaluation meaningfully independent from implementation?
- Does each important result carry reviewable evidence?
- Have we validated the deployed path?
- Can we detect degraded behavior?
- Can we recover without casually destroying newer work?
- Does a person own the response when a gate fails?
No single answer creates reliability. Together, they create a system that can earn trust.
Related work: Agent Reliability Toolkit and OperatorKit.