Agentic AI
Our agents kept answering after we deleted their knowledge
A one-line deploy config stripped the knowledge packs out of our agent runtime. Nothing errored, no metric moved, and the agents went on answering fluently for as long as they were deployed. We found it because one of them told us.
On 21 July we asked our operations agent to summarise a governance rule it had been given. It replied that the rule wasn’t in its grounding. The agent was correct — and it had been correct, silently, for as long as it had been running in the cloud.
This is a field note about a failure with no error message, because that class of failure is the one worth writing down.
What actually happened
Our agent runtime deploys as a cloud function. Its deploy config carried an ignore list, and that list contained "*.md". The intent was ordinary housekeeping: keep three documentation files — a README, a security doc, a deploy runbook — out of the shipped bundle.
The knowledge packs are also .md files. They live in knowledge/<agent-id>.md, and they are the canon each agent loads into every cognition call: the constitution, the standards, the laws it is supposed to reason from. One glob pattern, written for three documentation files, removed all of them from the bundle.
Locally, everything was present. In the cloud, the directory was empty.
Why nobody noticed
The loader looks like this:
KNOW_CACHE[agentId] = existsSync(f)
? '\n\n=== YOUR KNOWLEDGE PACK (BIXSO canon …) ===\n' + readFileSync(f, 'utf8')
: '';
A missing pack is not an exception. It is an empty string. The system prompt assembles as persona + grounded contract + '' + memory — structurally valid, no warning, no metric, nothing to alert on. The agent then does what a capable model does with a well-formed prompt and no reference material: it answers. Fluently, in the right voice, in the right format.
That is the part worth sitting with. A retrieval system that breaks usually goes quiet — empty results, a timeout, a stack trace. A grounding system that breaks gets louder. It removes the constraint that was slowing the model down and leaves the confidence intact. The output looks better to a casual reader in the moment it becomes worthless.
How we found it
Not from monitoring. From an answer.
We have a standing rule that every agent operates under a grounded contract: assert what the sources support, and say so plainly when they don’t. So when the agent was asked about a rule that was no longer in its context, it said the rule wasn’t in its grounding — instead of reconstructing a plausible version from training data, which is what the same model would happily have done without that contract.
The honesty rule wasn’t written as a monitoring tool. It turned out to be the only instrument that could see this failure at all.
The part we got wrong
We had already built a test suite for exactly this fear. It checks, per agent, whether knowledge/<id>.md exists — and it was written after an earlier incident where three agents were found running with no canon because their files sat under different names.
That suite reads the local filesystem. The bug was in the deploy bundle. Local passed. The cloud was empty. The check and the fault were one layer apart, and the check reported green the entire time.
It is the same blind spot as the config that caused it: we verified what we had, not what was running.
Where it stands
The ignore list now names the three documentation files explicitly instead of globbing. As of today the runtime carries 14 agent manifests and 14 knowledge packs, matched one to one.
The silent-empty loader is unchanged. A missing pack still resolves to an empty string, and an agent with no canon still starts up and answers. We patched the cause, not the class — and we have no automated check that reads the deployed bundle rather than the repository. That gap is open.
What this changes on the canvas
In the Business Solution Canvas, AI4 asks what data feeds the system and how it evolves. It is easy to treat that as a design-time question — name the corpus, note the pipeline, move on.
This is the operational form of the same question, and it is harder: can you show that the grounding is present where the system is actually executing, right now? Not in the repo, not in the design doc, not in the local test run. In production, on this deploy, today.
If the answer relies on a file being there, then something has to fail loudly when it isn’t. Otherwise the system’s most convincing output is the one produced with nothing behind it.
One thing to take away: an AI system that loses its grounding doesn’t go quiet — it keeps answering, with the same confidence and none of the substance. Test what is running, not what is committed.
BSC is BIXSO’s open framework for working through business model, analysis and agentic AI as one loop — bixso.ai/bsc.