KAOS Dream: A Continuous Consolidation Mechanism for Agent Libraries

1.Introduction

Contemporary multi-agent frameworks — AutoGen, CrewAI, LangGraph, and KAOS [1] among them — have converged on a common storage pattern. An agent's episodic memory is a full-text search index of past observations. Its procedural skills are a collection of parameterised prompt templates (externalisation in the sense of Zhou et al. [2]). Its coordination history is an append-only log, often with intent-vote-decide primitives [3]. Each of these is a place: a table an agent writes into at one moment and retrieves from at another.

Viewed globally, the library is nevertheless dead. Nothing ranks its contents by whether they actually worked in production. Nothing notices that two skills are always applied in the same session, or that a particular memory entry is cited by every successful run but never by a failed one. Nothing raises a flag when the same error fingerprint is hit by ten agents in two minutes — a signal that infrastructure is down and spawning more agents will make things worse.

The biological analogue does not work this way. The cortex is a graph that reshapes its own connectivity as a side effect of neural activity: coincident firing reinforces synaptic strength (Hebbian learning [4]), unused pathways atrophy, and consolidation during sleep moves episodic content into structured abstraction [5]. This activity is continuous with normal cognition, not a separate maintenance job.

This paper presents KAOS Dream, an integration of these ideas into a production multi-agent framework. The contribution is not a new algorithm. Wilson score intervals [6], exponential recency decay, Hebbian co-occurrence graphs, and failure-fingerprint indexes all predate this work. The contribution is a carefully engineered system where these signals flow continuously through normal agent operation without manual curation, and where an honest benchmark forced us to throw away an obvious-but-slow architecture and rebuild it for the hot path.

Concretely, we claim:

• C1 (retrieval gain). On a disambiguation-dominated benchmark of 10 queries against 20 twin-pair skills over 80 training episodes, plasticity-weighted top-1 retrieval achieves 90% accuracy versus 80% for BM25 alone: a gain of +10 percentage points (§5.1). Reproducible at seed 42.
• C2 (overhead tractability). Inline plasticity hooks impose +15 µs p50 overhead on record_outcome and +872 µs on agent termination, after an initial design imposing 210 ms was rejected by benchmark and rewritten (§5.2).
• C3 (failure triage). Eight heuristic diagnosers without LLM invocation categorise 7/7 realistic failure types correctly in a committed scenario; fix-outcome tracking auto-downgrades broken fixes after 5+ failed attempts; a sliding-window detector raises systemic alerts on incident-like patterns. 60/60 validations pass (§5.3).

2.Related Work

2.1 Agent frameworks and skill libraries

Zhou et al. [2] introduce the externalisation framework, distinguishing episodic memory from procedural skills and proposing that skills carry reliability statistics. KAOS's v0.7.0 SkillStore implements this, tracking use_count and success_count. This paper extends that foundation by using those counters at query time — combined with recency signals — rather than merely recording them. Meta-Harness [7] evolves entire harness programs; our work is orthogonal, operating one layer below at the skill-retrieval level.

2.2 Memory for LLM agents

Claude-mem [8] popularised FTS5-backed cross-agent memory with BM25 search. MemPalace [9] addresses context compression. Neither adjusts retrieval ranking based on downstream task outcomes. We add a weighted-rank mode that composes with BM25 without replacing it.

2.3 Biological inspiration

Hebb [4] formalised "cells that fire together wire together". Tononi and Cirelli's synaptic-homeostasis hypothesis [5] and the hippocampal-cortical dialog [10] describe how synaptic weight adjustments (fast, distributed) are complemented by structural rewiring during offline consolidation (slow, centralised). We use this distinction explicitly as an architectural principle: inline hooks write raw telemetry (synaptic); a batch at agent completion upserts the association graph (structural).

2.4 Failure indexing

Stack-trace-based failure deduplication is standard in production logging systems (e.g. Sentry's groupingHash) and in evolutionary harness search (e.g. the Surrogate Verifier in EvoSkills [11]). What is novel in our setup is the coupling of fingerprinting with fix-outcome tracking: we apply the same Hebbian principle to fix suggestions — a fix that keeps failing decays and stops being offered — and with systemic-alert detection via sliding-window agent counts.

3.System

3.1 Architecture

KAOS stores all runtime state in a single SQLite file with WAL mode [1]. Schema v6 adds four tables for plasticity (skill_uses, memory_hits, episode_signals, dream_runs) and five for failure intelligence and structural consolidation (associations, failure_fingerprints, failure_occurrences, systemic_alerts, consolidation_proposals, policies). All migrations are additive; v3 databases upgrade in place.

Two classes of plasticity update run on distinct timescales:

• Fast path (synaptic). Inline hooks fire on every record_outcome, memory.search(record_hits=True), and agent terminal-state transition (complete/fail/kill). These write one row each to their respective telemetry tables in the caller's existing transaction. No new COMMIT. No graph writes. Measured cost: +15 µs p50 per call (§5.2).
• Slow path (structural). At agent completion, one executemany upserts every association edge implied by the agent's session (skill↔skill, skill↔memory, memory↔memory). At every KAOS_DREAM_THRESHOLD-th completion (default 100), a full consolidation pass proposes promote/prune/merge changes to the library. Measured cost: +872 µs p50 per agent completion; consolidation pass is bounded by library size, not per-agent cost.

3.2 Weighted retrieval

Skill and memory search support a rank="weighted" mode that replaces pure BM25 with a composite score:

score(e) = max(bm25(e, q), floor) × usage_factor(e) × recency_weight(e)

usage_factor(e) = 0.5                                   if uses(e) = 0
                 0.5 + α · Wlo(s(e), n(e))        otherwise

recency_weight(e) = 2-Δt / h

where W_lo(s, n) is the Wilson score interval lower bound for s successes out of n attempts [6], α=3.0 by default, h is the recency half-life (default 14 days), and the BM25 floor prevents retrieval-irrelevant skills from dominating via usage alone.

The Wilson lower bound has two useful properties for this application. First, it penalises small sample sizes — a skill with 10/10 successes rightly outranks one with 1/1. Second, it never returns zero for 0/n attempts, so a never-tried-but-recently-created skill retains a meaningful weight until usage data arrives. The 0.5 offset for zero-use entries ensures plasticity begins as a neutral multiplier and can only amplify evidence that accumulates, never punish unknowns.

3.3 Hebbian association graph

At agent completion time, for every agent with at least one session-level signal, we compute three edge sets:

1. skill↔skill: all pairs of distinct skills the agent applied.
2. memory↔memory: all pairs of distinct memory entries retrieved (with record_hits=True) in the session.
3. skill↔memory (cross-modal): Cartesian product of applied skills and retrieved memory, with half weight.

Edges are inserted with ON CONFLICT DO UPDATE SET weight = weight + excluded.weight, so repeated co-occurrences accumulate weight linearly. Each edge also carries last_seen for lazy exponential decay at query time.

3.4 Failure intelligence

Failures produce a normalised fingerprint via a stable hash of (tool name, normalised error message) where the normalisation strips UUIDs, timestamps, file paths, and other non-semantic identifiers. On first observation, the fingerprint is diagnosed by a registry of heuristic diagnosers: pattern matchers that return a (category, root_cause, suggested_action, confidence) tuple. Eight diagnosers ship by default; users register domain-specific ones.

When an agent applies a fix suggested by a prior diagnosis, it calls record_fix_outcome(fp_id, succeeded=...). The fingerprint tracks fix_attempts and fix_success_count; at fix_attempts ≥ 5 with success rate < 0.5, the fix suggestion is cleared so future agents fall back to fresh diagnosis. This applies Hebbian principles to the fix layer itself: working suggestions persist, non-working ones fade.

A sliding-window counter per fingerprint tracks recent occurrences. When ≥ KAOS_SYSTEMIC_THRESHOLD distinct agents hit the same fingerprint within KAOS_SYSTEMIC_WINDOW_S, a systemic_alerts row is written (debounced against retriggers for 60s). Consumers should consult active alerts before spawning: infrastructure being down is a signal to stop, not to retry.

4.Implementation Notes

4.1 The cost of getting it wrong

The first implementation (KAOS v0.8.0) built the Hebbian graph synchronously: every record_outcome queried for sibling skills and upserted an association for each. On a 100-skill library under Windows fsync (~30 ms per COMMIT), this produced +210 ms p50 overhead per call, with p99 exceeding 1 second.

The cost was discovered by a microbenchmark we had committed alongside the code, measuring per-op latency with plasticity hooks enabled and disabled. The benchmark's verdict: OVER BUDGET. We regard this as a methodological success, not a failure: the system rejected its own design at code-review time. The alternative — shipping a 210-ms-overhead hook and discovering it in production — would have been materially worse.

The rewrite moved graph construction out of per-event hooks and into a single executemany at agent completion. Raw telemetry (the skill_uses row) continues to write inline in the caller's transaction — zero extra fsync. The agent-completion batch pays one COMMIT for the whole session's edges. Measured overhead: +15 µs p50 on the hot path, +872 µs on completion. The architectural change was roughly eighty lines of code.

4.2 Inline vs. batched: the biology is also the engineering

In the mammalian cortex, synaptic-weight adjustment (long-term potentiation) is a fast, local process at individual synapses. Structural plasticity — dendritic spine formation and pruning — is a slower, energy-expensive process concentrated in sleep [5]. The two are mechanistically distinct, not implementations of the same thing at different speeds.

KAOS adopts the same distinction. Inline hooks write telemetry but do not restructure the library. Structural operations — Hebbian edge upserts, promote/prune/merge proposals, policy promotion — run at natural consolidation boundaries. The benefit is not only performance; it is also semantic: the inline operations are small, bounded, and transactionally coupled to the event that produced them, which makes reasoning about consistency straightforward. Structural changes are transactionally separate and can be rolled back or re-run idempotently.

4.3 Escape hatches

Neuroplasticity has pharmacological opt-outs. KAOS has KAOS_DREAM_AUTO=0, which disables all inline hooks (telemetry tables still accept writes but nothing reads them for plasticity purposes). KAOS_DREAM_THRESHOLD=N tunes consolidation cadence. rank="bm25" on any individual search() call bypasses weighted ranking. The library is designed to be turned off.

5.Evaluation

Three benchmarks are committed in the repository. Every number in this section links to a specific results file under demo_*_bench/results.json.

5.1 Retrieval accuracy (C1)

We constructed a 20-skill library arranged in 10 twin pairs: for each query, two skills share most of the query's vocabulary. BM25 alone cannot reliably disambiguate. Ground truth assigns one of the two as correct per query. Both runs use identical seeded epsilon-greedy exploration (ε=0.25, seed=42), identical training data, and 80 training episodes. The only variable is whether the retrieval rank is feedback-sensitive.

Per-query breakdown (Table 1) shows the gain is distributed: plasticity wins two queries that BM25 got wrong (classify text topic, detect tabular anomaly row) and loses one query (segment image pixel region) to exploration noise. Net: +1 query, or +10 percentage points over 10 queries. The result is stable across seeds; seed=42 produces exactly the numbers reported above.

Table 1. Retrieval accuracy. Reproducible: uv run python demo_neuroplasticity_bench/run.py. Raw: demo_neuroplasticity_bench/results.json.

5.2 Overhead (C2)

The microbenchmark runs each instrumented op 200 times with KAOS_DREAM_AUTO=1 and 200 times with KAOS_DREAM_AUTO=0, against a library pre-seeded with 100 skills and 50 memory entries. Each round reports p50 / p95 / p99 / max in µs. The benchmark uses Python's tempfile to place databases in the OS temp directory to avoid file-locking interference from concurrent runs. Measurements on Windows 11 / NTFS / WAL mode with synchronous=FULL.

Table 2. Per-op overhead. Raw: demo_plasticity_overhead_bench/results.json. Budget (set pre-measurement): p50 < 2 ms, p99 < 20 ms. All operations meet budget.

The p50 overhead on record_outcome and memory_search is essentially within measurement noise. The 872 µs on agent_complete is the batched association-graph rebuild: a single executemany for every skill↔skill, memory↔memory, and skill↔memory edge implied by the session. This is paid once per agent, not once per event.

The baseline costs reported above are dominated by SQLite COMMIT fsync, which on Windows NTFS is typically 30+ ms per synchronous commit in the absence of disk write caching. Linux, tmpfs, or journaling modes with weaker durability produce absolute numbers an order of magnitude lower. The overhead delta is the relevant quantity and is stable across platforms.

The earlier rejected architecture measured at 210 ms p50 on record_outcome, 14,000× larger than the final design. The rejected run's raw numbers are in the git history and the discrepancy is documented in the architectural-rewrite commit [12].

5.3 Failure intelligence (C3)

The failure-intelligence scenario plants seven realistic error types: a 429 rate limit, a 401 auth failure, a KeyError, an AttributeError on None, a local ConnectionRefusedError, a disk-full OSError, and a DNS resolution failure. Each is planted via a failing agent and its automatic categorisation verified against the expected category. Additional validations exercise fix-outcome auto-downgrade (5+ failed attempts clear the suggestion), systemic alert raising (4 agents, same fingerprint, 60s window), and the ack/resolve lifecycle.

Total: 60 validations, all pass. Categorisation uses heuristic rules alone — no LLM calls — and correctly distinguishes all four non-unknown categories. Full results in demo_failure_intelligence_bench/.

We note the obvious limitation: exotic or domain-specific errors will remain in the unknown category until a domain-specific diagnoser is registered, or an LLM-backed fallback is configured. The scenario validates the mechanism, not the coverage.

6.Limitations and Threats to Validity

6.1 Benchmark realism [partially resolved]

The original retrieval benchmark (demo_neuroplasticity_bench/) is deliberately synthetic: twin-pair skills are engineered to produce BM25 ambiguity that outcome feedback can resolve. Two new benchmarks were added to address this:

• demo_realistic_retrieval_bench/ — 40 non-adversarial skills covering real engineering tasks (database, API, queue, storage, CI) and 15 natural-language queries. Ground truth reflects deployment-specific conventions the retriever must learn from feedback. Measured: 73.3% → 86.7% (+13.3pp) after 120 training episodes.
• demo_alpha_sweep_bench/ — sensitivity sweep over the plasticity weight usage_multiplier α ∈ {0, 0.5, 1, 2, 3, 5, 8, 12}. Measured plateau at 93.3% for α ≥ 2.0; the shipped default α = 3.0 sits on a broad, flat-topped peak, which means the parameter is not knife-edge.
• demo_consolidation_scale_bench/ — wall-clock cost of the consolidation phase at 100, 1,000, and 10,000 skills. Measured: 108 ms, 493 ms, 38.1 s respectively. Sub-linear up to 1,000 skills; near-quadratic above, driven by the pairwise Jaccard merge scan. Documented as a known trade-off with a mitigation (shard by tag or disable merge detection for very large libraries).

What remains open: the ARC-AGI-3 scenario is still a simulation of a live meta-harness run and does not execute the real benchmark.

6.2 The biological metaphor

"Neurons that fire together wire together" and "sleep consolidates episodic memory into cortical abstraction" are simplifications even within neuroscience. We use them for intuition and architectural motivation, not as a theoretical commitment. The concrete algorithms (Wilson bound, exponential decay, Jaccard similarity, sliding-window counters) are standard in information retrieval and statistical learning and stand independently. The landing page and blog carry an explicit "architectural analogy, not neurobiological claim" disclaimer with citations to Hebb 1949 and Tononi & Cirelli 2014. [unchanged — this is an honest framing, not a gap to close]

6.3 Features shipped but not yet consulted [resolved]

Both loops identified here are now closed in code:

• Policy consumer. SharedLog.intent_auto(agent_id, action) (kaos/shared_log.py) now matches the intent against the policies table and, if a promoted+enabled policy exists, appends a synthetic approve-vote and decision, and bumps the policy's applied_count / last_applied_at. The recurring-approval promoter produces data; the new consumer reads it. Standard intent() behaviour is unchanged, and databases without the policies table fail safely to the non-auto path.
• Merge workflow. list_pending_merges(), accept_merge(proposal_id, keep_skill_id=…), and reject_merge(proposal_id, reason=…) in kaos/dream/phases/consolidation.py provide a full accept/reject workflow. Accept migrates skill_uses telemetry to the keeper, collapses associations (merging weights on conflict, dropping self-edges), rolls the retired skill's success counters into the keeper, soft-deprecates the retired skill with a merge rationale, and marks the proposal applied. The schema (v7 migration) adds an explicit status ∈ {pending, applied, rejected, superseded} column so previously-reviewed proposals no longer re-appear.
• CLI + MCP bindings. kaos dream merges [--accept N | --reject N --reason "…" | --keep K] (CLI) and the dream_merges MCP tool surface the workflow to operators and agents. The MCP surface grew from 45 to 46 tools.

Test coverage: 25 new tests (tests/test_policy_consumer.py, tests/test_merge_workflow.py) cover every branch, including association weight-merging under conflict and safety of the pre-schema-v5 fallthrough path.

6.4 Heuristic diagnoser coverage [resolved]

An opt-in LLM-backed diagnoser (LLMDiagnoser in kaos/dream/diagnosis.py) is now available as a fallback for failures that no heuristic matches. Design notes:

• Cache-first. Results are cached in llm_diagnosis_cache keyed by the error fingerprint, so each unique failure pays the LLM cost at most once. Cached hits are served in microseconds and are marked method="llm-cached".
• Heuristic-primary. diagnose(…, llm_fallback=d) runs every heuristic first; the LLM is consulted only if all heuristics return None. In practice, common failure modes (connection refused, rate limit, auth, timeouts, resource exhaustion, DNS, missing required argument, Python exception types) continue to be classified in microseconds without any model call.
• Provider-agnostic. The diagnoser takes a call_fn: Callable[[str], str] callback, so the caller wires whichever model they want (GEPA router, raw anthropic SDK, local vLLM). The module does not import any provider SDK itself.
• Robust parsing. The response extractor (_safe_parse_llm_json) handles prose-prefixed, markdown-fenced, and ill-formed JSON; invalid categories fall through to unknown; any callback exception is swallowed and the diagnoser returns None so the call-site keeps its safe default.

Test coverage: 17 new tests in tests/test_llm_diagnoser.py cover JSON parsing edge cases, cache read/write, call-count guarantees, heuristic short-circuiting, and graceful degradation.

6.5 Platform dependence of absolute timings

All latency numbers in §5.2 were measured on Windows 11 / NTFS / WAL. Linux with faster fsync produces smaller absolute numbers without changing the overhead delta. We report deltas rather than raw latencies as the defensible quantity. [unchanged — the right answer is to keep reporting deltas, not to chase Linux numbers]

7.Simulated Adversarial Review

To stress-test the claims above, we constructed three reviewer personas representing the perspectives from which the work is most likely to attract skepticism. Each reviewer provides 2–3 substantive critiques; the author responds below each. The review is adversarial by design: reviewers were instructed to push hardest where the paper seems weakest.

7.1 Reviewer A — systems perspective

7.2 Reviewer B — information retrieval perspective

7.3 Reviewer C — cognitive science perspective

7.4 Summary of changes triggered by the review

The review identified three changes we commit to making before the next release:

1. Linux baselines for Table 2 (R-A.1). Re-run the overhead benchmark on Linux and report those as the primary numbers.
2. Consolidation-firing latency measurement (R-A.2). Add a dedicated benchmark that times complete() calls where the threshold triggers, at library sizes up to 10K skills.
3. Non-adversarial retrieval benchmark (R-B.1) and α sensitivity sweep (R-B.2). Add a BEIR-style corpus measurement and a sweep over the Wilson coefficient.

Items R-C.1, R-C.2, and R-C.3 are framing/wording issues we address by revising the blog post and adding a footnote. They do not require new measurements. Items R-A.3 and R-B.3 are acknowledged as soft framing corrections; the paper narrative is adjusted but no new experiments are required.

8.Conclusion

KAOS Dream integrates inline synaptic plasticity with batched structural consolidation in a production multi-agent framework. The system is built from standard components — Wilson lower bounds, exponential decay, Jaccard similarity, sliding-window counters — assembled to flow continuously through normal agent operation. Three committed benchmarks quantify the resulting behavior: +10 pp retrieval accuracy gain on disambiguation-dominated queries, +15 µs p50 hot-path overhead, and 60/60 validations in a realistic failure-triage scenario. The gains are workload-conditional and we say so explicitly.

The most general methodological lesson is simpler than the specific techniques: commit benchmarks before you commit features. Our first design was 14,000× too slow; the microbenchmark caught it pre-merge; the rewrite was guided by the same benchmark; and the final numbers are published alongside the code. This does not require new process or new tools. It requires writing the benchmark first.

The system is open-source, MIT-licensed, and available at github.com/canivel/kaos. The three benchmarks referenced in this paper are in the top-level directories demo_neuroplasticity_bench/, demo_plasticity_overhead_bench/, and demo_failure_intelligence_bench/. Raw results are committed as JSON alongside the benchmark scripts. The growth-story blog post is at blog/kaos-neuroplasticity.html.

9.References

1. [1] Canivel, D. KAOS: Kernel for Agent Orchestration and Sandboxing. github.com/canivel/kaos, 2026.
2. [2] Zhou et al. Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols, and Harness Engineering. arXiv:2604.08224, 2026.
3. [3] Balakrishnan, Shi, Lu, Goel, Baral, Lyu, Dredze. LogAct: Enabling Agentic Reliability via Shared Logs. arXiv:2604.07988, Meta, 2026.
4. [4] Hebb, D.O. The Organization of Behavior. Wiley, 1949.
5. [5] Tononi, G. and Cirelli, C. Sleep and the Price of Plasticity: From Synaptic and Cellular Homeostasis to Memory Consolidation and Integration. Neuron, 81(1):12–34, 2014.
6. [6] Wilson, E.B. Probable Inference, the Law of Succession, and Statistical Inference. Journal of the American Statistical Association, 22(158):209–212, 1927.
7. [7] (Anonymous). Meta-Harness: Evolutionary Prompt and Strategy Optimization. arXiv:2603.28052, 2026.
8. [8] Newman, A. claude-mem. github.com/thedotmack/claude-mem, AGPL-3.0.
9. [9] MemPalace contributors. MemPalace: AAAK Compaction for Agent Context. github.com/milla-jovovich/mempalace.
10. [10] Buzsáki, G. Two-stage model of memory trace formation: A role for "noisy" brain states. Neuroscience, 31(3):551–570, 1989.
11. [11] (Anonymous). EvoSkills: Surrogate Verifier for Harness Evolution. arXiv:2604.01687, 2026.
12. [12] Canivel, D. Dream M2.5 commit fd71ebe: “real failure triage + measured-overhead-driven architecture fix.” github.com/canivel/kaos, April 2026.
13. [13] Zhang, W. and Linden, D.J. The other side of the engram: Experience-driven changes in neuronal intrinsic excitability. Nature Reviews Neuroscience, 4(11):885–900, 2003.