847 KAOS AI Agents. 847 Files.
8 Minutes. Zero Regressions.

The job: migrate a 847-file Python 2 codebase to Python 3.11. One agent per file. 8 minutes 47 seconds. Zero regressions shipped.

Here's how 847 agents worked together — and what they accomplished that no single agent ever could.

847 agents spawn in 17 batches. 809 complete. 31 roll back. 0 regressions shipped. 2.45M tokens saved.

The Scale Problem Nobody Talks About

At 10 agents, isolation is a nice property. At 847 agents, it's load-bearing infrastructure.

Most people who think about multi-agent systems at scale focus on orchestration, rate limits, and error handling. Those are real problems. But the deeper one nobody mentions until they hit it: shared state explodes at scale.

Every shared filesystem is a race condition. Agent 312 writes to utils/compat.py while agent 447 is reading it to decide whether to apply its own patch. The result isn't a merge conflict — it's a silent corruption. The kind you discover three days later when tests start failing for reasons that look entirely unrelated.

Every shared context window is a token explosion. Naive multi-agent frameworks pool context across agents. At 100 agents, you're sending 100× the context tokens. The agents don't produce better results. They're just bloated.

The boring truth: isolation is not a feature at scale. It's a prerequisite. This is the run where that became undeniable.

The Setup: 1 Agent Per File

Task: migrate a 847-file Python 2 codebase to Python 3.11. One KAOS agent per file. Each agent gets its own isolated VFS, runs the migration, runs the test suite, and either commits the result or rolls back to the pre-migration checkpoint.

# Spawn 847 agents from a file manifest
kaos parallel spawn \
  --manifest migration-manifest.txt \
  --task "migrate to Python 3.11, run tests, checkpoint on success" \
  --model claude-sonnet-4-6 \
  --batches 17 \
  --batch-size 50

# [kaos] Reading manifest: 847 files
# [kaos] Spawning batch 1/17 (agents 1–50)...
# [kaos] Spawning batch 2/17 (agents 51–100)...
# ...
# [kaos] All 847 agents spawned in 00:00:17s
# [kaos] Running...

The kaos.yaml config driving this run:

# kaos.yaml
project: py2to3-migration

agents:
  model: claude-sonnet-4-6
  isolation: logical
  checkpoint_on_success: true
  rollback_on_test_failure: true

compression:
  aaak_level: 5          # ultra compression — 95% token reduction on digest
  blob_dedup: true       # SHA-256 + zstd deduplication across all agents

parallelism:
  max_concurrent: 50     # WAL-safe concurrency limit
  batch_size: 50
  retry_on_timeout: 2

hub:
  enabled: true          # CORAL hub coordination
  min_confidence: 0.85
  broadcast_on_discover: true

aaak_level: 5 is the key compression setting — ultra mode, 95% token reduction on each agent's context digest. We'll see what that means in tokens. The hub.enabled: true activates CORAL coordination, where agents share patterns discovered from failures. Both are single lines in config.

What "Isolated VFS" Means at 847 Agents

Every KAOS agent gets its own virtual filesystem — not a directory, but a SQLite-backed, content-addressable VFS where every write is recorded, every checkpoint is a snapshot, and blobs are deduplicated across the entire agent pool.

Here's what that means for storage at 847 agents:

Identical files across agents — stdlib imports, utility functions, common Python 2 patterns — share a single blob via SHA-256 + zstd. On a migration task, most files share a large percentage of content. 68% deduplication means 147MB of data was stored exactly once instead of once per agent.

The final number: 214MB SQLite file holds the complete history of every agent's VFS state, every event, every blob, every checkpoint. That's the entire audit trail for an 847-agent job.

AAAK Compression: The Math

AAAK (Adaptive Anchor-Aware K-compression) compresses each agent's context digest before it's passed back as system context on the next turn. At level 5 — ultra — it achieves ~95% token reduction on the digest.

Single agent, single turn comparison:

Cumulative across 847 agents, averaging 3.4 turns each:

Without AAAK, the job would have consumed 8.58M tokens. With AAAK L5, it ran on 6.13M — 2.45M tokens that never needed to be sent. The agents produce identical migrations either way. The compression is purely a context digest optimization; it doesn't touch working state.

Level 5 is aggressive and right for this task. For jobs where agents need richer cross-turn memory — complex refactoring decisions that reference earlier analysis — level 3 or 4 is usually the better tradeoff. It's one line in kaos.yaml.

When Agents Fail: Rollback Without the Blast Radius

31 agents found test failures during migration. In a traditional shared-filesystem setup, rolling back one file's changes risks affecting the state of other in-progress migrations. In KAOS, a rollback is a point-in-time restore of one agent's SQLite-backed state — it doesn't touch anything else.

Here's the exact event log for db/connections.py — one of the 31 rollbacks:

[agent-312] db/connections.py  migration applied
[agent-312] running pytest...
  FAILED tests/test_db.py::test_connection_pool_size
  FAILED tests/test_db.py::test_reconnect_on_timeout

  2 failed, 23 passed

[agent-312] test failures detected — rolling back to pre-migration checkpoint
[agent-312] restoring VFS to: pre-migration-312
[agent-312] restore complete in 0.08s
[agent-312] status: rolled_back
[agent-312] event logged: {
  "agent": "agent-312",
  "file": "db/connections.py",
  "failures": ["test_connection_pool_size", "test_reconnect_on_timeout"],
  "failure_pattern": "timeout_kwarg_renamed",
  "rollback_time_s": 0.08,
  "other_agents_affected": 0
}

Two numbers to notice: 0.08s restore time — sub-100ms, the agent's entire VFS is rewound in under a tenth of a second. And other_agents_affected: 0 — 846 other agents kept running without interruption. The rollback is scoped to one agent's VFS, nothing more.

Hub Coordination: Agents Teaching Each Other

31 agents failed and rolled back. But the hub prevented roughly 180 additional failures from ever happening.

When an agent rolls back, KAOS logs the failure pattern to the CORAL hub — a central coordination point where agents share discovered patterns. Other agents that haven't yet processed similar files can receive patterns pre-emptively and adjust their approach.

The most impactful pattern discovered during this run — none_guard_before_has_key:

[hub] New pattern discovered from agent-312 rollback
  pattern: none_guard_before_has_key
  confidence: 0.91
  trigger: dict.has_key() calls where dict may be None
  fix: add `if dict is not None` guard before .get() replacement
  source_failure: test_connection_pool_size, test_reconnect_on_timeout

[hub] Broadcasting to 23 agents with similar pending files...
  agent-089: db/session.py         → applying pattern pre-emptively
  agent-134: db/pool_manager.py    → applying pattern pre-emptively
  agent-201: cache/backend.py      → applying pattern pre-emptively
  ...
  [23 agents notified]

[hub] Pattern confirmed: 23/23 agents applied, 0 new failures on similar files

The hub shared 12 distinct patterns during this run. Agents that received a hub pattern pre-emptively had a 3.8% failure rate on similar files; agents that didn't had a 22.1% failure rate — a difference that accounts for approximately 180 prevented regressions across the job.

The Final Numbers

847 files. 8 minutes 47 seconds. 0 regressions shipped.

Outcome Summary

AAAK Compression Impact

Time Comparison

SQL Audit: The Complete Picture

Every event across all 847 agents is in one SQLite file. One query covers the whole job:

-- Outcome summary across all agents
SELECT status, COUNT(*) AS count,
  ROUND(COUNT(*) * 100.0 / 847, 1) AS pct
FROM agents
WHERE run_id = 'py2to3-migration'
GROUP BY status
ORDER BY count DESC;

status       count   pct
-----------  -----   ----
succeeded    809     95.5
rolled_back  31       3.7
failed       7        0.8

-- All rollback events with failure patterns
SELECT
  json_extract(notes, '$.failure_pattern') AS pattern,
  COUNT(*) AS occurrences,
  GROUP_CONCAT(file_path, ', ') AS affected_files
FROM vfs_events
WHERE run_id = 'py2to3-migration'
  AND event_type = 'restore'
GROUP BY pattern
ORDER BY occurrences DESC;

pattern                        occurrences  affected_files
-----------------------------  -----------  --------------------------------
none_guard_before_has_key      8            db/connections.py, ...
print_function_side_effect     6            scripts/report.py, ...
unicode_bytes_ambiguity        5            api/serializers.py, ...
iteritems_generator_consumed   4            core/registry.py, ...

-- Token savings: what AAAK eliminated across all agents
SELECT
  SUM(tokens_uncompressed) AS total_uncompressed,
  SUM(tokens_compressed)   AS total_compressed,
  SUM(tokens_uncompressed - tokens_compressed) AS tokens_saved
FROM aaak_compression_log
WHERE run_id = 'py2to3-migration';

total_uncompressed  total_compressed  tokens_saved
------------------  ----------------  ------------
4,949,663           2,498,600         2,451,063

One query. Every agent's complete behavior. Every failure pattern. Every token sent and saved. The 214MB SQLite file is not a summary — it's a structured, queryable record of everything that happened during the run. Every write, every rollback, every tool call, every hub broadcast, timestamped to the millisecond.

What Scales, What Doesn't

Honest assessment. Here's what KAOS handles cleanly at scale, and where you'd need to think carefully above the tested range.

What scales cleanly:

• VFS isolation scales linearly. 1 agent or 10,000 agents — each one's filesystem is independent. No contention, no coordination overhead per agent.
• Blob deduplication scales better than linearly. More agents with similar content means a higher dedup ratio, so storage overhead grows sublinearly with agent count.
• AAAK compression is entirely local computation. The savings accumulate directly with scale — each additional agent-turn saves the same proportion of tokens.
• Per-agent rollback has constant time complexity. It doesn't matter how many other agents are running — one agent's restore takes the same 0.08s.
• Hub coordination scales with pattern discovery rate, not agent count. 12 patterns across 847 agents; you don't accumulate 847 patterns just because you ran 847 agents.

What you'd need to think about above ~1,000 concurrent agents:

• SQLite WAL contention. KAOS uses WAL mode, which handles ~50 concurrent writers well. Above 200–300 concurrent writes you'll start seeing lock contention. The default max_concurrent: 50 is deliberately conservative. For higher concurrency, shard the database by agent pool or move to a distributed event backend.
• Hub broadcast latency. The hub is synchronous in the current implementation — fine at 847 agents, potentially a bottleneck at 5,000+ with high pattern discovery rates. Async broadcasts are the fix.
• In-flight VFS cache memory. Each active agent's hot state is cached in memory. 50 concurrent agents × ~2MB each = ~100MB. At 500 concurrent that becomes ~1GB. Plan your hardware accordingly.
• Manifest initialization time. kaos parallel spawn with 847 agents took 17 seconds to initialize. 5,000 agents would be ~100 seconds — not a dealbreaker, but worth knowing.

The architecture was designed for local-first operation. It runs on a MacBook Pro. For production scale beyond 1,000 concurrent agents, dedicated hardware and a distributed event store make sense — the VFS abstraction, event journal, and blob store are clean interfaces that can be backed by distributed systems without changing agent behavior.

847 agents. 809 files migrated. 31 rolled back cleanly. 0 regressions shipped. 8 minutes 47 seconds. And 2.45M tokens they never had to send — the cherry on top.

The audit trail lives in a 214MB SQLite file. Every agent's decision — every write, every rollback, every test failure, every hub pattern received — queryable forever.

KAOS is MIT-licensed and runs entirely locally. No data leaves your machine. github.com/canivel/kaos