The Skill Library: Agents That
Learn Procedures

The problem: agents reinvent the wheel every session

An ML agent starts a new classification task. It has no memory of prior runs. It tries single-call inference — too slow. It tries temperature sampling — inconsistent. It tries chain-of-thought — better but still wrong. By iteration 5, it discovers that ensemble voting with 3 models and majority vote reliably hits 83%+ accuracy. Iteration 6 is the first time it actually gets there.

Cost: $19.40. Iterations: 6. Time: ~40 minutes.

Next week, a different agent on a different project hits the same classification problem. It has no idea what the last agent learned. It starts from scratch. Goes through the same 5 failed iterations. Spends $19.40 again.

This is the core inefficiency of stateless agents. Every session starts from zero. Every discovery is lost. The cost of finding good solutions gets paid over and over — by every agent, on every project, forever.

---

Memory vs Skills: the key distinction

KAOS already has cross-agent memory — a shared FTS5-indexed store where agents record observations, errors, and results across sessions. So why add a separate SkillStore?

Because memory and skills answer different questions:

Dimension	Memory	Skills
What it stores	Facts, observations, outcomes	Reusable procedures
Cognitive type	Episodic (what happened)	Procedural (how to do it)
Structure	Free-form text	Parameterized template
Usage pattern	Read for context	Apply with parameters
Outcome tracking	No	Yes — per-use success/failure
Reliability signal	None	Accumulates over time
Example	"ensemble hit 83% on 2026-04-10"	"Use {n} models with {voting} voting"

Memory tells you what happened. Skills tell you how to reproduce it — on any task, with any parameters. They're complementary layers of the same externalized knowledge system.

---

How SkillStore works

A skill is a named, parameterized prompt template. Parameters are written as {param} placeholders. When an agent applies a skill, it fills in the placeholders and gets back a ready-to-use prompt string.

Under the hood, skills live in a dedicated SQLite table with an FTS5 virtual table for full-text search (porter stemming). Every application is tracked — success or failure. Over time, skills with high success_count float to the top of the library.

The schema

CREATE TABLE skills (
    skill_id      INTEGER PRIMARY KEY AUTOINCREMENT,
    name          TEXT NOT NULL UNIQUE,
    description   TEXT NOT NULL,
    template      TEXT NOT NULL,          -- {param} placeholders
    tags          TEXT NOT NULL DEFAULT '',
    use_count     INTEGER NOT NULL DEFAULT 0,
    success_count INTEGER NOT NULL DEFAULT 0,
    fail_count    INTEGER NOT NULL DEFAULT 0,
    created_at    TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%f','now')),
    updated_at    TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%f','now'))
);

-- FTS5 for BM25 search (porter stemming)
CREATE VIRTUAL TABLE skills_fts USING fts5(
    name, description, tags,
    content=skills, content_rowid=skill_id,
    tokenize='porter ascii'
);

The FTS5 index covers name, description, and tags — so a search for "classification accuracy" will match a skill tagged classification,ensemble whose description mentions "improve accuracy".

---

CLI walkthrough

Saving a skill after discovering what works

$ kaos skills save \
    --name ensemble_voting \
    --description "Ensemble voting improves classification accuracy" \
    --template "Use {n_models} models with {voting} voting on {task}. Threshold: {threshold}." \
    --tags classification,ensemble

Skill #1 saved
  name=ensemble_voting
  params=[n_models, voting, task, threshold]
  tags=classification, ensemble

KAOS automatically extracts the parameter list from the {placeholder} tokens in the template — no schema declaration needed.

Saving a second skill

$ kaos skills save \
    --name json_error_guard \
    --description "Wrap LLM output parsing in try/except for JSON errors" \
    --template "Always parse {output} inside try/except json.JSONDecodeError. Return {fallback} on failure." \
    --tags python,error-handling,reliability

Skill #2 saved
  name=json_error_guard
  params=[output, fallback]
  tags=python, error-handling, reliability

Searching before starting a new task

The next day, a different agent hits the same classification problem. Instead of starting blind, it searches first:

$ kaos skills search "classification accuracy"

2 results  (BM25 relevance order)

  #1  ensemble_voting  [classification, ensemble]
      Ensemble voting improves classification accuracy
      params=[n_models, voting, task, threshold]  use_count=0  success_rate=n/a

  #4  cot_numbered  [reasoning, chain-of-thought]
      Numbered chain-of-thought reduces multi-step errors
      params=[steps, domain]  use_count=14  success_rate=0.86

Applying the skill

$ kaos skills apply 1 -p n_models=3 -p voting=majority -p task="code review" -p threshold=0.5

Rendered prompt:
  Use 3 models with majority voting on code review. Threshold: 0.5.

The agent uses this rendered prompt at iteration 1 instead of discovering it at iteration 6.

Viewing the library by success rate

$ kaos skills ls --order success_count

  ID  Name                  Tags                         Uses  Success  Fail  Rate
  ──  ────────────────────  ───────────────────────────  ────  ───────  ────  ────
   4  cot_numbered          reasoning,chain-of-thought     14       12     2  0.86
   3  progressive_unfreezing  fine-tuning,training          9        7     2  0.78
   1  ensemble_voting       classification,ensemble         6        5     1  0.83
   2  json_error_guard      python,error-handling           4        4     0  1.00
   5  temp_zero_determinism  inference,sampling             3        2     1  0.67

Recording outcomes

# After a successful run using skill #1
$ kaos skills outcome 1 --success

# After a failed run
$ kaos skills outcome 1 --fail

Outcome tracking is how reliability accumulates. Skills that consistently work rise to the top. Skills that fail often get deprioritized — or removed.

---

The before/after

Metric	Without Skills	With Skills
Accuracy	83%	83%
Iterations to hit target	6	1
LLM cost	$19.40	$3.20
Time to first result	~40 min	~7 min
Cross-session transfer	None	Automatic

Same accuracy. 6x fewer iterations. 6x lower cost. The difference is that the second agent started from knowledge, not from scratch.

An agent saves the ensemble voting pattern, then a second agent finds it via FTS5 search and applies it directly — skipping 5 iterations of trial and error.

---

Python API

from kaos import Kaos
from kaos.skills import SkillStore

db    = Kaos("project.db")
store = SkillStore(db.conn)

# Save a skill
skill = store.save(
    name="ensemble_voting",
    description="Ensemble voting improves classification accuracy",
    template="Use {n_models} models with {voting} voting on {task}. Threshold: {threshold}.",
    tags=["classification", "ensemble"]
)
print(skill.skill_id)   # 1
print(skill.params)     # ['n_models', 'voting', 'task', 'threshold']

# Search
results = store.search("classification accuracy")
for s in results:
    print(s.name, s.success_rate)

# Apply — fills placeholders, returns rendered string
prompt = results[0].apply(
    n_models=3,
    voting="majority",
    task="fraud detection",
    threshold=0.45
)
# "Use 3 models with majority voting on fraud detection. Threshold: 0.45."

# Record outcome
store.outcome(skill_id=1, success=True)

# List by reliability
top = store.list(order="success_count", limit=10)

---

MCP tools for Claude Code

All five skill operations are available as MCP tools — meaning Claude Code agents running inside KAOS can save and search skills without any CLI calls:

• skill_save — save a new skill with name, description, template, and tags
• skill_search — FTS5+BM25 search across name, description, and tags
• skill_apply — render a skill template with parameter values
• skill_list — list the library, sortable by use count or success count
• skill_outcome — record success or failure after applying a skill

Tell your agent: "before writing any classification logic, search the skill library first." It will call skill_search, find relevant patterns, and apply them before proposing any new approach.

The simplest agentic pattern:

# At the start of any session
skills = await mcp.call("skill_search", {"query": task_description})
if skills:
    prompt = await mcp.call("skill_apply", {
        "skill_id": skills[0]["skill_id"],
        "params": relevant_params
    })
    # use prompt at iteration 1

# At the end of a successful session
await mcp.call("skill_save", {
    "name": "...",
    "description": "...",
    "template": "...",
    "tags": "..."
})

---

The externalization framework

The SkillStore is a direct implementation of externalized procedural memory from arXiv:2604.08224. Zhou et al. 2026 distinguish two types of externalized knowledge:

• Episodic externalization — recording what happened (observations, results, errors) so future agents have context. This is what KAOS memory does.
• Procedural externalization — encoding how to do something as a reusable, parameterized procedure. This is what SkillStore does.

The insight from Zhou et al. is that procedures decouple discovery cost from application cost. Discovery is expensive — it requires iteration, failure, and inference calls. Application is cheap — it requires one search and one fill-in. By externalizing procedures, you pay discovery cost once and application cost forever after.

KAOS implements this with FTS5+BM25 over porter-stemmed tokens, so skill discovery is a sub-millisecond SQL query even across thousands of saved skills.

---

Memory + Skills: the full externalized knowledge layer

Together, memory and skills form the complete externalized knowledge layer that makes KAOS agents smarter over time:

• Memory records what happened — prior results, errors to avoid, domain observations. Agents start with context instead of ignorance.
• Skills record how to do it — validated procedures ready to apply. Agents start from proven patterns instead of blind exploration.

Both are indexed by FTS5+BM25 with porter stemming. Both are cross-agent and cross-session. Both compound over time — the more you use KAOS, the smarter every agent in the system becomes.

The long-term dynamic: a team that saves skills consistently will see per-session costs drop quarter over quarter, while accuracy holds steady or improves. The skill library becomes a team asset — accumulated procedural knowledge that outlasts any individual session, agent, or project.

---

Where to go next

• Full SkillStore documentation — API reference, CLI, MCP, schema
• skill_library.py — end-to-end walkthrough in Python
• Cross-Agent Memory ← — the episodic layer that complements skills
• Shared Log ← — coordination and intent before acting