Knowledge System

Unified system documentation and personal knowledge — typed units, DAG hierarchy, and full-content retrieval.

Details

Two authority providers share a common KnowledgeUnit read model:

• System docs — one Markdown file per unit under knowledge/store/**/*.md (YAML frontmatter + body). Behavioral directions, system docs, skill declarations, workflow structure. Queried via agent_docs. Editing a unit is editing its file.
• Personal knowledge — versioned records in the personal-knowledge SQLite authority. User-authored patterns, feedback, preferences, calibration, aliases, relationships, lifecycle, and provenance are queried via knowledge_personal and mutated through knowledge_mint or the revisioned service. A one-time importer retains legacy files as Sources and aliases; they are not a live writable surface after the authority seal.

Unit types

Nine kinds, each positively anchored to a clear functional or structural definition:

• directions — behavioral guide loaded by a slash command ("how to do X")
• skill — callable from MCP via wb_run; an inert declaration that names an op the loader resolves against the Op registry (see architecture/data-first-skills)
• workflow — authored Step DAG with stable workflow_id, canonical workflow_name, and optional executable workflow_aliases; registry compilation derives its revision and WorkflowService governs invocation
• personal — user-authored knowledge backed by the personal-knowledge domain store
• system — coherent functional domain whose persistent state work-buddy owns (e.g. tasks, triage, journal)
• service — internal work-buddy component with a network surface (e.g. services/dashboard, architecture/embedding-service)
• integration — connection to an external system (e.g. obsidian/bridge, email, vault)
• reference — documents the API surface of one or more Python modules (entry-points-led)
• concept — architectural narrative, design philosophy, or domain-category heading

The system / integration boundary is anchored on memory ownership: a domain whose persistent state Work Buddy manages is a system; a domain whose state lives outside Work Buddy is an integration.

The service / integration boundary is internal vs. external: a service is something work-buddy runs (sidecar-managed); an integration is something work-buddy talks to (the bridge that wraps it may be a Flask app, but the unit's identity is the external dependency).

System-document file substrate

A unit at store path P lives at knowledge/store/<P>.md. A domain parent unit and its children's directory coexist (e.g. tasks.md beside tasks/). The path↔file mapping is bijective; no index.md convention is needed.

children is not stored — it is derived at load time from other units' parents, so the parent/child graph cannot drift. Only parents is authored.

Workflow units carry stable identity and their steps DAG in YAML frontmatter. Each Step's instruction prose lives in a ## <step-id> body section, split only at headings whose text exactly matches a known Step ID. Body text before the first matching heading is the Workflow's content.full narrative.

The system-document engine talks to the file seam (read_unit, write_unit, list_unit_paths, delete_unit, move_unit, load_units_from_dir) in work_buddy/knowledge/file_store.py. Personal retrieval instead resolves the active personal-knowledge provider and, after seal, reads SQLite without enumerating a vault.

Local patches live in knowledge/store.local/*.json (gitignored, JSON-shaped, deep-merged on top of the file-per-unit base on load) — the personal-overlay seam.

Workflow identity on the file substrate

A Workflow file stores its stable workflow_id; its file path, display name, and canonical workflow_name are not identity. The structured editor enforces the contract:

• create/scaffold assigns a fresh wfd_ ID unless a valid explicit ID is supplied;
• update cannot replace an existing ID;
• changing workflow_name preserves the old address in workflow_aliases;
• move preserves the ID and rewrites path-based parent references and Directions-to-Workflow bindings;
• a local overlay cannot replace a tracked Workflow's ID, while a local-only Workflow owns its own ID.

The docs_edit Workflow exposes a native file-edit step and carries the original Workflow ID from resolve to commit, where replacement is rejected alongside missing, malformed, duplicate, or colliding IDs. Authors must still retain the old canonical address on rename. A native edit performed entirely outside docs_edit has no pre-edit identity guard, so that author must preserve the existing ID explicitly.

workflow_aliases are executable lookup handles. The inherited KnowledgeUnit aliases remain descriptive search terms and do not enter the Workflow execution namespace.

The workflow_identity validator rejects missing, malformed, or duplicate IDs; malformed or duplicate executable addresses; addresses in the reserved stable-ID namespace; and collisions between Workflow addresses/IDs and Skill names.

DAG hierarchy and multi-parent nesting

Units have parents / children for hierarchical navigation. An agent querying journal/ sees children without loading siblings it doesn't need.

The parents field is a list, not a singleton. The DAG validator allows multiple parents and only forbids cycles. A subsystem can live at one path for navigation and declare additional parent systems via parents. When creating a system unit that is a subsystem of a larger system, include the parent system in parents regardless of where the path lives. Path picks one navigation entry point; the DAG carries the full relationship graph.

The "subsystem-of-system" relationship is derivable: walk a unit's parents and filter by kind == "system".

Progressive disclosure

depth="index" (name + children list) → "summary" (core info) → "full" (complete content). Typical pattern: "index to map the domain, summary to triage, full for the one unit you're acting on."

Inline placeholders

Content can reference other units inline. The syntax is two angle brackets, wb:, the target unit path, two angle brackets — e.g. a reference to obsidian/bridge is written as that path surrounded by the wb: prefix and angle-bracket markers. At depth="full", the placeholder is replaced with the referenced unit's content. Appending --recursive after the path opts in to transitive expansion. Parsed with argparse (extensible to --depth, --section, etc.).

Harness-aware references

Append --harness after the target path, before the closing markers, to select a child variant. For a target such as dev/testing/browser-surface, a caller on codexcli receives that path's codexcli child. If the selected child does not exist, resolution uses <path>/default. The parent holds shared framing and remains independently readable and searchable. Variants are ordinary knowledge units with their parent declared in frontmatter.

An explicit agent_docs(..., harness="codexcli") overrides detection for authoring and previews. With harness=None, the resolver reads the originating session's manifest using the full native session identity. An absent or unreadable manifest falls back to WORK_BUDDY_HARNESS_ID, then to codexcli when CODEX_THREAD_ID is present, otherwise to unknown. Lookup does not register a session or repair a manifest. The selected variant still falls back to the default child when missing.

The --harness and --recursive flags compose: one chooses content, the other controls expansion within it. Caller recursion and size limits still apply. recursive="none" preserves the entire placeholder literally, including its flags.

Caller-side knobs on agent_docs

Callers can override authorial defaults at query time:

• recursive="default" — each placeholder honours its own --recursive flag.
• recursive="all" — every placeholder expands transitively, ignoring per-flag choices.
• recursive="none" — placeholders are preserved literally (markup not resolved). Useful for editing or inspection.
• max_depth=N — caps recursion depth. -1 (default) selects the mode default: unlimited in default mode, 10 in all mode. 0 disables recursion entirely. Positive ints set an exact cap.
• harness=<id> selects the variant for harness-aware placeholders instead of detecting the caller. It changes rendered content, not the search corpus.

The search corpus is always indexed with recursive="default" so search relevance doesn't shift with caller intent. Harness-aware references concatenate all direct variant paths in sorted order, including the default child, with an HTML comment naming each harness. Both the combined index and partitioned retrieval use this representation. A search for a tool from either browser recipe can therefore find the containing directions unit, while a full-depth response expands only the requested or detected variant.

Three safety mechanisms layered around recursive expansion

1. Per-unit-occurrence cap (always on, no override). First appearance of a target gets the full content; subsequent references — through any branch — return the inline marker <!-- wb: <path> already expanded above -->. Catches diamond graphs cheaply.
2. Depth cap (configurable via max_depth). On overflow, emits <!-- wb: placeholder expansion truncated at depth N -->.
3. Size budget (~100KB total expanded output in all mode). Ultimate backstop. On overflow, emits <!-- wb: placeholder expansion truncated at 100KB cap -->.

Each cap emits a distinct visible marker so the reader sees exactly what was elided and why.

Authoring guardrails

• Write-time hint (informational). The internal create_unit / update_unit write primitives return a hints field flagging plain placeholders that target units with their own placeholders — the case where the author probably wanted --recursive but forgot. Never blocks an edit.
• Write-time hard reject (error). The same write path rejects content with duplicate placeholders within a single unit: the same target appearing more than once contributes zero readable content (the per-unit-occurrence cap renders subsequent references as back-ref markers), so it's never the right authorial choice. The editor returns {"error": "placeholder_duplicate", "duplicates": [...]} and does not persist.
• Validator parity. docs_validate runs a placeholder_duplicate check corpus-wide so direct-file bypasses are still caught.
• Harness fallback. The harness_placeholder_default check requires <target>/default for every harness-aware target referenced in unit content. Missing selected variants fall back at read time; a missing default produces a visible not-found marker and a validation error.

Retired: context chaining

A context_before / context_after mechanism on each unit (auto-prepend / -append of referenced units' content at depth="full") was retired in favour of inline placeholders. Placeholders give authors per-reference control over recursion; chains were always hardcoded one-level. The fields are no longer read by the resolver; loaders silently ignore them if stale frontmatter still has the keys.

Dev notes

Units can carry a dev_notes string — development-facing guidance that only surfaces when the agent is in dev mode (enabled via mode_toggle) or when the caller passes dev=True. Use for architectural constraints, non-obvious dependencies, and hard-won lessons that future agents could easily clobber. All subsequent knowledge queries in the session auto-include dev_notes once dev mode is active.

Search index

A persistent BM25 + dense vector index over full unit content is warmed eagerly on MCP server startup. Inline placeholders are resolved for system units before indexing so searching for content inside a referenced unit also surfaces the referencing unit. Personal records are projected from their current SQLite revisions. The index is derived and rebuildable; exact personal reads hydrate from the authority provider. See architecture/embedding-service for the model registry behind dense retrieval.

MCP skills

• knowledge — unified search across both system docs and personal knowledge
• knowledge_personal — personal database knowledge only (supports category and severity filters)
• knowledge_mint — create or update a personal knowledge record through the active authority seam
• agent_docs_rebuild — reload system files and refresh the personal provider/index projections
• mode_toggle — toggle a session mode (e.g. dev) on or off
• knowledge_index_rebuild — force rebuild knowledge search index with full embeddings
• knowledge_index_status — check index health
• docs_edit — the Workflow for editing or creating any unit kind. A new Workflow scaffold receives a stable ID; commit validates the kind-specific schema and reconciles the store cache and index.
• docs_delete / docs_move — structural operations. Workflow moves preserve definition identity and rewrite path-based Directions bindings.
• docs_validate — kind-aware structural validation covering DAG integrity, placeholders, harness fallback children, Skill Op resolution, Workflow identity/address collisions, Workflow Step DAGs, Directions bindings, and Workflow delegation references.