Skip to content

Conversation Observability

Durable session-attributed activity DB for Claude Code: commits, file writes, GitHub PR activity, uncommitted work, observed-session metadata, and optional LLM topic summaries. Replaces ad-hoc per-call JSONL scans in sessions/inspector.py.

Details

A SQLite-backed store of session-derived facts for Claude Code: commits, file writes, uncommitted-work attribution, observed-session metadata. The subsystem is a thin SQLite store plus refresh functions; raw JSONL parsing stays in work_buddy/sessions/inspector.py (single source of truth for the parser), and Git context source continues to own commit/status collection. Per-session LLM topic summaries are produced and stored by the summarization framework (architecture/summarization-framework); this subsystem owns the discovery side (observed-session enumeration) and the conv_obs-shaped read API that maps the framework's tree storage to a flat row.

Why

sessions/inspector.py accumulated five orthogonal responsibilities: raw browsing, span mapping, commit extraction, write extraction, uncommitted attribution. The last three were a private cache for GitSource's session annotation, recomputed per call with a process-local mtime dict. Restarts wiped the cache; other consumers (journal, context bundle, dashboard) couldn't read it independently.

Surface

Four tables in <data_root>/conversation_observability/conversation_observability.db (path overridable via conversation_observability.db_path):

  • observed_sessions — per-JSONL ledger. Carries metadata (start/end, message_count, span_count, tool_names) plus per-concern scan-mtime columns: source_mtime (metadata load), commits_scanned_mtime (commits refresh), writes_scanned_mtime (writes refresh), prs_scanned_mtime (PR refresh). Each refresher owns its column so running them in any order doesn't conflate staleness state.
  • session_commits — one row per git commit attributed to a session. Keyed by full SHA, indexed on short_sha for GitSource lookups.
  • session_file_writes — one row per (session, file_path). Carries the tool that wrote it, the latest write timestamp, an optional committed_sha cross-reference, and a currently_dirty snapshot (best-effort — git state is mutable, so consumers should treat this as not authoritative without a refresh).
  • session_prs — one row per (session, PR, action) GitHub pull-request event, attributed by detecting gh pr create|merge|close|review Bash invocations in the JSONL (structural detection, not commit-message Closes #NNN parsing). Carries pr_number, pr_url, repo, action, and the invocation ts. UNIQUE(session_id, pr_number, action, ts) makes re-ingestion idempotent.

Per-session tldr + ordered topic segments live in the summarization framework's <data_root>/summarization/summarization.db under namespace conversation_session (see architecture/summarization-framework). The legacy read API (session_summary_get, plus the deprecated alias conversation_observability_summary_get) is preserved via thin shims in session_summary_row.py that map between the framework's tree-shaped storage and the flat row shape consumers expect (dashboard /api/chats/<id>/topics, the claude_session_summary context collector, /wb-session-identify's tldr triage).

Foreign-key cascades use SqliteRowsStorage.post_delete_sql rather than SQLite's FK enforcement. Deleting an observed_sessions row removes every child in the same transaction within conversation_observability.db; the corresponding summary_items + summary_nodes rows in summarization.db are dropped explicitly by the orphan-prune in refresh_observed_sessions as a best-effort follow-up.

Refresh model

Two sidecar crons keep the DB fresh independent of caller demand:

  • conversation-observability-refresh.md — every 5 minutes (offset from ir-index-rebuild by 2 minutes), max_sessions=5, stale_only=true. Runs all four non-LLM refreshers (observed-sessions, commits, writes, PRs) AND auto-enqueues changed sessions into the summarization queue when summaries.use_incremental is on. Because the cron uses a 7-day window, PR (and commit/write) attribution for older sessions requires a one-off wide-window backfill (refresh_session_prs(days=…)) after the table first lands.
  • summarization-worker.md — every 5 minutes (offset 3 minutes from observability-refresh). Drains the summarization queue FIFO over the cooldown-passed subset, bounded by the daily cost budget. Feature-gated on summaries.use_incremental.

The claude_session_summary context source also triggers a stale-only refresh inline before rendering so bundle collections never read a cold DB. The /ir/index endpoint is deliberately NOT hooked — stale-only DB-backed scans are cheap enough that an independent cron is cleaner than embedding-service coupling.

Lifecycle

The artifact uses INFINITE_LIFECYCLE (paired with the NeverExpires lifecycle trigger): every row is derived from JSONL session files that may have been deleted, so losing the DB means losing data that cannot be recovered. The sweep tick will see the artifact but never remove rows.

Summary invalidation lives on the framework side: bumping any of the four version constants on the active strategy/store (prompt_version, schema_version, selection_version, cache_version) invalidates cached summaries. The framework's composer re-bridges the strategy's versions into the store on every refresh, so a bump takes effect on the next sidecar fire.

Consumers

  • work_buddy/collectors/claude_session_summary_collector.py — context source rendering one block per project, listing each session's commits and uncommitted files. With include_topics=True, nests a topic-level timeline under each session bullet. Sibling to chat (raw inventory) and session_activity (current MCP session ledger).
  • work_buddy/collectors/git_collector.py — receives {short_sha: full_session_id} via inspector.build_session_map(), which now reads from the DB instead of computing per-call.
  • MCP capabilities: conversation_observability_refresh, conversation_observability_uncommitted, conversation_observability_get, conversation_observability_list, session_summary_get, session_prs_get, summarization_worker_tick. The conversation_observability_summarize and conversation_observability_summary_get capabilities remain as deprecated aliases routed through legacy shims. The reverse session→tasks linkage is exposed via the tasks-domain session_tasks_get capability (reads task_sessions, enriched from the SQLite task store — bridge-independent).
  • Dashboard Chats view (/api/chats → _load_observability_for_sessions): aggregates per-session PR counts (authored/merged) and task-assignment counts into the chat cards' badge row, alongside the existing commit badge, and renders per-session Pull requests + Tasks sections in the chat detail panel beside the commit list (commits green, PRs purple, tasks orange). PR rows are enriched with title + current merge state (OPEN/MERGED/CLOSED) via a best-effort, TTL-cached gh pr list per repo (_load_pr_meta_for_repos) — the JSONL only yields number/url/action, so title/merge-state come from GitHub at the display layer; offline/un-authenticated gh simply omits them.
  • Journal directions (journal/update-directions) require claude_session_summary.md alongside git_summary.md / chat_summary.md so multi-hour sessions without commits get logged as exploration rather than silently dropped.