Embedding Service

Local HTTP service on port 5124 providing dense vector embeddings for search and similarity; exposes a symmetric default model plus an asymmetric query/document pair via role-aware client wrappers.

Details

Overview

A long-running sidecar service providing dense vector embeddings for work-buddy's search and similarity features. Exposes an HTTP API on localhost:5124. Interactive query models are eager; the larger document model is lazy and loads only if a document request actually resolves to a local route.

Endpoints

• POST /embed: embed a batch of texts. Document-model requests pass through the service's authoritative provider router and return route provenance with the vectors. A required remote route that cannot run returns a structured 503 instead of loading the local model
• POST /similarity — cosine similarity between a query and candidate texts
• POST /search — BM25 + embedding hybrid search over candidates
• POST /ir/search, POST /ir/index — indexed IR search over registered sources. Every IR source shares one SQLite DB, so builds serialize on a DB-wide advisory lock; a build that arrives while another build is running returns {"skipped": true, "reason": "build_in_progress"} instead of queueing. action="status" is never gated and stays readable during a build
• POST /vault/search, POST /vault/index — vault semantic index search / build, run in-process so the resident vector matrix stays warm and the bulk encode shares the broker (see architecture/vault-index)
• POST /index/search, POST /index/search_many — consolidated-index hybrid search over one or more partitions (/index/search_many shares ONE query-encode across a batch of queries); POST /index/build — incremental build of a partition (or all) into the separate db/index-consolidated DB. Run in-process so the resident matrices stay warm and the bulk encode shares the broker (see architecture/consolidated-index)
• GET /health: liveness plus model-residency and document-route evidence, including configured policy, circuit-breaker cooldown, and the last actual provider route

Client

work_buddy/embedding/client.py wraps the HTTP API:

• embed(texts) — plain batch embedding, uses the service's default model
• embed_with_route(texts): document-oriented batch embedding that preserves the service's structured route or failure envelope
• health_status(): full health and route-provenance snapshot for operator surfaces
• embed_for_ir(texts, role="query"|"document") — asymmetric IR encoding via the query↔document model pair
• similarity_search(query, candidates) — rank candidates by similarity
• hybrid_search(query, candidates) — BM25 + dense blend
• ir_search(query) — search a pre-built IR index
• ir_index(action, source, ...) — build or check an IR index
• vault_search(query, ...), vault_index(action, ...) — vault semantic index search / build (see architecture/vault-index)
• index_search(query, ...), index_search_many(queries, ...) — consolidated-index hybrid search over its partitions (see architecture/consolidated-index)

Graceful degradation

Connection failures still return None (or an empty list) for clients that opt into graceful degradation. Provider failures are different: the route-aware client preserves the service's structured error. Under Require LM Studio, document and passage embedding fails closed; it must not silently load the local document model. Index builds remain resumable, while similarity and search callers use their lexical path or skip dense ranking where that fallback is supported.

Consumers

Knowledge search, IR conversation search, native vault search, task-triage similarity, and other semantic-scoring sites throughout the codebase all route through this service. Local model loading happens once here; remote document vectors require no local leaf-ir load.

Document execution policy

The large passage encoder (leaf-ir / snowflake-arctic-embed-m-v1.5) has three restart-gated modes under System → Embeddings:

• Local only runs every document batch in this process.
• Prefer LM Studio tries the configured LM Studio model and deliberately loads the local model as fallback when the remote route is unavailable.
• Require LM Studio never loads leaf-ir locally. An unavailable endpoint, missing model alias, failed embedding request, or provider cooldown blocks document-vector production with a structured failure. Index refreshes still commit lexical documents and durable ledger progress, but remain incomplete while dense work is pending; similarity/search consumers use their supported non-dense fallback. Absence from LM Studio's /v1/models snapshot alone is not treated as failure because LM Studio may load the configured model on demand.

The route is caller → local /embed → ProviderRouter → LM Studio or local provider. LM Studio may in turn place the model on an LM Link peer. Query-side encoders (leaf-ir-query, leaf-mt) remain local for latency and are unaffected by this setting.

The dashboard distinguishes endpoint reachability from whether the configured model ID is currently advertised, without claiming that an unadvertised model is unavailable, and distinguishes the saved/effective setting from the policy reported by the running service. It also reports whether local document weights are resident and the last actual provider/fallback. Its approximately 526 MB figure is a model-weight estimate, not a guaranteed reduction in process private commit; runtime buffers and allocator high-water commit vary and may persist until process restart.

YAML supplies the bootstrap default, LM Studio base URL, and remote model alias. Settings owns the user's execution mode after bootstrap, and the embedding service promotes a pending choice only when it starts. If Settings authority cannot be read at startup, the service keeps query models available but forces the document route to LM Studio with no local fallback and eager: false; /health and the dashboard expose that authority error. This fail-closed safety state avoids accidentally loading the large local model from stale YAML. See architecture/inference/broker for admission control and features/lmstudio-offload-setup for model setup and verification.

Key files

• work_buddy/embedding/service.py — Flask service, model registry, lazy loading. _get_model() uses a per-entry Condition so a cold load of one model doesn't block concurrent access to another.
• work_buddy/embedding/client.py: HTTP client with role-aware and route-preserving wrappers.
• work_buddy/embedding/providers/lmstudio_config.py: lightweight endpoint/model configuration used by probes.
• work_buddy/embedding/providers/lmstudio.py — optional LM Studio provider (broker-wrapped).
• work_buddy/embedding/__main__.py — service entry point (launched by the sidecar).