All skills
vectorize-io avatar

/hindsight-docs

@5bfef3c
by vectorize-iovectorize-io/hindsight44k stars
5,845

Complete Hindsight documentation for AI agents. Use this to learn about Hindsight architecture, APIs, configuration, and best practices.

Use this Skill: https://skilld.dev/gh/vectorize-io/hindsight/hindsight-docs

This session only. Nothing lands on disk.

referencesdeveloperapiknowledge-pages.md

≈5.7k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Knowledge Pages

Living markdown documents, organized in a folder tree, that rewrite themselves as the bank learns.

A page is backed by a mental model but is configured as a document: it is built from the bank's observations only, it never reads other pages, and it refreshes incrementally after each consolidation. See Knowledge Pages for the concepts behind the API.

{/* Import raw source files */}

All endpoints below are relative to a bank:

/v1/default/banks/{bank_id}/knowledge-base

Get the Tree

Returns the whole knowledge base as a nested tree of folders and pages. Page bodies are not included — fetch a page to read its content.

Python

# Fetch the whole knowledge base as a nested folder/page tree (no page bodies)
tree = client.get_knowledge_base_tree(BANK_ID)

for root in tree.roots:
    print(f"{root.kind}: {root.name}")
    for child in root.children:
        print(f"  {child.kind}: {child.name} (stale: {child.is_stale})")

Node.js

// Fetch the whole knowledge base as a nested folder/page tree (no page bodies)
const tree = await client.getKnowledgeBaseTree(BANK_ID);

for (const root of tree.roots) {
    console.log(`${root.kind}: ${root.name}`);
    for (const child of root.children ?? []) {
        console.log(`  ${child.kind}: ${child.name} (stale: ${child.is_stale})`);
    }
}

CLI

# Show the folder/page tree (no page bodies)
hindsight knowledge-base tree "$BANK_ID"

Go

// Fetch the whole knowledge base as a nested folder/page tree (no page bodies)
tree, _, _ := client.KnowledgeBaseAPI.GetKnowledgeBaseTree(ctx, kpBankID).Execute()

for _, root := range tree.Roots {
	fmt.Printf("%s: %s\n", root.Kind, root.Name)
	for _, child := range root.Children {
		fmt.Printf("  %s: %s (stale: %v)\n", child.Kind, child.Name, child.GetIsStale())
	}
}
{
  "roots": [
    {
      "id": "kf-9f2c...",
      "kind": "folder",
      "name": "Operations",
      "parent_id": null,
      "mental_model_id": null,
      "managed": false,
      "description": null,
      "tags": [],
      "timestamp": "2026-08-01T11:04:02+00:00",
      "is_stale": null,
      "children": [
        {
          "id": "kp-2e85...",
          "kind": "page",
          "name": "Deploying the API",
          "parent_id": "kf-9f2c...",
          "mental_model_id": "mm-77ab...",
          "managed": false,
          "description": "How is the API deployed?",
          "tags": ["ops"],
          "timestamp": "2026-08-03T09:12:44+00:00",
          "is_stale": true,
          "children": []
        }
      ]
    }
  ]
}
Field Description
kind folder or page
mental_model_id The backing mental model (pages only)
description The page's source query — the question that rebuilds it
timestamp Last refresh for a page, last update for a folder
is_stale Pages only: true when a memory in this page's scope has been written since the page last read the memories (see below)
last_refresh_failed_at Pages only: when this page's last refresh failed, or null when it succeeded. While it is set the page does not rebuild itself on its trigger — an explicit refresh still runs, and a successful one clears it
managed true when the node is flagged as system-owned rather than hand-authored

How is_stale is decided

Each page is answered against its own scope — its tags and its fact_types — using the same staleness check that decides whether a scheduled refresh does any work. A flagged page is one a refresh would actually rewrite; an unflagged one is a page a refresh would leave alone. Activity elsewhere in the bank does not flag a page whose own scope is quiet.

The whole tree is answered in one query, so the flag costs the same whether the bank has three pages or three hundred, and GET /mental-models/{id} returns the identical value for the page's backing model.

One thing it does not see: deletions. The check asks what has been written since the page last read the memories, and deleting an in-scope memory leaves no write behind — a page that cites a deleted fact keeps reporting itself up to date.


Create a Page

Creating a page stores it with placeholder content and schedules the first build in the background. Poll the returned operation_id via the operations API to know when the content is ready.

Python

# Create a page — content is generated in the background
page = client.create_knowledge_page(
    BANK_ID,
    name="Deploying the API",
    source_query="How is the API deployed?",
    parent_id=folder.id,
    tags=["ops", "type:runbook"],
)

# Poll the operation to know when the first build has finished
print(f"Page ID: {page.page_id}, operation: {page.operation_id}")

Node.js

// Create a page — content is generated in the background
const page = await client.createKnowledgePage(
    BANK_ID,
    'Deploying the API',
    'How is the API deployed?',
    { parentId: folder.id, tags: ['ops', 'type:runbook'] },
);

// Poll the operation to know when the first build has finished
console.log(`Page ID: ${page.page_id}, operation: ${page.operation_id}`);

CLI

# Create a page — content is generated in the background
hindsight knowledge-base create-page "$BANK_ID" \
  "Deploying the API" \
  "How is the API deployed?" \
  --parent-id "$FOLDER_ID" \
  --tags ops,type:runbook

Go

// Create a page — content is generated in the background
page, _, _ := client.KnowledgeBaseAPI.CreateKnowledgePage(ctx, kpBankID).
	CreatePageRequest(hindsight.CreatePageRequest{
		Name:        "Deploying the API",
		SourceQuery: "How is the API deployed?",
		ParentId:    *hindsight.NewNullableString(&folder.Id),
		Tags:        []string{"ops", "type:runbook"},
	}).Execute()

// Poll the operation to know when the first build has finished
fmt.Printf("Page ID: %s, operation: %s\n", page.PageId, page.GetOperationId())
{
  "page_id": "kp-2e85...",
  "mental_model_id": "mm-77ab...",
  "operation_id": "op-1d0f..."
}

Parameters

Parameter Type Required Description
name string Yes Page name. Must be unique within its folder, case-insensitively — a duplicate returns 409. (Enforced on PostgreSQL only.)
source_query string Yes The question the page answers, re-asked on every refresh.
parent_id string No Folder to create the page in. null (or omitted) creates it at the root.
tags list No Tags that scope which memories the page is built from — see Tags Are a Filter. A type:<x> tag also sets the page's rendered type, and still counts as part of the filter.
max_tokens int No Content budget. Defaults to 4096 (a plain mental model defaults to 2048).
trigger object No Refresh configuration — see below.

Tags Are a Filter

tags are not labels on the page. They are the scope the page is built from, and a tagged page matches memories with all_strict by default: a memory must carry every one of the page's tags, and untagged memories are excluded entirely.

So a page created like this:

{
  "name": "Homelab Infrastructure",
  "source_query": "NAS, ThinkPad, docker containers, jellyfin",
  "tags": ["type:runbook", "homelab", "infrastructure"]
}

is built only from memories tagged type:runbook and homelab and infrastructure. If your memories were retained without those exact tags — which is the usual case when the tags are invented at page-creation time to describe the topic — the page matches nothing and generates as "I don't have information about this." A direct recall for the same query still returns everything, because recall was not given the same filter.

The type:<x> tag makes this easy to trip over: it is documented as setting the page's rendered type, but it narrows retrieval like any other tag.

Three ways to get this right:

You want Do this
The page built from the whole bank Omit tags (or pass [])
The tags to scope the page, but untagged memories still included Keep tags, add "trigger": {"tags_match": "all"}
Only memories carrying every tag (strict isolation, e.g. per-user pages) Keep tags and the all_strict default

To repair a page that already generated empty, PATCH it with {"tags": []} or with the widened tags_match, then refresh it — the tags are stored on the backing mental model, not baked into the content.

See tag matching modes for the full semantics of any, all, any_strict, all_strict, and exact.

Default Trigger

When trigger is omitted, the page is created with a document-oriented configuration:

{
  "mode": "delta",
  "fact_types": ["observation"],
  "exclude_mental_models": true,
  "refresh_after_consolidation": true
}

This makes the page a living document built from consolidated observations only, refreshed incrementally whenever consolidation produces new knowledge in its scope, and never influenced by other pages.

Setting Why
fact_types: ["observation"] The page reads consolidated beliefs, not the raw conversational noise underneath them. Observations are already deduplicated and evidence-backed, so a page reads as a settled document instead of a transcript. Enforced structurally — with only observation in scope, the refresh agent isn't given the raw-memory recall tool at all.
exclude_mental_models: true A page never reflects on sibling pages. Without this, pages would cite each other and drift into a feedback loop where one wrong claim propagates across the knowledge base.
mode: "delta" Each refresh edits the existing document with what is new since the last refresh instead of regenerating it, so hand-tuned structure and wording survive. See Refresh Mode.
refresh_after_consolidation: true The page rewrites itself whenever consolidation produces new knowledge in its scope — gated by the same staleness check as any mental model, so unrelated bank activity doesn't trigger rebuilds.

Page Lifecycle

  1. Create — the page is stored with placeholder content and a background refresh is submitted; the call returns immediately with an operation_id.
  2. First build — a full generation, since there is no prior document to edit.
  3. Consolidation — new memories are retained and consolidated into observations.
  4. Staleness check — pages whose trigger asks for it are checked against their own scope (tags and fact_types both apply).
  5. Delta refresh — stale pages are rewritten by editing the existing document with the new observations only.

Observations are what a page is built from, but it can still inspect the evidence underneath them: the refresh agent can expand a memory to its original chunk or document (unless store_document_text is disabled for the bank), and if HINDSIGHT_API_REFLECT_SOURCE_FACTS_MAX_TOKENS is enabled — off by default — observation search also returns each observation's grounding facts.

ℹ️ Info

A supplied trigger is a patch: only the fields you actually send are applied, and the rest keep the defaults above. Sending {"trigger": {"tags_match": "all"}} widens the tag filter and leaves mode, fact_types, exclude_mental_models, and refresh_after_consolidation as they are. The one exception is the two refresh triggers, which stay mutually exclusive: setting refresh_cron clears refresh_after_consolidation, and vice versa. Every mental model trigger setting is accepted here — including refresh_cron for scheduled rebuilds instead of consolidation-driven ones, and tag_groups for compound tag scoping.


Create a Folder

Python

# Create a folder (omit parent_id, or pass None, to create it at the root)
folder = client.create_knowledge_folder(BANK_ID, name="Operations")

print(f"Folder ID: {folder.id}")

Node.js

// Create a folder (omit parentId, or pass null, to create it at the root)
const folder = await client.createKnowledgeFolder(BANK_ID, 'Operations');

console.log(`Folder ID: ${folder.id}`);

CLI

# Create a folder (omit --parent-id to create it at the root)
hindsight knowledge-base create-folder "$BANK_ID" "Operations"

Go

// Create a folder (leave ParentId unset to create it at the root)
folder, _, _ := client.KnowledgeBaseAPI.CreateKnowledgeFolder(ctx, kpBankID).
	CreateFolderRequest(hindsight.CreateFolderRequest{Name: "Operations"}).
	Execute()

fmt.Printf("Folder ID: %s\n", folder.Id)
Parameter Type Required Description
name string Yes Folder name
parent_id string No Parent folder. Omit or pass null for the root.

A parent_id that does not exist, or that points at a page rather than a folder, returns 400.


Read a Page

Returns the page rendered as a markdown document.

Python

# Read a page as a markdown document
document = client.get_knowledge_page(BANK_ID, page.page_id)

print(document.type)      # "runbook" — from the type:runbook tag
print(document.body)      # the synthesized markdown body
print(document.markdown)  # YAML frontmatter + body

Node.js

// Read a page as a markdown document
const document = await client.getKnowledgePage(BANK_ID, page.page_id);

console.log(document.type);      // "runbook" — from the type:runbook tag
console.log(document.body);      // the synthesized markdown body
console.log(document.markdown);  // YAML frontmatter + body

CLI

# Read a page as a markdown document
hindsight knowledge-base get-page "$BANK_ID" "$PAGE_ID"

Go

// Read a page as a markdown document
document, _, _ := client.KnowledgeBaseAPI.GetKnowledgePage(ctx, kpBankID, page.PageId).Execute()

fmt.Println(document.Type)      // "runbook" — from the type:runbook tag
fmt.Println(document.GetBody()) // the synthesized markdown body
fmt.Println(document.Markdown)  // YAML frontmatter + body
{
  "id": "kp-2e85...",
  "name": "Deploying the API",
  "type": "runbook",
  "description": "How is the API deployed?",
  "tags": ["ops"],
  "timestamp": "2026-08-03T09:12:44+00:00",
  "body": "# Deploying the API\n\n...",
  "markdown": "---\nid: \"kp-2e85...\"\ntype: \"runbook\"\n...\n---\n\n# Deploying the API\n\n..."
}
  • body is the synthesized markdown body on its own.
  • markdown is the full document: a YAML frontmatter block (id, type, title, description, tags, timestamp) followed by the body.
  • type comes from a type:<x> tag and defaults to knowledge-page. The type: tag is removed from the returned tags.

Search Pages

Document-level hybrid search: a full-text (BM25) match and a vector-similarity match, fused with Reciprocal Rank Fusion. There is no reranking step, which keeps it fast enough to be an agent's first call.

Python

# Hybrid search (full-text + vector) over whole pages
results = client.search_knowledge_base(BANK_ID, q="how do we deploy", limit=5)

for hit in results.results:
    print(f"{hit.score:.3f}  {hit.name}: {hit.snippet}")

Node.js

// Hybrid search (full-text + vector) over whole pages
const results = await client.searchKnowledgeBase(BANK_ID, 'how do we deploy', { limit: 5 });

for (const hit of results.results) {
    console.log(`${hit.score.toFixed(3)}  ${hit.name}: ${hit.snippet}`);
}

CLI

# Hybrid search (full-text + vector) over whole pages
hindsight knowledge-base search "$BANK_ID" "how do we deploy" --limit 5

Go

// Hybrid search (full-text + vector) over whole pages
results, _, _ := client.KnowledgeBaseAPI.SearchKnowledgeBase(ctx, kpBankID).
	Q("how do we deploy").Limit(5).Execute()

for _, hit := range results.Results {
	fmt.Printf("%.3f  %s: %s\n", hit.Score, hit.Name, hit.Snippet)
}
{
  "results": [
    {
      "id": "kp-2e85...",
      "name": "Deploying the API",
      "mental_model_id": "mm-77ab...",
      "snippet": "The API is deployed via ...",
      "score": 0.032,
      "updated_at": "2026-08-03T09:12:44+00:00"
    }
  ],
  "total": 1
}
Parameter Type Default Description
q string — Required. Search query (min length 1).
limit int 10 Maximum results, 1–50.
tags string[] — Only pages carrying these tags, matched per tags_match. See Filter by Tags.
tags_match string any any, all, any_strict, all_strict, or exact.
tag_groups string — JSON-encoded compound filter, same shape as recall's tag_groups.

This searches whole pages. To search individual memories, use recall.

Filter by Tags

The tree and search take the same tag filter as recall: tags with tags_match, or a compound tag_groups expression (JSON-encoded in the query string, since both are GET requests). It matches each page's own tags. Search ranks only among the pages that match, and the tree keeps only matching pages plus the folders above them. A folder with no matching page under it is left out, because its name can reveal as much as its pages.

As in recall, any and all also return untagged pages. Use any_strict, all_strict, or exact when only tagged pages should come back, for example to show a user only their own pages.

Python

# Narrow the tree and search to pages carrying a tag (recall's tags / tags_match / tag_groups)
tree = client.get_knowledge_base_tree(BANK_ID, tags=["type:runbook"], tags_match="any_strict")
ops_hits = client.search_knowledge_base(BANK_ID, q="how do we deploy", tags=["ops"], tags_match="all_strict")
print(f"{len(tree.roots)} runbook roots, {len(ops_hits.results)} ops hits")

# Compound filters use tag_groups: here, runbooks that are not drafts
not_drafts = client.search_knowledge_base(
    BANK_ID,
    q="how do we deploy",
    tag_groups=[{"and": [{"tags": ["type:runbook"]}, {"not": {"tags": ["draft"]}}]}],
)
print(f"{len(not_drafts.results)} non-draft runbooks")

Node.js

// Narrow the tree and search to pages carrying a tag (recall's tags / tags_match / tag_groups)
const runbooks = await client.getKnowledgeBaseTree(BANK_ID, { tags: ['type:runbook'], tagsMatch: 'any_strict' });
const opsHits = await client.searchKnowledgeBase(BANK_ID, 'how do we deploy', { tags: ['ops'], tagsMatch: 'all_strict' });
console.log(`${runbooks.roots.length} runbook roots, ${opsHits.results.length} ops hits`);

// Compound filters use tagGroups: here, runbooks that are not drafts
const notDrafts = await client.searchKnowledgeBase(BANK_ID, 'how do we deploy', {
    tagGroups: [{ and: [{ tags: ['type:runbook'] }, { not: { tags: ['draft'] } }] }],
});
console.log(`${notDrafts.results.length} non-draft runbooks`);

CLI

# Narrow the tree and search to pages carrying a tag (recall's tags / tags_match / tag_groups)
hindsight knowledge-base tree "$BANK_ID" --tags type:runbook --tags-match any_strict
hindsight knowledge-base search "$BANK_ID" "how do we deploy" --tags ops --tags-match all_strict

# Compound filters use --tag-groups: here, runbooks that are not drafts
hindsight knowledge-base search "$BANK_ID" "how do we deploy" \
    --tag-groups '[{"and":[{"tags":["type:runbook"]},{"not":{"tags":["draft"]}}]}]'

Go

// Narrow the tree and search to pages carrying a tag (recall's tags / tags_match / tag_groups)
runbooks, _, _ := client.KnowledgeBaseAPI.GetKnowledgeBaseTree(ctx, kpBankID).
	Tags([]string{"type:runbook"}).TagsMatch("any_strict").Execute()
fmt.Printf("runbook roots: %d\n", len(runbooks.Roots))

// Compound filters use tag_groups, sent as one JSON-encoded query param
notDrafts, _, _ := client.KnowledgeBaseAPI.SearchKnowledgeBase(ctx, kpBankID).
	Q("how do we deploy").
	TagGroups(`[{"and":[{"tags":["type:runbook"]},{"not":{"tags":["draft"]}}]}]`).Execute()
fmt.Printf("non-draft hits: %d\n", len(notDrafts.Results))

Update or Move a Node

One PATCH renames a node, moves it, and/or updates a page's options. Each field applies only when present in the body.

Python

# Rename a node, move it, and/or update a page's options.
# Changing source_query rebuilds the page against the new question.
client.update_knowledge_node(
    BANK_ID,
    page.page_id,
    name="Deploying the API (v2)",
    tags=["ops", "type:runbook", "reviewed"],
)

Node.js

// Rename a node, move it, and/or update a page's options.
// Changing sourceQuery rebuilds the page against the new question.
await client.updateKnowledgeNode(BANK_ID, page.page_id, {
    name: 'Deploying the API (v2)',
    tags: ['ops', 'type:runbook', 'reviewed'],
});

CLI

# Rename a node, move it, and/or update a page's options.
# Changing --source-query rebuilds the page against the new question.
hindsight knowledge-base update "$BANK_ID" "$PAGE_ID" \
  --name "Deploying the API (v2)" \
  --tags ops,type:runbook,reviewed

Go

// Rename a node, move it, and/or update a page's options.
// Changing SourceQuery rebuilds the page against the new question.
client.KnowledgeBaseAPI.UpdateKnowledgeNode(ctx, kpBankID, page.PageId).
	UpdateNodeRequest(hindsight.UpdateNodeRequest{
		Name: *hindsight.NewNullableString(hindsight.PtrString("Deploying the API (v2)")),
		Tags: []string{"ops", "type:runbook", "reviewed"},
	}).Execute()
Parameter Type Applies to Description
name string Both New name
parent_id string | null Both New parent folder. Pass null explicitly to move to the root.
source_query string Pages New question. Changing it schedules an async refresh so the page rebuilds against the new question.
tags list Pages Replaces the page's tags, and so the scope it is built from. Pass [] to clear them and rebuild the page from the whole bank.
max_tokens int Pages New content budget
trigger object Pages Refresh settings to change, applied as a patch. {"tags_match": "all"} keeps the page's tags but stops excluding untagged memories.

Sending an empty body returns 400; an unknown node returns 404.


Delete a Node

Deletes a folder or page and its entire subtree. The mental models backing the deleted pages are removed too.

Python

# Delete a folder or page — deleting a folder removes its whole subtree
client.delete_knowledge_node(BANK_ID, folder.id)

Node.js

// Delete a folder or page — deleting a folder removes its whole subtree
await client.deleteKnowledgeNode(BANK_ID, folder.id);

CLI

# Delete a folder or page — deleting a folder removes its whole subtree
hindsight knowledge-base delete "$BANK_ID" "$FOLDER_ID" -y

Go

// Delete a folder or page — deleting a folder removes its whole subtree
client.KnowledgeBaseAPI.DeleteKnowledgeNode(ctx, kpBankID, folder.Id).Execute()
{"status": "deleted"}

Export as a Markdown Bundle

Returns the whole knowledge base as a flat set of portable markdown files: a nested index.md, one <page-id>.md per page, and a <page-id>.log.md refresh history for pages that have been rebuilt.

Python

# Export the knowledge base as a portable markdown bundle
bundle = client.export_knowledge_base(BANK_ID)

for file in bundle.files:
    print(file.path)  # index.md, <page-id>.md, <page-id>.log.md

Node.js

// Export the knowledge base as a portable markdown bundle
const bundle = await client.exportKnowledgeBase(BANK_ID);

for (const file of bundle.files) {
    console.log(file.path);  // index.md, <page-id>.md, <page-id>.log.md
}

CLI

# Export the knowledge base as a portable markdown bundle
hindsight knowledge-base export "$BANK_ID"

Go

// Export the knowledge base as a portable markdown bundle
bundle, _, _ := client.KnowledgeBaseAPI.ExportKnowledgeBase(ctx, kpBankID).Execute()

for _, file := range bundle.Files {
	fmt.Println(file.Path) // index.md, <page-id>.md, <page-id>.log.md
}
{
  "files": [
    {"path": "index.md", "content": "---\ntype: \"index\"\n..."},
    {"path": "kp-2e85....md", "content": "---\nid: \"kp-2e85...\"\n..."},
    {"path": "kp-2e85....log.md", "content": "---\ntype: \"log\"\n..."}
  ]
}

💡 Mirror it to disk

hindsight fs mount --bank my-bank keeps a local folder in sync with this bundle via a background refresh loop, so ls, grep, rg, and your editor work against real files.

Storage

Table Holds
knowledge_pages The tree: folders and pages, their names, parents, and ordering. A page row references its backing mental model; a folder row has none.
mental_models The content: the document body, its source query, tags, token budget, trigger, and refresh history.

The page layer owns only tree structure — everything about the content lives on the backing mental model, which is why every mental model capability applies to pages unchanged.


Endpoint Summary

Method Path Description
GET /knowledge-base/tree Nested folder/page tree with staleness
POST /knowledge-base/folders Create a folder
POST /knowledge-base/pages Create a page (async first build)
GET /knowledge-base/pages/{page_id} Read a page as markdown
GET /knowledge-base/search Hybrid search over pages
PATCH /knowledge-base/nodes/{node_id} Rename, move, or reconfigure a node
DELETE /knowledge-base/nodes/{node_id} Delete a node and its subtree
GET /knowledge-base/export Export the whole base as markdown files

Source: SKILL.md on GitHub

1 alerttoday5 checks · Risk SAFE
  • Gen Agent Trust Hubtoday

    The skill is a comprehensive documentation set for the Hindsight memory system, providing architecture overviews, API references, and integration guides for multiple AI agent frameworks. No security risks were identified in the documentation or provided examples.

  • Sockettoday

    No alerts

  • Snyktoday

    Risk: LOW · No issues

  • Runlayer6mo

    30/42 files flagged

  • ZeroLeaks5mo

    Score: 93/100 · 2 sections analyzed

Signed by skilld at 5bfef3c. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 15 hours ago.

Activeupdated 2 months ago

README badge

README badge for vectorize-io/hindsight/hindsight-docs