Data Flow

How data moves through ContextManager’s pipelines - from capture to injection.

---

Three Pipelines

ContextManager has three main data pipelines that operate concurrently:

1. Tool Pipeline — LM tool invocations → immediate processing
2. Auto-Capture Pipeline - all chat responses → background processing
3. Hook Pipeline - VS Code Copilot transcript → file-based queue

---

1. Tool Pipeline

When agents invoke ContextManager’s Language Model Tools, data flows synchronously:

graph TD
    A[Agent invokes #ctx or #saveCard] --> B[Tool handler receives request]
    B --> C[Query project data]
    C --> D[Return structured result]
    D --> E[Post-processing]
    E --> F[Auto-capture records observation]

    style A fill:#1f6feb,stroke:#388bfd,color:#fff
    style C fill:#1158c7,stroke:#388bfd,color:#fff
    style F fill:#238636,stroke:#3fb950,color:#fff

---

2. Auto-Capture Pipeline

Runs in the background for all chat participants:

graph TD
    A[Any Model Response] --> B[AutoCaptureService]
    B --> C[Strip private tags]
    C --> D{Content hash dedup}
    D -->|New| E[Classify observation type]
    D -->|Duplicate| X[Skip]
    E --> F[Extract file paths]
    F --> G[Calculate token economics]
    G --> H[Add to observation buffer]
    H --> I[Update FTS index]
    I --> J{autoLearn enabled?}
    J -->|Yes| K[LLM extraction]
    K --> L[Conventions + Working Notes]
    J -->|No| N[Done]

    style A fill:#1f6feb,stroke:#388bfd,color:#fff
    style H fill:#1158c7,stroke:#388bfd,color:#fff
    style K fill:#9e6a03,stroke:#d29922,color:#fff
    style L fill:#238636,stroke:#3fb950,color:#fff
    style X fill:#21262d,stroke:#30363d,color:#8b949e

---

3. Hook Pipeline

Captures from VS Code Copilot’s native transcript system:

graph TD
    A[Copilot processes tool call] --> B[Chat Hooks API fires PostToolUse]
    B --> C[capture.ps1 runs]
    C --> D[Write PostToolUse entry]
    C --> E[Find Copilot transcript]
    E --> F[Get-LastCompletedTurn]
    F --> G{New turn?}
    G -->|Yes| H[Write synthetic Stop entry]
    G -->|No| I[Skip]
    H --> J[HookWatcher detects change]
    J --> K[Parse JSONL entries]
    K --> L{Route by hookType}
    L --> L1[Stop: autoCapture + cardQueue]
    L --> L2[PostToolUse: observation]
    L --> L3[PreCompact: multi-turn]

    style A fill:#1f6feb,stroke:#388bfd,color:#fff
    style C fill:#9e6a03,stroke:#d29922,color:#fff
    style J fill:#1158c7,stroke:#388bfd,color:#fff
    style L1 fill:#238636,stroke:#3fb950,color:#fff
    style L2 fill:#238636,stroke:#3fb950,color:#fff
    style L3 fill:#238636,stroke:#3fb950,color:#fff
    style I fill:#21262d,stroke:#30363d,color:#8b949e

---

Storage Architecture

Disk-Backed JSON

All project data persists to JSON files in globalStorageUri:

globalStorageUri/
├── projects.json              # All project metadata (single file for all projects)
├── observations.json          # All observations (managed by AutoCaptureService)
└── search-fts4.db             # SQLite FTS4 search index

In-memory caching: Data loaded once at startup, served from memory on read, flushed to disk on every write. Zero I/O overhead for reads.

File-Based Queue

The hook system uses a simple append-only JSONL file:

~/.contextmanager/
├── hook-queue.jsonl      # Append-only entries from capture.ps1
├── .queue-offset         # Byte offset (HookWatcher resumption point)
├── session-context.txt   # Written by HookWatcher (project name + root path only)
├── knowledge-index.txt   # Card index (actively written for PreCompact hook)
├── seen-turn-<sid>       # Dedup files (last processed userId per session)
├── stop-debug.log        # Debug log for Stop hook
└── scripts/
    ├── capture.ps1       # The hook script (Windows)
    └── capture.sh        # The hook script (Linux/macOS)

---

Context Injection Flow

When context is injected into a prompt:

graph TD
    A[Project data changes] --> B[syncInstructions triggers]
    B --> C[Write copilot-instructions.md managed block]
    C --> D[Pinned cards + tool discovery]
    D --> G[Always included by VS Code on every request]

    E["Agent invokes #ctx tool"] --> F[On-demand search / getCard / learn]
    F --> H[Context returned to agent]

    I["Agent invokes #getCard"] --> J[Knowledge cards returned]
    J --> H

    G --> K[Language model receives context]
    H --> K

    style A fill:#1f6feb,stroke:#388bfd,color:#fff
    style C fill:#238636,stroke:#3fb950,color:#fff
    style G fill:#1158c7,stroke:#388bfd,color:#fff
    style E fill:#9e6a03,stroke:#d29922,color:#fff
    style K fill:#238636,stroke:#3fb950,color:#fff

---

Token Budget Summary

Component	Setting	Default	Range
Intelligence Tier 1	intelligence.tier1MaxTokens	400 tokens	100–1000
Intelligence Tier 2	intelligence.tier2MaxTokens	400 tokens	100–1000
Knowledge Cards	maxKnowledgeCardsInContext	10 cards	1–20

---

Next Steps

Hook System →