Maintain multiple granularity levels for AI knowledge retrieval
When using AI systems with your knowledge base, maintain multiple granularity levels of the same material (fine-grained for precise retrieval, coarse-grained for contextual reasoning) rather than forcing a single chunk size, because different query types require different resolutions.
Why This Is a Rule
RAG systems and semantic search have a fundamental tension: precise factual queries ("what is the threshold for X?") need small, atomic chunks to avoid burying the answer in context. Broad reasoning queries ("what are the tradeoffs between approaches A and B?") need large chunks that preserve relationships between ideas. A single chunk size optimizes for one query type at the expense of the other.
This rule resolves the tension architecturally: maintain the same material at multiple granularity levels simultaneously. Fine-grained atoms for lookup, coarse-grained summaries for reasoning, and mid-range chunks for exploration. The storage cost is trivial compared to the retrieval quality difference.
When This Fires
- Setting up a personal knowledge base that will be queried by AI (Obsidian + Copilot, Notion AI, custom RAG)
- Designing embedding strategies for semantic search systems
- Noticing that AI retrieval quality is inconsistent — sometimes precise, sometimes missing obvious connections
- Building any system where humans write notes that AI will later retrieve
Common Failure Mode
Choosing a single "optimal" chunk size based on average performance across query types. This produces mediocre results everywhere instead of excellent results anywhere. The 512-token chunk that works for factual lookup fragments the narrative context needed for synthesis. The 2,000-token chunk that preserves context buries specific facts in noise.
The Protocol
For each piece of knowledge, maintain at least two representations: (1) an atomic note — one claim, one fact, one decision — tagged for precise retrieval, and (2) a contextual summary that preserves the relationships between related atomic notes. When querying, match the granularity to the question type. Factual questions go to atoms. Reasoning questions go to summaries.