Chatting with Text, Capturing Insight: The hyperscope-capture-derived Workflow
RCD Notes & Hyperscope — The Dynamic Knowledge Repository for GNU Emacs
Introduction
In the RCD Notes & Hyperscope system, every hyperdocument is a node in
a living knowledge graph. The hyperscope-llm-chat-with-column
function lets you take the text of a hyperdocument, hand it to an LLM
as context, and ask questions about it. The
hyperscope-capture-derived function completes the loop: it takes the
LLM’s answer — displayed in a buffer — and records it as a new
hyperdocument, linked to the original as its CHILD OF
relationship.
This is not merely a convenience. It is a concrete implementation of Doug Engelbart’s vision of a Dynamic Knowledge Repository (DKR) and CoDIAK — the Concurrent Development, Integration, and Application of Knowledge. Every question you ask, and every answer you receive, becomes part of the group brain: captured, connected, and navigable.
Why Chat with Text?
A hyperdocument in Hyperscope is more than a note — it is a structured artifact with a name, a type, a subtype, markup, and body text. When you “chat with text,” you are doing something profound:
You interrogate your own knowledge. The LLM reads the context you have accumulated and answers questions about it. This turns a static archive into a conversational partner.
You surface what is implicit. The answer may connect facts, reveal gaps, or propose interpretations that were not obvious when the text was written.
You create new knowledge. The LLM’s answer is itself a new artifact — a synthesis, a summary, a design rationale, a course correction.
The hyperscope-llm-chat-with-column function extracts data from a
specific column of a hyperdocument, uses the column’s comment as
additional context, and sends it to the LLM with your question. The
response opens in a new buffer — a draft of new knowledge, waiting to
be captured.
Why Capture the Answer as CHILD OF the Original?
The critical design decision in hyperscope-capture-derived is the
CHILD OF (type 5) relationship. The new hyperdocument is not a
loose, floating note — it is explicitly derived from its parent. This
matters for several reasons:
1. Provenance and Traceability
Every captured answer carries its lineage. When you later encounter the derived hyperdocument, you can follow the relationship back to the original text that produced it. You know what the answer was, where it came from, and why it exists. This is the difference between a pile of notes and a knowledge repository.
2. The Group Brain
Engelbart’s DKR is a “living, evolving collection of all artifacts generated throughout a project.” The CHILD OF relationship is how the group brain connects the dots. A question asked today, answered by an LLM, and captured as a child of its source text — that is a new synapse in the collective intelligence. Future readers (human or machine) can traverse the graph and see not just the answer, but the reasoning path that produced it.
3. Design Rationale and Historical Context
When a team makes a course correction, the DKR should preserve why. The captured LLM answer — stored as a child of the document it interprets — becomes part of the historical context. Months later, the question “why did we decide this?” can be answered by walking the relationship graph.
4. CoDIAK in Practice
CoDIAK describes the core processes of developing, integrating, and applying knowledge. The workflow embodies all three:
| CoDIAK Process | How hyperscope-capture-derived Fulfills It |
|---|---|
| Develop | The LLM generates a new artifact (the answer) from existing text |
| Integrate | The answer is inserted into the hyobjects table and linked via relatedhyperdocuments |
| Apply | Both IDs are opened with hyperscope-by-id-list, making the new knowledge immediately navigable |
The Workflow in Practice
Step 1: Chat with Text
From a hyperdocument, invoke hyperscope-llm-chat-with-column. The function:
- Extracts data from the selected column of the hyperdocument(s)
- Uses the column’s comment as context
- Sends the data plus your question to the LLM
- Opens the response in a new buffer
Step 2: Review the Answer
The response buffer is your draft. You can edit it, refine it, or leave it as-is. The buffer’s major mode is markdown-mode (or text-mode), and it is not a Hyperscope list buffer — which is exactly what hyperscope-capture-derived requires.
Step 3: Capture as Derived
Invoke hyperscope-capture-derived. The function:
- Verifies the context — the buffer must not be in
hyperscope-list-modeortabulated-list-mode; it must betext-mode,markdown-mode, orfundamental-mode. - Validates the source —
rcd-db-current-tablemust be"hyobjects",rcd-db-current-table-idmust be an integer, and that ID must exist in the database. - Captures the buffer — the full contents become the new hyperdocument’s text.
- Inserts the hyperdocument — with:
- Name:
Chat with text: <original name> - Markup:
2(markdown) or1(plain text) - Type:
70 - Subtype:
1 - Text: the buffer contents
- Name:
- Records the relationship — the new ID is linked to the original ID via
hyperscope-relate-hyperdocumentswith relation type5(CHILD OF). - Opens both —
hyperscope-by-id-listdisplays the original and the derived hyperdocument together.
The Relationship Structure
The relatedhyperdocuments table stores the link between the original and the derived hyperdocument:
| Column | Value |
|---|---|
relatedhyperdocuments_hyperdocument1 |
New (derived) hyperdocument ID |
relatedhyperdocuments_hyperdocument2 |
Original hyperdocument ID |
relatedhyperdocuments_description |
(empty) |
relatedhyperdocuments_hyobjectrelationtypes |
5 (CHILD OF) |
The relationship is directional: the derived hyperdocument is a child of the original. This mirrors the natural structure of knowledge — answers derive from sources, conclusions derive from evidence, and insights derive from data.
Why This Matters for Collective IQ
Engelbart’s ultimate goal was to boost Collective IQ — the ability of a group to solve complex problems by leveraging shared knowledge. The hyperscope-capture-derived workflow is a small but essential piece of that vision:
- Every interaction leaves a trace. Nothing is lost. Questions, answers, and interpretations accumulate in the repository.
- Knowledge is connected, not isolated. The CHILD OF relationship ensures that new knowledge is always anchored to its source.
- The repository is dynamic. It grows with every conversation, every capture, every relationship. It is not a static archive — it is a living system.
- The dots are connected. When a future reader encounters the derived hyperdocument, they can traverse to the original, understand the context, and see the reasoning path.
The Function
```elisp (defun hyperscope-capture-derived () “Capture current buffer as a new hyperdocument derived from the current one.
The function:
1. Ensures the buffer is not in hyperscope-list-mode' or
tabulated-list-mode' (allows text-mode',markdown-mode',
fundamental-mode').
2. Validates thatrcd-db-current-table' is \“hyobjects\” and
rcd-db-current-table-id' is a valid hyperdocument ID.
3. Captures buffer content and inserts it as a new hyperdocument.
4. Records a CHILD OF (type 5) relationship from the new ID to
the original ID.
5. Opens both IDs withhyperscope-by-id-list'.“
(interactive)
(cond ((derived-mode-p ‘hyperscope-list-mode)
(user-error "Cannot capture derived content from hyperscope-list-mode”))
((derived-mode-p 'tabulated-list-mode)
(user-error “Cannot capture derived content from tabulated-list-mode”)))
(unless (string= rcd-db-current-table “hyobjects”) (user-error “rcd-db-current-table is not \"hyobjects\”: %s" rcd-db-current-table)) (unless (integerp rcd-db-current-table-id) (user-error “rcd-db-current-table-id is not an integer: %s” rcd-db-current-table-id)) (unless (hyperscope-id-p rcd-db-current-table-id) (user-error “hyobjects_id %s does not exist in the database” rcd-db-current-table-id))
(let* ((original-id rcd-db-current-table-id) (original-name (hyperscope-name original-id)) (new-name (format “Chat with text: %s” original-name)) (buffer-text (buffer-substring-no-properties (point-min) (point-max))) (is-markdown (derived-mode-p ‘markdown-mode)) (new-id (hyperscope-add-generic new-name ;; name nil ;; link (if is-markdown 2 1) ;; markup 70 ;; type 1 ;; subtype nil ;; set nil ;; argument “derived” ;; description nil ;; date nil ;; person nil ;; assignee nil ;; people-list buffer-text))) ;; text (hyperscope-relate-hyperdocuments new-id original-id nil 5) (hyperscope-by-id-list (list original-id new-id) (format “Recorded chat with text: %s” original-name)))) ```
Conclusion
The hyperscope-capture-derived workflow transforms a one-off LLM conversation into a permanent, connected artifact in your Dynamic Knowledge Repository. It embodies Engelbart’s vision: knowledge that is captured in real time, integrated into the group brain, and navigable through relationships. Every chat with text becomes a node in the graph — a child of its source, ready to inform the next question, the next decision, and the next course correction.
The highest potential of a DKR is realized as a dynamic, searchable, and highly navigable resource that significantly boosts collective intelligence. — and with hyperscope-capture-derived, every LLM answer becomes part of that resource.