Incremental ingestion of chats and issue lists into OpenClaw with knowledge-source-ingest
機械翻訳 / Machine-translated
Original article: Incremental ingestion of chats and issue lists into OpenClaw with knowledge-source-ingest (cccskills)
Skill page: knowledge-source-ingest
Key takeaways from this article
If you're periodically ingesting chats or issue lists into OpenClaw's Knowledge, knowledge-source-ingest can help.
It fetches data within a defined range, compares it against the previous fetch, and lets you review additions, changes, and deletions before applying them.
The fetch range and a snapshot are recorded every time.
This article explains what the returned results contain, how to install the skill, example requests, and what to verify before applying changes.
When you use this skill, you first receive a "diff plan" — the fetched data compared against the previous fetch.
Applying the changes to Knowledge comes after.
Here is an example request:
Please create a manifest from the Discord channel export for September 21 through 28 and compare it with the previous fetch.
The agent starts by pulling messages for the specified period using a read-only adapter for Discrawl.
The period boundary is "start time inclusive, end time exclusive."
The results are saved as a JSONL file containing each poster's stable ID.
A human-readable transcript that can be cited by line number is also generated separately, for cases where you want to review the content manually.
Next, a manifest (an index of the fetched content) is created by listing each message's ID alongside a hash of its content.
The manifest is then compared against past manifests whose ranges overlap, and the following are returned: covered and uncovered periods, plus five categorized lists:
| Category | Treatment |
|---|---|
| Added | Ingest as a new item |
| Changed | Reconcile against the existing page or entry and update |
| Deleted | Treated as an observation that the item was removed at the source; Knowledge is not deleted |
| Unchanged | Reuse saved data and records |
| Unresolved | Cannot determine presence or absence (e.g., partial fetch); left unresolved |
Once changes are applied, the snapshot page is published.
It records the fetch range, observation time, item count, hashes, and whether the fetch was complete.
Finally, each page and attachment is read back to verify that hashes and ranges match.
This skill's role is to manage external data in terms of "where it came from, what range was fetched, and when."
The work of organizing the actual content of fetched conversations or documents is handed off to separate ingestion skills.
Sources are identified by a combination of provider and source key, not by display name.
This means that even if a channel is renamed, it is still treated as the same source.
Each source has one canonical page, which is reused across every ingestion run.
Each fetch creates a snapshot page with the manifest attached.
The manifest stores item IDs and hash values only — it does not include the body text itself.
Only a complete-fetch snapshot can indicate that an item has disappeared within its range.
For partial fetches or cases where fetch status is unknown, missing items remain unresolved.
Before comparing, the skill searches Knowledge for past snapshots from the same source.
After verifying the attached hashes, it uses all snapshots whose ranges overlap for the comparison.
Because it does not rely on conversational memory, it won't misidentify the previous state even across multi-day workflows.
Documents and conversations are passed to knowledge-ingest; meetings are passed to knowledge-meeting-ingest.
Chats are grouped by reply thread, and knowledge-entities is used to enrich pages for people, companies, projects, and concepts.
New developments are reflected in the current-status section of the most specific existing page.
Only that section is updated — no new sections are added with each ingestion run.
Install with the following command.
When you run it, you can choose which agent to install into, such as Claude Code or Codex.
npx skills add https://github.com/openclaw/openclaw --skill knowledge-source-ingest
To have your agent install it for you, send the following message as-is:
次の Agent Skill をインストールしてください。
https://github.com/openclaw/openclaw/tree/main/extensions/knowledge/skills/knowledge-source-ingest
まず SKILL.md と付属ファイルの中身を確認し、不審な処理(外部への送信、認証情報の読み取りなど)がないことを確かめてから、Claude Code なら ~/.claude/skills/knowledge-source-ingest/、Codex なら ~/.agents/skills/knowledge-source-ingest/ に配置してください。
Choose the installation location based on whether you want to use the skill across all projects or only within a specific project.
| Agent | All projects (global) | Per project |
|---|---|---|
| Claude Code | ~/.claude/skills/ | .claude/skills/ |
| Codex | ~/.agents/skills/ | .agents/skills/ |
In your request, clearly state the target source, the time range, and what you want to review.
Please create a plan that lets me review added, changed, and deleted items in the issue list.
A manifest is created from the issue list fetched via approved read-only means, and a plan file comparing it against previous fetches is returned.
The plan file lists added, changed, deleted, unchanged, and unresolved items, along with covered and uncovered periods.
Please pass the fetched meeting data to the Knowledge meeting ingestion skill.
The meeting data is forwarded to knowledge-meeting-ingest.
This skill handles source identification and fetch-range recording, and keeps the snapshot as a dependency reference.
I skipped organizing people and projects last time — please continue from the saved data.
The saved source data is reused to continue the enrichment that was skipped.
Even if the manifest hasn't changed, that doesn't mean enrichment has been completed.
Before applying changes, always review items flagged as deleted and the fetch-range records.
A deletion flag only reflects what was observed at the source.
Corrections or retractions in Knowledge should only be made when sufficient evidence has been gathered.
Things to verify before installation:
To organize information about people and organizations from ingested conversations, combine this skill with knowledge-entities.
It identifies people, companies, projects, and concepts that appear in chats and documents, and maintains Knowledge pages with sourced evidence and cross-links.
In chat ingestion workflows, it is called from knowledge-source-ingest.
No.
No scheduler or persistent connection process is provided.
This is a skill that supports agent-driven execution with explicit approval.
If you want to set up periodic ingestion, use knowledge-automation first.
Yes.
Any source from which you can fetch data within a defined range is supported — issue lists, feeds, mailboxes, and so on.
However, the only bundled adapter is for Discrawl.
For other sources, fetch the data using approved read-only means and then run it through the general-purpose manifest tool.
No.
Deletion is treated as an observed state at the source; it does not authorize removing content from Knowledge.
Corrections and retractions are only made when sufficient evidence is available.
No.
The manifest only stores item IDs and content hash values.
Source data is preserved separately as evidence, within permitted boundaries.
knowledge-source-ingest is a skill for anyone who repeatedly ingests chats or issue lists into OpenClaw's Knowledge.
It fetches data within a defined range, presents a diff against the previous fetch for review, and then applies the changes.
Because the fetch range and a snapshot are saved every time, the same baseline is available for comparison on the next run.
Installation instructions and bundled files are available on the cccskills page.
This article is a reprint of the cccskills editorial piece "Incremental ingestion of chats and issue lists into OpenClaw with knowledge-source-ingest."
Skill details and installation instructions are available on the cccskills knowledge-source-ingest page.