clawtext-ingest_skill
- Python
2.5k
GitHub Stars
33
Bundled Files
2 months ago
Catalog Refreshed
3 months ago
First Indexed
Readme & install
Copy the install command, review bundled files from the catalogue, and read any extended description pulled from the listing source.
Installation
Preview and clipboard use veilstrat where the catalogue uses aiagentskills.
npx veilstrat add skill openclaw/skills --skill clawtext-ingest- _meta.json461 B
- AGENT_GUIDE.md14.7 KB
- API_REFERENCE.md9.9 KB
- ASSESSMENT_COMPLETE.md6.5 KB
- CLAWHUB_GUIDE.md8.9 KB
- CLAWHUB_PUBLICATION.md3.6 KB
- CLAWHUB_READY.md10.5 KB
- CLAWHUB_SUBMIT.md4.9 KB
- clawhub.json8.5 KB
- CLI-GUIDE.md11.3 KB
- COMPLETE_DELIVERY.md7.0 KB
- DISCORD_BOT_SETUP.md4.0 KB
- ENHANCEMENT_COMPLETE.md8.3 KB
- ENHANCEMENT_REVIEW.md12.1 KB
- IMPROVEMENTS.md9.1 KB
- INDEX.md8.8 KB
- package-lock.json68.2 KB
- package.json879 B
- PHASE1_DELIVERY.md8.0 KB
- PHASE1_SUMMARY.md9.3 KB
- PHASE2_CLI_GUIDE.md8.9 KB
- PHASE2_DELIVERY.md6.3 KB
- postprocess.mjs1.8 KB
- QUICK_REFERENCE.md4.1 KB
- QUICKSTART.md5.2 KB
- README_DISCORD_PHASE1.md7.4 KB
- README.md10.4 KB
- SKILL.md11.8 KB
- test-discord-cli.mjs5.9 KB
- test-discord-integration.mjs9.1 KB
- test-discord.mjs9.6 KB
- test-idempotency.mjs2.7 KB
- test.mjs2.0 KB
Overview
This skill provides multi-source memory ingestion for OpenClaw agents, converting files, URLs, JSON exports, and raw text into structured memory files with automatic YAML frontmatter. It includes SHA1 deduplication, entity extraction, and integration with the ClawText RAG layer so imported memories become available for retrieval-augmented prompts.
How this skill works
The skill reads data from globs, JSON arrays, URLs, or raw strings, generates a YAML header (date, project, type, entities, keywords) and writes normalized memory files into the configured memory directory. Each item is hashed (SHA1) to prevent duplicates; hashes are persisted to a hash store. After ingestion, ClawText clusters can be rebuilt so new memories are indexed for RAG injection.
When to use it
- Import documentation or knowledge bases into agent memory
- Archive chat exports (Discord/Slack) or API responses for later retrieval
- Automate daily or hourly ingestion jobs without creating duplicates
- Bulk-process mixed sources (files, JSON, raw text) into a single memory store
- Prepare data for RAG-enabled agents that rely on indexed memories
Best practices
- Provide consistent metadata (project, type) to group related memories and improve indexing
- Use keyMap when ingesting JSON exports to map message, date, and author fields correctly
- Leave deduplication enabled for recurring ingest runs; disable only when you control source uniqueness
- Run cluster rebuild (ClawText) after large imports to refresh RAG indexes
- Use transform hooks to normalize or redact sensitive fields before writing memories
Example use cases
- Ingest a docs folder: glob markdown files into the documentation project for searchable facts
- Import a Discord chat export: map content, timestamp, and author to create team decision memories
- Daily sync: run a scheduled job to ingest recent notes; dedupe prevents duplicates
- Batch pipeline: combine files, JSON exports, and ad-hoc text into one ingestAll call and commit hashes
- Data transformation: apply an async transform to clean or enrich content before storing
FAQ
Yes. Each item is hashed with SHA1 and checked against the persisted hash store so repeated ingestion skips previously stored memories.
Is clustering automatic after ingest?
The skill writes memories and updates hashes; ClawText handles cluster indexing. Call the cluster build script (or let ClawText rebuild on load) to index new memories.
Can I customize field mappings for JSON exports?
Yes. Use keyMap to map contentKey, dateKey, and authorKey, and supply an optional transform function to modify content before it is saved.
