Source-first ingestion for the zettelkasten agent (ingest_source + content-hash + full-text cache)¶

Angelo › zettelkasten › Phase 2: Literature Review System

Type: Decision · Status: active · Created: 2026-06-18

Tags: zettelkasten,ingestion,zotero,content-hash,fulltext-cache,dedup

Made every source enter the zettelkasten through one canonical front door so we always have (1) an anchored citation record and (2) a reproducible full-text reference. Zotero is preferred; a local file is the fallback. Implemented in three steps:

Step 1 — canonical workflow (docs/persona). Rewrote the zettelkasten-extractor persona in .cursor/agents.yaml so source acquisition via ingest_source is step 1, removed the stale "PDFs stored only in Zotero are not reachable" line (obsolete since zotero_fetch_pdf), and updated the coverage step. Documented the source-first workflow + new tools and added content_hash/source_path to source_meta_fields in _schema.yaml.

Step 2 — local-dir fallback + content hashing. New zettelkasten/ingest.py: a unified resolver ingest(key|path|title) that returns extracted text, annotations, a stable content_hash (sha256 of the file bytes), and provenance. Reuses zotero._extract_pdf_text for PDFs, reads text files directly, resolves bare filenames against ZK_SOURCES_DIR (default sources/, gitignored). New ingest_source MCP tool wraps it and flags existing_source. Extended create_source with content_hash/source_path and dedup-by-hash (re-ingesting the same file reuses the existing source).

Step 3 — full-text cache + idempotency + auto-coverage. Full text cached at .angelo/zettel/fulltext/<content_hash>.txt (gitignored, re-derivable from the committed pointer). New get_fulltext(source) tool returns cached text or re-derives from source_path/zotero_key when cold. New set_coverage tool; coverage.agent auto-advances none -> skimmed on the first add_note.

Cache-location decision: gitignored content-addressed store + committed pointer in _meta.yaml (keeps git lean; full text reproducible offline) rather than committing full text (bloat/copyright). Verified end-to-end in an isolated zettelkasten dir: ingest -> create_source dedup -> coverage bump -> get_fulltext (cached then re-derived) -> set_coverage. Note: the angelo-zettelkasten MCP server must be restarted in Cursor for the three new tools to appear.

Referenced files¶

Source: memory entry deci-d9ccd806.