memory.sharing.bundle¶
memory.sharing.bundle ¶
On-disk bundle format: filter -> encrypt (publish) and decrypt -> materialize.
This module defines the contract between a producer (who publishes a filtered, per-recipient-encrypted bundle) and a consumer (who git-fetches the bundle and decrypts only what its key opens). The downstream publish/sync CLI is built on top of these two functions; it never needs to read this module's internals.
On-disk bundle layout¶
A bundle is a plain directory (typically the working tree of a producer-owned "bundle repo") laid out as::
<bundle_root>/
manifest.json # cleartext structural index (see below)
project.age # the project.md node, age-encrypted
units/<digest>.age # one age ciphertext per shared entry
Each units/*.age file is a self-contained age ciphertext of one entry's
rendered markdown (the exact bytes memory/storage.py wrote, frontmatter
included), encrypted natively to every recipient allowed to see that entry. A
recipient decrypts a unit iff their private key is among its recipients.
Manifest — what is in the clear vs encrypted¶
The manifest is deliberately MINIMAL and privacy-first. In the clear it carries
only: the format id/version, a creation timestamp, an optional producer id (for
attribution / peer registration), and a list of {kind, path} unit pointers.
It intentionally OMITS entry ids, titles, parent links, and recipient identities. Everything about an entry — including its id — lives INSIDE the encrypted unit (in the frontmatter) and is recovered only after a successful decrypt. Consumers therefore learn a unit's id only if they can open it.
Metadata-leak tradeoff (accepted, per the encryption-first design decision):
a ciphertext store still unavoidably leaks structural metadata — the number of
units, their kind (entry vs project), and each unit's ciphertext size. It does
NOT leak entry ids, contents, or who-can-see-what (recipient sets are encoded
only inside the age header stanzas, not in the manifest). Unit filenames are
opaque SHA-256 digests of the entry id: stable across republishes (nice git
deltas) and meaningless to anyone who doesn't already know the id. Fully hiding
unit count/sizes would require padding or an all-in-one encrypted archive; that
is deferred.
Revocation-in-history (accepted limitation)¶
Revocation is NOT retroactive. A bundle is typically the working tree of a git
"bundle repo", so every previously published ciphertext persists in that repo's
git history, and the per-unit age recipient keys are never rotated. Once a
recipient has fetched (or could re-fetch) an old commit, narrowing a scope or
dropping a grant only affects future bundles — it cannot claw back bytes the
recipient already holds or can read from history. In short: you can un-share the
future, not the past. Truly retroactive revocation would require key rotation
plus re-encryption of the whole history, which is out of scope for this design.
Scrub scope (what is redacted vs what still leaks)¶
The per-unit scrubs redact references to entries a recipient cannot open:
- ENTRIES — :func:
_scrub_cross_linksredacts ungranted entry ids in the STRUCTURED frontmatter link fields (related_to/invalidates/invalidated_by/parent_id/session_id). - DOCUMENTS — :func:
_scrub_document_linksdropslinks_totargets the recipient cannot open. - SESSIONS — :func:
_scrub_session_refsscrubs BOTH the structured ref arrays (entries_created/accessed/modified) AND the auto-derived FREE-TEXT (thesearch_textbody plus atitle/summarybuilt from entry titles), so a session shared to R never names — by id OR title, in any AUTO-DERIVED field — an entry R cannot open. Note this covers only auto-derived fields: user-authored prose (a session'snotes, or ahandofftitle/summarythe user typed) is NOT auto-derived and ships verbatim, the same accepted body-leak class as an entry body (see residual leaks below).
Accepted residual leaks (out of scope for this slice):
- Free-text prose that is NOT auto-derived from entry titles still ships verbatim
— an ungranted id/title in an entry's BODY, in
files/node_label, in a session'snotes, or in a document'sdescription/path(the latter two ride a narrower intersection audience, the same class as an entry body leak). - The cleartext manifest still reveals STRUCTURAL metadata (unit count, kind, and each unit's ciphertext size) but no ids/contents/recipients; and revocation is forward-only (see below).
- The
project.mdstub (id+nameonly, empty body) is INTENTIONALLY encrypted to every grantee — including skill-only recipients — so their cache can render the peer.
Decrypted-cache layout¶
:func:decrypt_bundle writes a .memory-shaped directory so the existing
:func:memory.federation.read_repo can read it UNCHANGED::
<output_memory_dir>/
project.md
entries/<id>.md
The caller passes the .memory directory of a federation cache (e.g.
.angelo/federation-cache/<peer_id>/.memory); registering that cache root as a
federated repo then renders it like any other peer.
BundleUnit
dataclass
¶
A pointer to one encrypted unit inside a bundle.
kind is one of "entry" / "document" / "skill" / "session"
(opaque units/<sha256>.age files) or "project" (the project.age
stub); path is the unit's location relative to the bundle root. No id or
recipient info is stored here — that is the deliberate privacy-first manifest
design (see module docstring).
Source code in memory/sharing/bundle.py
BundleManifest
dataclass
¶
The cleartext manifest.json at the bundle root.
Source code in memory/sharing/bundle.py
BuildResult
dataclass
¶
Summary of a :func:build_bundle run (returned in memory, never written).
Source code in memory/sharing/bundle.py
DecryptResult
dataclass
¶
Summary of a :func:decrypt_bundle run.
Source code in memory/sharing/bundle.py
read_manifest ¶
read_manifest(bundle_dir: Path | str) -> BundleManifest
Read and parse a bundle's cleartext manifest.json.
Source code in memory/sharing/bundle.py
build_bundle ¶
build_bundle(output_dir: Path | str, *, registry: IdentityRegistry, source_memory_dir: Path | str | None = None, sharing: SharingConfig | None = None, producer_id: str | None = None, include_discarded: bool = False) -> BuildResult
Filter memory entries by grants, encrypt per recipient, and write a bundle.
Pipeline (filtering happens BEFORE any encryption — ungranted/private entries are physically omitted, never encrypted-and-dropped):
- Read entries + project from
source_memory_dir(default: the local.memorystore). - Resolve each entry's recipient id set from
sharing(default: parsed from the sourceconfig.yaml), expanding afirmscope to theregistrymembers. Entries resolving to no recipients are dropped. - Encrypt each retained entry's on-disk markdown to its recipients' public
keys and write
units/<digest>.age. Before encryption, cross-link frontmatter fields (related_to/invalidates/invalidated_by/parent_id) are scrubbed PER UNIT: a reference to id X survives only if X is itself bundled AND every recipient of this unit is also a recipient of X (plus the shared project root is always allowed). This keeps a recipient from ever learning the id/relationship of an entry it was not also granted, and is a byte-level no-op for a granted entry that has no dangling cross-links. - Encrypt a MINIMAL
project.mdSTUB to the UNION of all recipients and writeproject.age. The stub keeps only structural frontmatter (id/name/status/created_at) with an EMPTY body — enough forread_repo/configured_reposto render the peer, but WITHOUT the project root description/body a bilateral recipient was never granted. - Write the cleartext
manifest.json.
This function OWNS output_dir/units, output_dir/project.age and
output_dir/manifest.json — the units directory is rebuilt from scratch on
every call so stale/revoked units never linger. Other files in output_dir
(e.g. a .git directory) are left untouched.
Returns a :class:BuildResult. With no sharing policy configured this writes
an empty bundle (no units, no project) — the additive "nothing shared"
default.
Source code in memory/sharing/bundle.py
1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 | |
decrypt_bundle ¶
decrypt_bundle(bundle_dir: Path | str, output_memory_dir: Path | str, *, identity: LocalIdentity | Any | None = None, private_key: str | None = None) -> DecryptResult
Decrypt every unit this key can open and materialize a .memory tree.
For each manifest unit the private key is tried; units that fail to decrypt
(i.e. were not encrypted to this key) are silently skipped. Successfully
decrypted units are written into a .memory-shaped output_memory_dir
(project.md plus entries/<id>.md, documents/<id>.md,
sessions/<id>.md and skills/<name>.md) so that
:func:memory.federation.read_repo can read the result unchanged.
Each unit's id/name is read from the decrypted frontmatter (it is never
stored in the clear) and validated / path-jailed before being used as a
filename, so a hostile bundle cannot escape output_memory_dir.
Each cleartext manifest.json unit path is validated BEFORE it is read
(rejecting absolute paths, .. traversal, and anything but the opaque
units/<sha256>.age / project.age layout — enforced regardless of the
manifest's self-declared format) so a malicious publisher cannot use this
function to read arbitrary local files. A missing or corrupt manifest is
treated as an empty bundle (the cache is still reconciled), never a crash.
The output .memory tree is RECONCILED to the current bundle: stale files
(units that dropped out of a newer, re-scoped bundle or are no longer openable
by this key) are removed from EACH owned dir (entries/, documents/,
skills/, sessions/), and project.md is refreshed or removed to
match. Only the .md files in those four dirs and project.md this
function owns are touched — unrelated files (e.g. .git) are left alone. A
transient I/O fault during the pass SKIPS all eviction that run (never
destroys valid cached plaintext); revocation still works on any clean pass.
Provide the key via identity (a :class:LocalIdentity or pyrage
identity), private_key (an AGE-SECRET-KEY-... string), or neither (to
load ~/.angelo/identity.yaml).
Source code in memory/sharing/bundle.py
1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 | |
open_bundle ¶
open_bundle(bundle_dir: Path | str, output_memory_dir: Path | str, *, identity: LocalIdentity | Any | None = None, private_key: str | None = None) -> DecryptResult
Alias for :func:decrypt_bundle (decrypt + materialize a .memory tree).