memory.drift_symbols

memory.drift_symbols

Symbol-granularity refinement for code-tethered memory drift.

angelo pins every entry to the git SHAs of the files it references (path@sha). The baseline drift check (:func:memory.server._check_file_drift) compares the whole file against HEAD, so a one-line edit anywhere in a large module flips the stale flag even when the code the memory actually describes is untouched. On the real self-hosted corpus that over-counts semantic staleness by ~12.5% (see the Leg-3 corpus-aging analysis).

This module recovers granularity purely from strings — no subprocess git, no repo handle — so it is fast enough for the ~20-hit recall/search hot path and trivially unit-testable. Given the file's source at the pinned SHA, its source at HEAD, and the entry body, it decides whether a symbol the entry names actually changed:

1. parse the def/class symbols defined in the file at the pinned SHA;
2. keep only those the entry body mentions by name;
3. diff old-vs-new with :mod:difflib and collect the old-side line numbers that were deleted or modified (pure insertions do not invalidate an existing symbol's description);
4. a named symbol is drifted iff a changed old line falls inside its span.

The verdict is deliberately conservative: it only ever downgrades a file-level stale flag to fresh when it is confident (the file changed but no named symbol was touched). When it cannot judge — a non-Python file, unreadable source, or a body that names no symbol defined in the file — it reports assessable=False and the caller keeps the coarse file-level flag, so no true staleness is hidden.

Symbol dataclass

A def/class definition span in a source file (1-indexed, inclusive).

Attributes:

Name	Type	Description
name	str	The defined name (function or class).
start	int	1-indexed line of the def/class header.
end	int	1-indexed last line of the definition body (inclusive).

Source code in memory/drift_symbols.py

@dataclass
class Symbol:
    """A ``def``/``class`` definition span in a source file (1-indexed, inclusive).

    Attributes:
        name: The defined name (function or class).
        start: 1-indexed line of the ``def``/``class`` header.
        end: 1-indexed last line of the definition body (inclusive).
    """

    name: str
    start: int
    end: int

Refinement dataclass

Symbol-level drift verdict for one entry against one file.

Attributes:

Name	Type	Description
assessable	bool	Whether a symbol-level judgment was possible at all — i.e. the entry body names at least one symbol defined in the pinned file. When False the caller must fall back to the file-level flag.
stale	bool	Whether a named symbol actually changed. Only meaningful when assessable is True.
drifted_symbols	list[str]	Names of the entry-referenced symbols whose definition span overlaps a changed line.

Source code in memory/drift_symbols.py

@dataclass
class Refinement:
    """Symbol-level drift verdict for one entry against one file.

    Attributes:
        assessable: Whether a symbol-level judgment was possible at all — i.e.
            the entry body names at least one symbol defined in the pinned file.
            When ``False`` the caller must fall back to the file-level flag.
        stale: Whether a *named* symbol actually changed. Only meaningful when
            ``assessable`` is ``True``.
        drifted_symbols: Names of the entry-referenced symbols whose definition
            span overlaps a changed line.
    """

    assessable: bool
    stale: bool
    drifted_symbols: list[str] = field(default_factory=list)

python_symbols

python_symbols(source: str) -> list[Symbol]

Approximate the def/class symbol spans in Python source.

A symbol spans from its header line until the next line whose indentation is less than or equal to the symbol's own — the standard block-by-indent heuristic. Nested defs/classes are captured as their own symbols. Spans are 1-indexed and end-inclusive.

This is a lexical approximation (no full parse), which is intentional: it must tolerate the file at an old SHA that may not even parse under the current Python, and it must never raise on the hot path.

Parameters:

Name	Type	Description	Default
source	str	The full text of a Python source file.	required

Returns:

Type	Description
list[Symbol]	The symbols defined at top level or nested, in definition order.

Source code in memory/drift_symbols.py

def python_symbols(source: str) -> list[Symbol]:
    """Approximate the ``def``/``class`` symbol spans in Python ``source``.

    A symbol spans from its header line until the next line whose indentation is
    less than or equal to the symbol's own — the standard block-by-indent
    heuristic. Nested defs/classes are captured as their own symbols. Spans are
    1-indexed and end-inclusive.

    This is a lexical approximation (no full parse), which is intentional: it must
    tolerate the file at an *old* SHA that may not even parse under the current
    Python, and it must never raise on the hot path.

    Args:
        source: The full text of a Python source file.

    Returns:
        The symbols defined at top level or nested, in definition order.
    """
    lines = source.splitlines()
    heads: list[tuple[str, int, int]] = []  # (name, indent, start_line)
    for i, line in enumerate(lines, start=1):
        m = _PY_DEF_RE.match(line)
        if m:
            heads.append((m.group("name"), len(m.group("indent")), i))

    symbols: list[Symbol] = []
    for idx, (name, indent, start) in enumerate(heads):
        end = len(lines)
        for _, next_indent, next_start in heads[idx + 1 :]:
            if next_indent <= indent:
                end = next_start - 1
                break
        symbols.append(Symbol(name, start, end))
    return symbols

changed_old_lines

changed_old_lines(old_source: str, new_source: str) -> set[int]

Old-side (1-indexed) line numbers deleted or modified between two versions.

Uses :class:difflib.SequenceMatcher line opcodes: replace and delete mark old lines as changed; insert contributes none (adding code between two functions does not invalidate an existing symbol's description); equal is skipped. This mirrors the "- lines of a unified diff" semantics used by the offline corpus-aging analysis, but without shelling out to git.

Parameters:

Name	Type	Description	Default
old_source	str	File contents at the pinned SHA.	required
new_source	str	File contents at HEAD.	required

Returns:

Type	Description
set[int]	The set of changed old-side line numbers (1-indexed).

Source code in memory/drift_symbols.py

def changed_old_lines(old_source: str, new_source: str) -> set[int]:
    """Old-side (1-indexed) line numbers deleted or modified between two versions.

    Uses :class:`difflib.SequenceMatcher` line opcodes: ``replace`` and
    ``delete`` mark old lines as changed; ``insert`` contributes none (adding
    code between two functions does not invalidate an existing symbol's
    description); ``equal`` is skipped. This mirrors the "``-`` lines of a unified
    diff" semantics used by the offline corpus-aging analysis, but without
    shelling out to ``git``.

    Args:
        old_source: File contents at the pinned SHA.
        new_source: File contents at HEAD.

    Returns:
        The set of changed old-side line numbers (1-indexed).
    """
    old_lines = old_source.splitlines()
    new_lines = new_source.splitlines()
    sm = difflib.SequenceMatcher(a=old_lines, b=new_lines, autojunk=False)
    changed: set[int] = set()
    for tag, i1, i2, _j1, _j2 in sm.get_opcodes():
        if tag in ("replace", "delete"):
            # i1..i2 are 0-indexed half-open old-side ranges; +1 → 1-indexed.
            changed.update(range(i1 + 1, i2 + 1))
    return changed

mentions

mentions(body: str, name: str) -> bool

True if name appears as a whole word in body (case-sensitive).

Source code in memory/drift_symbols.py

def mentions(body: str, name: str) -> bool:
    """True if ``name`` appears as a whole word in ``body`` (case-sensitive)."""
    return re.search(rf"\b{re.escape(name)}\b", body) is not None

refine

refine(old_source: str, new_source: str, body: str) -> Refinement

Judge whether a symbol the entry names actually changed between versions.

Parameters:

Name	Type	Description	Default
old_source	str	The pinned file's contents at the pinned SHA.	required
new_source	str	The same file's contents at HEAD.	required
body	str	The memory entry's free-text body (used to find named symbols).	required

Returns:

Name	Type	Description
A	Refinement	class:Refinement. assessable is False (and the caller should
	Refinement	keep the file-level flag) when the body names no symbol defined in the
	Refinement	pinned file; otherwise stale reflects whether any named symbol's span
	Refinement	overlaps a changed line, with the offending names in drifted_symbols.

Source code in memory/drift_symbols.py

def refine(old_source: str, new_source: str, body: str) -> Refinement:
    """Judge whether a symbol the entry names actually changed between versions.

    Args:
        old_source: The pinned file's contents at the pinned SHA.
        new_source: The same file's contents at HEAD.
        body: The memory entry's free-text body (used to find named symbols).

    Returns:
        A :class:`Refinement`. ``assessable`` is ``False`` (and the caller should
        keep the file-level flag) when the body names no symbol defined in the
        pinned file; otherwise ``stale`` reflects whether any named symbol's span
        overlaps a changed line, with the offending names in ``drifted_symbols``.
    """
    named = [s for s in python_symbols(old_source) if mentions(body, s.name)]
    if not named:
        return Refinement(assessable=False, stale=False)
    changed = changed_old_lines(old_source, new_source)
    drifted = sorted(
        {s.name for s in named if any(s.start <= ln <= s.end for ln in changed)}
    )
    return Refinement(assessable=True, stale=bool(drifted), drifted_symbols=drifted)

is_python_path

is_python_path(path: str) -> bool

True if path names a Python source file (symbol parsing is Python-only).

Source code in memory/drift_symbols.py

def is_python_path(path: str) -> bool:
    """True if ``path`` names a Python source file (symbol parsing is Python-only)."""
    return path.replace("\\", "/").endswith(".py")