• From “Second Brain” to a Living Memory Environment

    hide-key • 2 hours ago • 0 comments

    This log is a snapshot of how GHOST evolved from a conventional Second Brain idea into a broader experiment in long-term human-AI memory. It is intentionally dense: part design history, part working theory, part implementation guardrail.

    I started this project with a fairly familiar idea:

    Build a Second Brain, attach AI agents to it, and stop starting from zero every time.

    That was the practical motivation.

    I had years of scattered personal data: notes, project files, AI conversations, CAD data, fabrication records, web clippings, images, old documents, and fragments of thoughts spread across different devices and services.

    The obvious plan was simple:

    • collect everything,
    • normalize it,
    • convert what was useful into Markdown,
    • store processed knowledge in GitHub,
    • use AI to search, summarize, connect, and maintain it,
    • then let autonomous agents keep it alive.

    At first, this sounded like a normal Personal Knowledge Management problem.

    It gradually stopped being one.

    What I thought was a storage-and-retrieval problem turned into a much stranger question:

    What does a healthy long-term memory environment actually look like when humans and multiple AIs share it?

    That question changed the project.

    The working name is GHOST.

    v0.0 — The Ordinary Second Brain Assumption

    The first version of the idea was very conventional.

    Raw data would live somewhere scalable. Processed knowledge would live in Markdown. GitHub would provide history and ownership. AI would organize, summarize, classify, and connect the data.

    The expected loop was:

    capture → organize → cross-reference → update

    That is still useful.

    But it carries an assumption that I did not notice at first:

    More organized = better. More connected = better. More current = better. More coherent = better.

    That assumption is common in Second Brain systems, LLM-maintained wikis, graph knowledge systems, and agent memory designs.

    The system becomes more useful as it compiles context.

    That sounds reasonable.

    Until the system begins to over-compile the person using it.

    v0.1 — The Components Already Exist

    The next realization was that most of the individual technical parts are not especially novel.

    We already have:

    • vector search,
    • semantic retrieval,
    • Personal Knowledge Graphs,
    • agent memory,
    • provenance systems,
    • temporal memory,
    • RAG,
    • knowledge graph extraction,
    • proactive recall,
    • multi-agent coordination,
    • autonomous research agents,
    • embeddings,
    • structured metadata,
    • event logs,
    • version history.

    So the interesting question was probably not:

    “What new component should I invent?”

    It became:

    “What happens when familiar components are combined under a different definition of a healthy memory system?”

    That shifted the project away from feature accumulation and toward behavior.

    v0.2–v0.3 — Relationships Matter More Than Modules

    I started thinking less about modules and more about verbs.

    What should the system actually do with memory?

    Not just:

    • remember,
    • retrieve,
    • summarize,
    • link.

    But also:

    • defer,
    • disagree,
    • resurface,
    • preserve difference,
    • avoid premature connection,
    • question its own map.

    This became important because most AI systems have a strong optimization tendency.

    When they encounter two similar notes, they want to merge them.

    When they see contradictions, they want to reconcile them.

    When they see an isolated node, they want to connect it.

    When they see an old interpretation, they want to update it.

    When they see a user profile, they want to refine it.

    These are normally considered helpful behaviors.

    But in a long-term cognitive environment, they may also erase variance.

    v0.4 — Maybe Memory Does Not “Disappear.” Maybe It Sleeps.

    A major conceptual shift came from thinking about human memory.

    Humans do not consciously access everything they know all the time.

    Something may be inaccessible for years, then suddenly return because of:

    • a place,
    • a smell,
    • a phrase,
    • a piece of music,
    • a new...
    Read more »