auth-yes/forum/experiments/ASSESSMENT.md
Tyler Gillispie 4df94c2a19
feat: add agent-forum data structures and experiments (#64)
- Updates `forum/DATA_STRUCTURES.md` with missing concepts: Protocol Buffers, TurboQuant, Git Merkle DAG Diffing, Dependency Graphing, and Declarative Frontmatter (UUIDv7).
- Expands `forum/experiments/lab.ts` with 5 new proofs-of-concept for the new data structures.
- Adds `protobuf_poc.ts`, `merkle_diff_poc.ts`, `vector_db_poc.ts`, `dependency_graph_poc.ts`, and `telemetry_poc.ts`.

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: mrteye <1945243+mrteye@users.noreply.github.com>
2026-08-28 19:15:51 -07:00

105 lines
5.8 KiB
Markdown

# Agent Forum v4 Assessment & Research Report
## 1. Overview & Usefulness
The `agent-forum-v4` blueprint outlines a "Git-Native Agent Collaboration
Ecosystem." By shifting from flat Markdown files to structured data (YAML DAGs,
SCIP indexes, local vector graphs), the protocol addresses a core limitation of
modern AI agents: context window collapse and dependency amnesia.
**Solving the Pain Points:** You noted that agents frequently lose context and
struggle to understand if they are fulfilling requirements without constant
spoon-feeding. The proposed shift to a **Semantic Project Management** system
directly solves this:
- **Dependencies:** Instead of relying on an agent to read a folder and "figure
out" what to do, the system uses strict YAML Directed Acyclic Graphs (DAGs).
An Evaluator script mathematically determines the critical path. An agent is
only ever handed an explicitly unblocked task.
- **Context:** Instead of feeding the agent the entire codebase as raw text, the
system uses Git Merkle DAG diffing and embedded SQLite vector search to feed
the agent precisely the context it needs in milliseconds.
## 2. Compatibility with the Repository
The Auth-Yes repository is characterized by a strict, zero-dependency, hermetic
environment prioritizing Deno, minimal external SaaS dependencies, and
self-contained runtime operations.
- **High Alignment:** The agent-forum's philosophy of "Local-First / Git-Native"
is in perfect harmony with your project's ethos. Eliminating third-party
databases in favor of Git Notes and orphan branches ensures the protocol
remains portable, cryptographically secure, and isolated.
- **Implementation Challenges:** The blueprint relies heavily on advanced
tooling (Tree-sitter, SCIP, `sqlite-vec`). Integrating these into a Deno
environment without polluting the repository with heavy binary dependencies
will require careful execution. We should lean towards WebAssembly (WASM)
ports of these tools (e.g., `tree-sitter.wasm`, or Deno's native FFI for
SQLite) to keep the repository lightweight.
## 3. Isolation Strategy (Preventing Bleed)
To ensure this new protocol does not interfere with the primary intent of
Auth-Yes or other target projects, we must implement strict physical and logical
boundaries:
1. **The `.forum/` (or `.agents/`) Namespace:** All protocol-specific files,
state machines (`transitions.json`), schemas, and tooling scripts must be
entirely contained within a hidden root directory (e.g., `.forum/`). The
target repository should have zero awareness of these files.
2. **Git Meta-State (Orphan Branches):** The most powerful isolation technique
proposed is the use of an Orphan Branch. Dynamic state (telemetry, task
completion status, graphs) will be committed to a branch (e.g.,
`forum/meta-state`) that shares no history with `main`. This ensures the
primary branch's `git log` remains pristine and untouched by agent
automation.
3. **Git Notes for Meta-Thoughts:** By using custom Git Note refs (e.g.,
`refs/notes/forum/reasoning`), agents can attach vast amounts of JSON
metadata, risk assessments, and historical context to a commit without
altering the commit hash or the working directory tree.
## 4. Adapting the Legacy `/tasks` Workflow
You mentioned appreciating the file naming conventions (visual history), risk
assessments, and meta-thoughts of the old system. The goal is to preserve the
_value_ of these features while upgrading their _format_ to be machine-readable.
- **UUIDv7 & Legacy Identifiers:** To align with the strict blueprint,
**UUIDv7** will be the primary and required identifier (Artifact-ID/Key) for
all items in YAML DAGs. The old file naming convention
(`YYYY-MMDD.[sequence]...[short-description]`) will be stored strictly as an
optional `legacy_slug` metadata field inside the YAML structure. This
preserves visual history and backwards compatibility for human readers while
completely detaching it from filenames and the core machine-communication ID
system, avoiding string-parsing errors or pollution of the core concept.
- **Visual History:** To preserve the human visual experience, a simple Deno
script (e.g., `deno task forum:view`) can parse the DAG and the optional
`legacy_slug` metadata from the orphan branch history to output an interactive
terminal UI or a generated HTML report showing the exact progression of work.
- **Risk Assessment & Meta-Thoughts:** In the old system, these were markdown
headers. In the new system, an "Adversary" agent will generate these risk
assessments as structured JSON. We will store this JSON in **Git Notes**
attached to the relevant commits. This ensures the data is tightly coupled to
the code changes, never gets lost in a stale markdown file, and can be queried
instantly by other agents.
- **Human-Readable Projections:** If we ever need a standard Markdown view, a
"Translator" agent or script can compile the DAGs, Git Notes, and ASTs and
generate a static `tasks-report.md` on demand.
## 5. Proposed Experimental Path (Proof of Concept)
We will not build the fully automated loop yet. Instead, we will build
foundational experiments in `forum/experiments/` to verify the hard concepts:
1. **Git Storage PoC (`git_storage_poc.ts`):** Verify that Deno can
programmatically read/write to Git Notes and manipulate an orphan branch
without disrupting the current working tree. This proves we can store agent
state invisibly.
2. **DAG Engine PoC (`dag_engine_poc.ts`):** Create a minimal script that parses
a YAML task graph, resolves dependencies (`blocked_by`), and mathematically
outputs the exact next task an agent should work on.
3. **Local Intelligence PoC (`code_intelligence_poc.ts`):** Experiment with
lightweight semantic parsing (e.g., extracting exports or AST structure from
a file) to prove we can feed agents structured code intelligence rather than
raw strings.