Tyler Gillispie
34a1f073eb
- Created `CONCEPTS.md` to track coverage of blueprint structures - Added 5 new PoCs: CFG, Constitution, Frontmatter, Mutation, and Orphan Branch - Registered all 15 experiments in `lab.ts` runner - Ensured zero-dependency Deno execution for tests Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: mrteye <1945243+mrteye@users.noreply.github.com>
227 lines
14 KiB
Markdown
227 lines
14 KiB
Markdown
# **Agent Forum**
|
|
|
|
## **Git-Native Agent Collaboration Ecosystem**
|
|
|
|
This galactic report defines the architectural blueprint for a Git-native,
|
|
hyper-efficient AI agent ecosystem. By constraining all state, memory, and
|
|
tooling to the local repository, cloud SaaS dependencies are replaced with
|
|
embedded data structures (Merkle DAGs, SCIP indexes, local vector graphs, YAML
|
|
task DAGs). This creates a zero-latency, cryptographically immutable pipeline
|
|
where AI agents interact with structural code physics and semantic ontologies
|
|
rather than raw text.
|
|
|
|
## **1\. The Git-Native Forcing Function & Embedded Storage**
|
|
|
|
Constraining the state and tooling entirely within the repository format acts as
|
|
a brilliant forcing function. It shifts the architecture from a "Cloud-Native"
|
|
distributed system to a "Local-First / Git-Native" operating system. Removing
|
|
third-party databases preserves project isolation and provides cryptographic
|
|
immutability with zero-latency access.
|
|
|
|
- **Memory Storage (Git Notes & Orphan Branches):**
|
|
- **Git Notes (refs/notes/commits):** Arbitrary metadata—such as JSON
|
|
transcripts of an AI agent's decision-making process—is attached directly to
|
|
a commit without altering the commit hash. The Historian agent can read git
|
|
log \--show-notes="ai" to understand why a specific line of code was
|
|
written, keeping the working directory clean.
|
|
- **The Meta-State Orphan Branch:** Ongoing project state, such as CI/CD
|
|
telemetry and Requirements Traceability Matrices (RTM), is tracked in a
|
|
parallel orphan branch. Agents commit dynamic state JSONs here, isolated
|
|
within the same .git folder but completely separate from the main source
|
|
code.
|
|
- **Serialization (JSON vs. Protocol Buffers):** While JSON is utilized for
|
|
human-readable state tracking, engineering teams should evaluate Protocol
|
|
Buffers (Protobuf) for high-performance, conversion-less data transfer
|
|
between agents. Protobuf integrates natively with SCIP indexes and works in
|
|
tandem with TurboQuant (which compresses the vector math), drastically
|
|
reducing I/O latency.\
|
|
**Impact:** Eliminates reliance on external databases while maintaining
|
|
perfect, version-controlled state isolation.
|
|
- **Fuzzy Retrieval via Embedded Vector Search:**
|
|
- **sqlite-vec & TurboQuant:** Traditional databases require exact keyword
|
|
matches, but Locality-Sensitive Hashing (LSH) and Hierarchical Navigable
|
|
Small World (HNSW) algorithms compress high-dimensional concepts into binary
|
|
hashes. Using the sqlite-vec extension with 2-bit to 4-bit "TurboQuant"
|
|
quantization allows massive semantic knowledge (PRDs, ADRs) to be compressed
|
|
into a tiny local file (often under 30MB). Agents can query these
|
|
associative memories in milliseconds without network calls.
|
|
- **Multi-Vec Isolation:** Rather than dumping all embeddings into a single
|
|
vector database, the meta-state branch should consider isolated sqlite-vec
|
|
files (e.g., docs\_graph.sqlite and telemetry\_graph.sqlite). This
|
|
"Multi-Vec" architecture prevents semantic bleed, ensuring a query about
|
|
code performance does not cross-contaminate with team communication logs.\
|
|
**Impact:** Reduces context window bloat and eliminates cloud database
|
|
latency.
|
|
- **Protocols & Governance:**
|
|
- **Declarative Frontmatter:** Every Markdown artifact requires YAML
|
|
frontmatter containing a unique UUID (Artifact-ID).
|
|
- **Bounded Model Checking (BMC):** A local state machine reads a static
|
|
.agents/transitions.json file to dictate the execution pipeline. This
|
|
ensures strict governance (e.g., "The Coder agent cannot run until the
|
|
Gatekeeper agent has signed off").
|
|
|
|
## **2\. Structured Code Intelligence**
|
|
|
|
To prevent context window collapse and massive compute costs, agents must not
|
|
ingest raw text. Instead, they require a highly efficient I/O pipeline built on
|
|
structured code intelligence.
|
|
|
|
- **Git Merkle DAG Diffing:** Because Git is fundamentally a Merkle Tree, the
|
|
system uses zero-overhead diffing (git ls-tree and git diff-tree) to instantly
|
|
identify changed file hashes. The AI's knowledge base updates in milliseconds
|
|
by walking down the tree to the exact modified file.\
|
|
**Impact:** Guarantees O(1) context updates by passing only cryptographic
|
|
diffs rather than full file strings.
|
|
- **From Syntax to Code Property Graphs (CPGs):**
|
|
- **Tree-sitter & SCIP Indexes:** Instead of regex, Tree-sitter incrementally
|
|
parses code into a structured Abstract Syntax Tree (AST). A pre-commit hook
|
|
then generates a SCIP (Semantic Code Intelligence Protocol) index—a
|
|
lightweight database of code symbols providing statically guaranteed "Find
|
|
References" and "Go to Definition" capabilities.
|
|
- **Control Flow Graphs (CFGs):** Extracted from the AST, CFGs map every
|
|
possible path a variable can take. The Adversary agent can feed this JSON
|
|
dataset into its prompt to deterministically prove if unsanitized user input
|
|
can ever reach a database query.\
|
|
**Impact:** Transforms ambiguous text processing into deterministic,
|
|
mathematically verifiable graph traversals.
|
|
- **Human-Grade Quality Tools:** Agents ingest the JSON/XML outputs of
|
|
industry-standard tools:
|
|
- **Static Analysis (Semgrep / SonarQube):** Feeds vulnerabilities and code
|
|
smells directly to triage agents.
|
|
- **Mutation Testing (Stryker / Mutmut):** Injects bugs to test the tests.
|
|
Feeding mutation scores to the Adversary agent forces the generation of
|
|
edge-case coverage rather than superficial line-coverage.
|
|
- **Dependency Graphing (CodeSee / Madge):** Generates adjacency matrices to
|
|
calculate the exact "blast radius" of a code change.\
|
|
**Impact:** Roots agent decision-making in industry-standard, compiler-grade
|
|
telemetry rather than LLM guesswork.
|
|
|
|
## **3\. Orchestration Matrix & Governance**
|
|
|
|
The AGENTS.md file serves as the strict, machine-readable constitution. To
|
|
ensure agent autonomy, instructions must rely on this repository documentation
|
|
rather than micromanaging or spoon-feeding step-by-step logic in individual
|
|
system prompts.
|
|
|
|
| Role | Inputs | Outputs | Primary Directive |
|
|
| :------------- | :--------------------------------------- | :--------------------------------------- | :-------------------------------------------------------------------------------- |
|
|
| **Gatekeeper** | Ontologies, YAML DAGs | Verification checklists | Bridge human requirements with technical reality. |
|
|
| **Historian** | sqlite-vec, Git Notes | Contextual injection | Prevent regression and historical repetition. |
|
|
| **Adversary** | SCIP graphs, CFGs, Mutation, OTel Traces | Edge-case tests, mutations, bottlenecks | Expose security flaws, enforce test coverage, and identify execution bottlenecks. |
|
|
| **Translator** | SCIP diffs, existing docs | API references, guides | Maintain code-to-documentation parity. |
|
|
| **Analyst** | Telemetry, PR threads | Workflow optimizations, Protocol updates | Optimize human-to-agent collaboration. |
|
|
| **Evaluator** | transitions.json, DAGs | Pipeline progression | Govern pipeline integrity (R/W access to meta-state). |
|
|
|
|
### **The Adversary's Expanded Scope**
|
|
|
|
Traditionally associated solely with security, this agent wears three distinct
|
|
hats to comprehensively stress-test the repository:
|
|
|
|
1. **The Security Auditor:** Feeds on Control Flow Graphs (CFGs) to
|
|
deterministically prove if unsanitized user input reaches database queries.
|
|
2. **The Quality Engineer:** Consumes mutation scores (from Stryker/Mutmut) to
|
|
hunt for edge cases and enforce strict test coverage.
|
|
3. **The Performance Engineer:** Ingests OpenTelemetry .trace.json files from
|
|
Section 4 to identify real-world execution bottlenecks.
|
|
|
|
### **Target Application Stack Boundaries**
|
|
|
|
All agents must adhere to the defined stack: Native Web Components with
|
|
Declarative Shadow DOM, Vue.js, Pinia, Tailwind CSS, Deno (TypeScript), Rust for
|
|
local systems, and PostgreSQL.
|
|
|
|
_Implementation Note:_ The Vue.js/Deno/Rust stack listed above is a strictly
|
|
defined example. The core architectural rule is that the target tech stack must
|
|
be explicitly and exhaustively defined in the AGENTS.md file. By locking in the
|
|
stack, agents are statically prevented from hallucinating unauthorized
|
|
libraries, frameworks, or legacy dependencies into the codebase.
|
|
|
|
## **4\. Semantic Project Management & Telemetry**
|
|
|
|
By mapping the syntactic structure of code to the semantic structure of a
|
|
project, the system establishes concrete datasets that act as the connective
|
|
tissue between code, schedules, and business logic.
|
|
|
|
- **Replacing Jira (The Project DAG):** Project stories are serialized into the
|
|
meta-state branch as strict YAML DAGs (e.g., Task\_44 explicitly declares
|
|
blocked\_by: \[Task\_42, Task\_43\]). On every commit, the Evaluator agent
|
|
reads the DAG to calculate the critical path, unblocking tasks and preventing
|
|
agents from executing code out of order.
|
|
- **Replacing DOORS (The Ontology):** Deep traceability is achieved by embedding
|
|
JSON-LD (Linked Data) blocks at the top of markdown documents (@type:
|
|
"Requirement"). A script compiles these into a single ontology.graph file.
|
|
Agents query this graph mathematically to find all components with
|
|
relationship edges to specific business requirements.
|
|
- **Execution Traces (The Physics):** OpenTelemetry (OTel) traces are generated
|
|
during test runs as .trace.json files, capturing millisecond execution
|
|
latency. The Adversary agent uses this to understand how the code actually
|
|
runs, identifying bottlenecks with precision.
|
|
- **Communication Telemetry:** The Analyst consumes specific metrics—Mean Time
|
|
to Resolution (MTTR), PR Comment-to-Code Ratio, Idle Handoff Duration,
|
|
Artifact Override Frequency, and Thread Friction Markers—serialized as JSON
|
|
payloads in the meta-state branch to map team friction.
|
|
|
|
## **5\. The Execution Pipeline**
|
|
|
|
The entire system operates as a continuous, structured data flywheel. All
|
|
artifacts are embedded into the local database, providing agents with a perfect,
|
|
multi-dimensional understanding of the repository.
|
|
|
|
1. Analyst interprets telemetry to update project protocols.
|
|
2. Gatekeeper reads these new protocols to constrain the next cycle.
|
|
3. The loop resets, returning to code generation with updated guardrails.
|
|
|
|
| | Artifact | Generated Data Structure | Primary Consumer Role |
|
|
| :------ | :---------------------------- | :----------------------- | :--------------------- |
|
|
| **1\.** | **The Code** (Architecture) | SCIP/ASTs | Adversary / Translator |
|
|
| **2\.** | **The Tests** (Physics) | OTel Traces | Adversary |
|
|
| **3\.** | **The Docs** (Business Logic) | JSON-LD Ontologies | Gatekeeper |
|
|
| **4\.** | **The Process** (Schedule) | YAML DAGs | Evaluator |
|
|
| **5\.** | **The Team** (Friction) | JSON Telemetry | Analyst |
|
|
|
|
This pipeline is not a linear checklist; it is a continuous, self-correcting
|
|
feedback loop. As demonstrated above, Step 5 (The Team generates Telemetry)
|
|
feeds directly back into Step 1\. When the Analyst identifies workflow friction,
|
|
it updates project protocols, which directly dictates how the Gatekeeper sets
|
|
constraints for the next cycle of Code generation.
|
|
|
|
## **6\. Filtered Explorations (Architectural Graveyard)**
|
|
|
|
During the design phase, several bleeding-edge tools were evaluated but
|
|
ultimately altered to respect the strict repo-native constraints.
|
|
|
|
- **Doc-to-LoRA (D2L) Hypernetworks:** A Perceiver-based latent mapping system
|
|
designed to internalize external context by generating LoRA weights in a
|
|
single forward pass, eliminating KV-cache overhead.
|
|
- _The Verdict:_ While incredibly fast for inference, committing thousands of
|
|
`.safetensors` adapter weights to Git would inevitably bloat the repository.
|
|
D2L was swapped out in favor of context-caching via `sqlite-vec`.
|
|
- **PASTE (Pattern-Aware Speculative Tool Execution):** A framework that
|
|
predicts tool calls using historical patterns and executes them while the LLM
|
|
is still generating to achieve near-zero latency.
|
|
- _The Verdict:_ Highly valuable for meta-routing, but its implementation
|
|
requires careful tuning to ensure speculative executions do not violate the
|
|
local computing and Bounded Model Checking constraints of the repository
|
|
graph.
|
|
|
|
## **Appendix A: Example Toolchain Catalog**
|
|
|
|
To extract the structured data required by the AI agents, the following external
|
|
and custom utilities constitute some example utilities. Engineering teams should
|
|
expand this catalog as specific disciplinary data sets are required or
|
|
perfected. Custom options are ok if they provide valuable benefit.
|
|
|
|
- **Syntax & Architecture (SCIP/AST Extraction):** Tree-sitter (Local
|
|
WebAssembly binaries for generating Abstract Syntax Trees) and SCIP CLI
|
|
(Generates the Semantic Code Intelligence Protocol graphs).
|
|
- **Security & Static Analysis:** Semgrep / SonarQube (Compiles vulnerabilities
|
|
and code smells into JSON payloads for the Adversary).
|
|
- **Quality & Mutation Testing:** Stryker / Mutmut (Injects bugs during the CI
|
|
cycle to generate edge-case mutation scores).
|
|
- **Physics & Telemetry:** OpenTelemetry / OTel (Extracts millisecond execution
|
|
latency into `.trace.json` files).
|
|
- **Dependency & Blast Radius:** CodeSee / Madge (Generates adjacency matrices
|
|
to map downstream impact of code changes).
|
|
- **Data Storage & Retrieval:** sqlite-vec (Embedded SQLite extensions handling
|
|
local vector indexing and TurboQuant compression).
|