825 lines
49 KiB
Markdown
825 lines
49 KiB
Markdown
# Festinger — Agent0 Inference Middleware
|
||
|
||
**Status:** In progress — iterative specification
|
||
**Owner:** jenstandstad
|
||
**Location:** `plugins/festinger/`
|
||
|
||
> Named after Leon Festinger (1919–1989), social psychologist who introduced the theory of cognitive dissonance in 1957. Festinger observed that minds — human or artificial — cannot comfortably hold contradictory beliefs simultaneously, and that the tension this creates drives resolution. This system is built on the same principle.
|
||
|
||
---
|
||
|
||
## Purpose
|
||
|
||
Festinger is an Ollama-compatible HTTP proxy that sits between Agent0's agent-zero containers and the local Ollama inference endpoint. It solves two related problems that emerge with local inference:
|
||
|
||
1. **Reasoning loops** — agents repeat the same output and cannot break out, even when the framework tells them to try something else.
|
||
2. **Stale and incoherent memory** — Agent Zero's FAISS-based memory accumulates facts without contradiction detection, causing agents to act on outdated or conflicting beliefs.
|
||
|
||
Festinger addresses both at the inference layer, transparently, without modifying agent-zero internals. The memory layer it introduces is called **Recollections** — short, structured, non-contradicting facts injected spontaneously into every prompt as context enrichment. Agents do not search for recollections; they appear automatically.
|
||
|
||
Like its namesake's theory, Festinger treats contradiction not as an error to suppress but as a signal to act on.
|
||
|
||
---
|
||
|
||
## Architecture
|
||
|
||
```
|
||
agent-zero containers
|
||
│
|
||
▼
|
||
┌───────────────────────────────────────────┐
|
||
│ Festinger Proxy │
|
||
│ │
|
||
│ ┌─────────────┐ ┌──────────────────┐ │
|
||
│ │ Loop │ │ Saliency Engine │ │
|
||
│ │ Detector │ │ (tokenise+score) │ │
|
||
│ └─────────────┘ └────────┬─────────┘ │
|
||
│ │ │
|
||
│ ┌─────────▼─────────┐ │
|
||
│ │ SOAS │ │
|
||
│ │ concept vocab │ │
|
||
│ │ + saliency store │ │
|
||
│ └─────────┬─────────┘ │
|
||
│ │ │
|
||
│ ┌───────────────────┤ │
|
||
│ │ │ │
|
||
│ ┌───────▼──────┐ ┌────────▼─────────┐ │
|
||
│ │ Recollection │ │ Memory Writer │ │
|
||
│ │ Engine │ │ (cloud LLM + │ │
|
||
│ │ (read IN → │ │ NL → IN parser) │ │
|
||
│ │ inject │ └────────┬─────────┘ │
|
||
│ │ <recollec- │ │ │
|
||
│ │ tion> block)│ ┌────────▼─────────┐ │
|
||
│ └──────────────┘ │ Conflict │ │
|
||
│ │ │ Resolver │ │
|
||
│ │ └────────┬─────────┘ │
|
||
│ └──────────┬───────┘ │
|
||
│ │ │
|
||
│ ┌▼──────────────────┐ │
|
||
│ │ IN table │ │
|
||
│ │ acyclic concept │ │
|
||
│ │ graph │ │
|
||
│ └───────────────────┘ │
|
||
└───────────────────────────────────────┬──┘
|
||
│
|
||
▼
|
||
Ollama (host)
|
||
│
|
||
▼ (write path only)
|
||
Cloud LLM API
|
||
```
|
||
|
||
---
|
||
|
||
## Data Model
|
||
|
||
### models — LLM provider configuration
|
||
|
||
Festinger uses a cloud LLM for two purposes: the saliency-triggered write path and the nightly resolution job. Each purpose can use a different model.
|
||
|
||
| Column | Type | Notes |
|
||
|--------|------|-------|
|
||
| id | SERIAL PK | |
|
||
| provider | VARCHAR | `claude` or `openai` |
|
||
| model_name | VARCHAR | e.g. `claude-opus-4-6`, `gpt-4o` |
|
||
| api_key | VARCHAR | stored encrypted at rest |
|
||
| created_at | TIMESTAMPTZ | |
|
||
|
||
### config — runtime configuration
|
||
|
||
Key-value store for settings changeable without redeployment.
|
||
|
||
| Key | Default | Purpose |
|
||
|-----|---------|---------|
|
||
| `write_model_id` | — | FK into models; used for saliency-triggered write path |
|
||
| `resolve_model_id` | — | FK into models; used for nightly resolution job |
|
||
| `saliency_read_threshold` | `0.5` | Minimum saliency to trigger recollection lookup |
|
||
| `saliency_write_threshold` | `1.2` | Minimum saliency to trigger cloud LLM write |
|
||
| `recollection_confidence_floor` | `0.6` | Minimum URD edge confidence to include in recollection |
|
||
| `recollection_recency_days` | `90` | URD edges older than this are excluded |
|
||
| `resolution_schedule` | `0 2 * * *` | Cron expression for nightly resolution job |
|
||
|
||
### SOAS — concept vocabulary and saliency
|
||
|
||
One row per token. No minimum token length — all tokens are indexed; frequency and novelty determine whether they surface. Common English words are pre-seeded at saliency 0 via a dictionary corpus. Dimensions are themselves SOAS concepts — no separate table needed.
|
||
|
||
| Column | Type | Notes |
|
||
|--------|------|-------|
|
||
| id | INT PK | auto-increment |
|
||
| token | VARCHAR | unique, lowercase normalised |
|
||
| encounter_count | INT | raw count across all intercepted prompts |
|
||
| last_seen | TIMESTAMP | for recency tracking |
|
||
| saliency | FLOAT | log-scaled encounter saliency; 0 = common English |
|
||
| novelty | FLOAT | domain-specificity score; set to 1.0 when first confirmed by cloud LLM write path, 0 for pre-seeded dictionary words |
|
||
|
||
**Saliency vs novelty** are distinct scores serving different purposes:
|
||
- `saliency` measures how frequently a concept appears — it drives read-threshold triggering
|
||
- `novelty` measures how domain-specific a concept is — a system name that rarely appears but is clearly project-specific should still be treated as important
|
||
|
||
### URD table — the acyclic concept graph
|
||
|
||
Named `urd` (the SQL reserved word `IN` cannot be used as a table name). All three FK columns reference SOAS, so dimensions are concepts in the same vocabulary — new dimensions emerge from the same token space without schema changes.
|
||
|
||
| Column | Type | Notes |
|
||
|--------|------|-------|
|
||
| id | INT FK | references SOAS — the concept being placed |
|
||
| parent_id | INT FK | references SOAS — the containing concept |
|
||
| dim_id | INT FK | references SOAS — which dimension this edge belongs to |
|
||
| is_isa | BOOLEAN | true = ISA (type/classification); false = ISPART (membership/containment). Outside the index — does not affect collision detection but drives conflict resolution semantics and recollection rendering |
|
||
| confidence | FLOAT | reliability of this edge, 0.0–1.0; set by cloud LLM at write time, updated by conflict resolver |
|
||
| last_confirmed | TIMESTAMPTZ | when was this edge last corroborated by an intercepted prompt; used for recency decay in recollection injection |
|
||
| source | VARCHAR | `cloud_llm` (saliency write path), `inferred` (cue pattern), `festinger` (resolution job), `gutask` (gutask iknowthat) |
|
||
|
||
**Index structure:**
|
||
- **PK**: `(id, parent_id, dim_id)` — full triple, prevents duplicate edges
|
||
- **Unique index**: `(id, dim_id)` — one parent per concept per dimension; this is the acyclicity and contradiction-resistance mechanism
|
||
- **Root nodes**: rows where `id = parent_id = dim_id` — the named root of a dimension tree; the one allowed self-reference
|
||
|
||
**Single relation:** IN — "this concept is semantically contained within that concept, within this dimension." ISA and ISPART are not separate relations or tables; they are the same IN relation, with `is_isa` annotating which flavour the edge represents.
|
||
|
||
### In-Memory Cache Layer
|
||
|
||
The proxy maintains three in-memory structures populated at startup from Postgres. All read operations hit these structures only — zero network on the hot path. Writes are write-through: in-memory first, then Postgres async (saliency updates) or sync (URD inserts).
|
||
|
||
```python
|
||
# SOAS — primary lookup by token string (mirrors UNIQUE index on token)
|
||
soas_by_token: dict[str, SoasRow]
|
||
|
||
# SOAS — reverse lookup by id (for pre-joining URD results)
|
||
soas_by_id: dict[int, str]
|
||
|
||
# URD — recollection reads: concept_id → list of edges (tokens pre-joined)
|
||
urd_by_concept: dict[int, list[UrdEdge]]
|
||
|
||
# URD — collision detection: (concept_id, dim_id) → edge
|
||
# Mirrors the Postgres UNIQUE index on (id, dim_id) exactly
|
||
urd_by_concept_dim: dict[tuple[int, int], UrdEdge]
|
||
|
||
# Resolution queue — concepts with pending conflicts (for ? marker in recollections)
|
||
pending_conflicts: set[int]
|
||
```
|
||
|
||
**New SOAS token flow** — ids always originate from Postgres:
|
||
```
|
||
token not in soas_by_token
|
||
→ INSERT into Postgres SOAS → Postgres returns auto-increment id
|
||
→ add to soas_by_token[token] and soas_by_id[id]
|
||
→ proceed with that id
|
||
```
|
||
|
||
**URD insert flow** — collision detected in-memory, Postgres is the safety net:
|
||
```
|
||
key = (concept_id, dim_id)
|
||
if key in urd_by_concept_dim:
|
||
→ collision detected in-memory
|
||
→ classify type (is_isa flags), route to resolution queue
|
||
→ return — no Postgres write attempted
|
||
else:
|
||
→ INSERT into Postgres URD
|
||
→ on success: update urd_by_concept[concept_id] and urd_by_concept_dim[key]
|
||
→ on UniqueViolation (race condition): reload row, route to resolution queue
|
||
```
|
||
|
||
**Saliency update flow** — batched to avoid per-token Postgres writes:
|
||
```
|
||
every token encounter → update soas_by_token[token].encounter_count in-memory
|
||
every 30 seconds → flush encounter count deltas to Postgres in one batch UPDATE
|
||
```
|
||
|
||
**Cache reload after nightly job** — nightly job POSTs to `/reload` endpoint on proxy:
|
||
```
|
||
proxy receives /reload
|
||
→ re-SELECT all URD rows from Postgres with pre-joined tokens
|
||
→ rebuild urd_by_concept and urd_by_concept_dim
|
||
→ rebuild pending_conflicts from resolution queue
|
||
→ SOAS dict unchanged (nightly job does not modify SOAS)
|
||
```
|
||
|
||
This separation means tests can inject mock data directly into the dicts without touching Postgres, enabling full unit testing of collision detection, recollection rendering, and queue routing.
|
||
|
||
---
|
||
|
||
### Canonical Recollection Query
|
||
|
||
The recollection engine executes this query for each salient concept found in an intercepted prompt. No chain traversal — depth is always 1. The result is a flat enumeration of all edges where the concept is the subject, across all dimensions.
|
||
|
||
```sql
|
||
SELECT
|
||
u.id,
|
||
u.parent_id,
|
||
u.dim_id,
|
||
u.is_isa,
|
||
p.token AS parent_token,
|
||
d.token AS dim_token
|
||
FROM urd u
|
||
INNER JOIN soas p ON p.id = u.parent_id
|
||
INNER JOIN soas d ON d.id = u.dim_id
|
||
WHERE u.id = $1
|
||
AND u.confidence >= $2
|
||
AND u.last_confirmed >= $3
|
||
ORDER BY u.id, u.dim_id DESC;
|
||
```
|
||
|
||
`$1` = SOAS id of the concept, `$2` = confidence floor (from config), `$3` = recency cutoff (from config).
|
||
|
||
**Injection position:** the recollection block is prepended to the content of the existing system message if one is present. If no system message exists in the messages array, a new `{"role": "system"}` message containing only the recollection block is inserted at position 0. The system message is the highest-attention position in most instruction-tuned models — it is where grounding facts anchor most reliably.
|
||
|
||
**Rendering:** iterate rows; for each row emit `[dim_token] parent_token`. If the concept also has a pending entry in the resolution queue, append `?` to that dimension token. Group by concept when multiple concepts are queried in a single block.
|
||
|
||
**Zero-hit rendering:** if a concept is above the read threshold but has no URD entries, it is a salient domain-specific term the world model has not yet encountered. Instead of silently omitting it, the recollection block emits an explicit prompt to the agent:
|
||
|
||
```
|
||
? gnommoweb: no recollection. If this is a typo, ignore.
|
||
If you know what it is, store it before proceeding:
|
||
gutask iknowthat 'gnommoweb -isa <parent> in context of <dimension>'
|
||
gutask iknowthat 'gnommoweb -ispart <system> in context of <dimension>'
|
||
```
|
||
|
||
This turns every unknown salient concept into an active instruction. The agent either confirms it is a typo, asks for clarification from a human or peer agent, or fills the gap itself via `gutask iknowthat`. The world model grows organically through use.
|
||
|
||
**Full example recollection block:**
|
||
|
||
```
|
||
<recollection>
|
||
gnommoweb: [glitch_university] repo [geography] ramanujan [type?] service
|
||
dobby: [agent_pool] worker [tech] python
|
||
? ramanujan: no recollection. If you know what it is, store it before proceeding:
|
||
gutask iknowthat 'ramanujan -isa <parent> in context of <dimension>'
|
||
gutask iknowthat 'ramanujan -ispart <system> in context of <dimension>'
|
||
</recollection>
|
||
```
|
||
|
||
`[type?]` signals a pending conflict on that dimension — the world model is not wrong, resolution is in progress.
|
||
|
||
### gutask iknowthat — Manual Write Path
|
||
|
||
`gutask iknowthat` is the highest-confidence write path into URD. It bypasses the saliency threshold and the cloud LLM entirely.
|
||
|
||
**Location:** `/Users/jenstandstad/Projects/gutasktool` (sibling of Agent0 repo). The command POSTs to Festinger's `/iknowthat` HTTP endpoint — gutasktool does not connect to Postgres directly.
|
||
|
||
**Syntax:**
|
||
|
||
```sh
|
||
gutask iknowthat 'gnommoweb -isa repo in context of glitch_university'
|
||
gutask iknowthat 'gnommoweb -ispart glitch_university in context of membership'
|
||
```
|
||
|
||
- `-isa` sets `is_isa=true`; `-ispart` sets `is_isa=false`
|
||
- `in context of <dimension>` specifies the dimension token; defaults to `type` for `-isa`, `membership` for `-ispart`
|
||
- All tokens run through the standard tokeniser (compound token rule applies)
|
||
- Inserts into SOAS if any token is new
|
||
- Inserts into URD with `confidence=1.0`, `source=gutask`
|
||
- On collision: enters resolution queue with priority flag; `gutask`-source conflicts are reviewed first at `/conflicts`
|
||
|
||
**Festinger `/iknowthat` endpoint:** accepts `POST` with JSON body `{fact: string}`. Parses, tokenises, writes to SOAS/URD, returns the inserted or conflicted result. This decouples gutasktool from the Festinger schema.
|
||
|
||
This command is the agent's direct interface to the world model. When the recollection block surfaces a zero-hit concept, `gutask iknowthat` is the prescribed response.
|
||
|
||
---
|
||
|
||
## The IN Relation and ISA/ISPART
|
||
|
||
### Why a single operator
|
||
|
||
ISA and ISPART are both instances of a more general semantic containment relation. The dimension carries the semantic weight:
|
||
|
||
- `gnommoweb IN repo` (dimension: `type`) → gnommoweb ISA repo
|
||
- `gnommoweb IN Glitch-University` (dimension: `membership`) → gnommoweb ISPART Glitch-University
|
||
- `State IN Country` (dimension: `type`) → State ISA a country-level granularity
|
||
|
||
The `type` dimension IS the ISA relation. Every other dimension IS an ISPART relation scoped to that domain. One table, one index, one operator.
|
||
|
||
### Why this resolves the bleed
|
||
|
||
The classic bleed case: Michigan ISA State, Michigan ISPART USA, State ISPART Country.
|
||
|
||
With the single IN operator and dimensions:
|
||
|
||
```
|
||
Michigan IN State (dimension: type) is_isa: true
|
||
Michigan IN USA (dimension: geography) is_isa: false
|
||
State IN Country (dimension: type) is_isa: true
|
||
USA IN Country (dimension: type) is_isa: true
|
||
```
|
||
|
||
Two coherent chains, no collision, no ambiguity. "State ISPART Country" (class-level generalisation) becomes "State IN Country in the type dimension" — a perfectly valid ISA statement: State is a kind of country-level subdivision.
|
||
|
||
---
|
||
|
||
## Collision Semantics — The `is_isa` Flag
|
||
|
||
The unique index on `(concept_id, dimension_id)` fires when a concept already has a parent in a given dimension. The `is_isa` flag on both the existing and incoming rows determines what the collision means:
|
||
|
||
| Existing | Incoming | Interpretation | Action |
|
||
|----------|----------|---------------|--------|
|
||
| ISA | ISA | Dimension too coarse — both facts simultaneously true about the concept's nature | Trigger **dimension decomposition** |
|
||
| ISPART | ISPART | Factual contradiction — thing can only be in one place per dimension | Trigger **arbitration** (which is correct?) |
|
||
| ISA | ISPART | Dimension misclassification — these should have been in different dimensions | Flag as misclassification, suggest correct dimension |
|
||
|
||
---
|
||
|
||
## Dimension Decomposition — Dimensions as Evolving Vocabulary
|
||
|
||
When an ISA+ISA collision occurs, the dimension is too coarse to hold both simultaneously-true facts. Example:
|
||
|
||
- Existing: `gnommoweb IN container` (dimension: `type`, is_isa: true)
|
||
- Incoming: `gnommoweb IN repo` (dimension: `type`, is_isa: true)
|
||
|
||
Both are true. The `type` dimension cannot hold them both. The conflict resolver sends this prompt to the cloud LLM:
|
||
|
||
> "gnommoweb is already `container` in dimension `type`. New fact also places gnommoweb as `repo` in `type`. If both are simultaneously true, propose two more specific dimension names to replace `type` — one where gnommoweb as `container` remains valid, one where gnommoweb as `repo` is valid. Return JSON: `{"existing_dimension": "...", "new_dimension": "..."}`. Choose from this taxonomy where possible: [...]. Create new names only if nothing fits."
|
||
|
||
The LLM might return: `{"existing_dimension": "deployment-type", "new_dimension": "artifact-type"}`.
|
||
|
||
The system then:
|
||
1. Inserts `deployment-type` and `artifact-type` into SOAS (if not present)
|
||
2. Creates root nodes for each new dimension
|
||
3. Inserts the new fact under `artifact-type`
|
||
4. Leaves the existing `type` facts untouched — no migration
|
||
|
||
Dimensions are SOAS concepts. New dimensions emerge from the same token vocabulary. The graph grows its own taxonomy under pressure from real contradictions, starting coarse and decomposing on demand.
|
||
|
||
---
|
||
|
||
## The Two Memory Operations
|
||
|
||
### Tokenisation Rules
|
||
|
||
All text — prompts, system messages, agent outputs — passes through the same tokeniser before any saliency or relationship work is done.
|
||
|
||
**Token extraction:**
|
||
1. Split on whitespace and punctuation boundaries
|
||
2. **Compound token rule**: scan for runs of consecutive tokens where each begins with a capital letter. Merge the run into a single token, joined with underscores, then lowercase. This canonicalises proper nouns and multi-word concepts into single SOAS entries.
|
||
- `Glitch University` → `glitch_university`
|
||
- `Agent Zero` → `agent_zero`
|
||
- `New York City` → `new_york_city`
|
||
- A lowercase or short token breaks the run: `the Glitch University` → `the` breaks the run, `Glitch University` → `glitch_university`
|
||
3. Lowercase all tokens
|
||
4. Keep tokens with **5 or more characters** (strictly >4); discard shorter tokens unless they are part of a matched relationship cue pattern (see below)
|
||
5. Strip leading/trailing punctuation from each token
|
||
|
||
**Example:**
|
||
`"gnommoweb is a repo of Glitch University"`
|
||
→ tokens: `gnommoweb`, `repo`, `glitch_university`
|
||
→ relationship extracted: `gnommoweb IN repo IN dim:glitch_university` (is_isa=true, via "is a … of" pattern)
|
||
|
||
---
|
||
|
||
### Relationship Cue Patterns
|
||
|
||
Certain keyword patterns in intercepted text are direct cues to the memory layer that a semantic relationship is being expressed. The middleman scans every prompt for these patterns. When matched, the relationship is extracted and queued for insertion into the IN table — bypassing the saliency threshold, since the relationship has been made explicit.
|
||
|
||
Agents and humans interacting with Agent0 should be aware that using these patterns causes the middleman to build or update the world model.
|
||
|
||
**ISA patterns** (`is_isa = true`) — the subject is a type or instance of the object:
|
||
|
||
| Pattern | Example |
|
||
|---------|---------|
|
||
| `{X} is a {Y}` | gnommoweb is a repo |
|
||
| `{X} is an {Y}` | gnommoweb is an API |
|
||
| `{X} ISA {Y}` | gnommoweb ISA repo |
|
||
| `{X} is a kind of {Y}` | State is a kind of region |
|
||
| `{X} is a type of {Y}` | gnommoweb is a type of service |
|
||
| `{X} is an instance of {Y}` | dobby is an instance of agent |
|
||
| `{X} kind of {Y}` | gnommoweb kind of repo |
|
||
| `{X} type of {Y}` | gnommoweb type of service |
|
||
| `{X} instance of {Y}` | dobby instance of agent |
|
||
|
||
**ISPART patterns** (`is_isa = false`) — the subject is a member, part, or component of the object:
|
||
|
||
| Pattern | Example |
|
||
|---------|---------|
|
||
| `{X} is part of {Y}` | gnommoweb is part of Glitch University |
|
||
| `{X} ISPART {Y}` | gnommoweb ISPART glitch_university |
|
||
| `{X} part of {Y}` | gnommoweb part of Agent0 |
|
||
| `{X} belongs to {Y}` | gnommoweb belongs to Glitch University |
|
||
| `{X} is owned by {Y}` | gnommoweb is owned by jenstandstad |
|
||
| `{X} owned by {Y}` | gnommoweb owned by jenstandstad |
|
||
| `{X} member of {Y}` | dobby member of agent_pool |
|
||
| `{X} is a member of {Y}` | dobby is a member of agent_pool |
|
||
| `{X} runs on {Y}` | gnommoweb runs on Docker |
|
||
| `{X} hosted by {Y}` | gnommoweb hosted by ramanujan |
|
||
| `{X} deployed on {Y}` | gnommoweb deployed on Docker |
|
||
| `{X} contained in {Y}` | gnommoweb contained in agent0_stack |
|
||
|
||
**The `of {Z}` dimension modifier:**
|
||
|
||
When an ISA pattern is followed by `of {Z}`, the named entity `{Z}` becomes the dimension for the extracted edge. This allows natural language to directly specify the context in which a classification holds:
|
||
|
||
```
|
||
"gnommoweb is a repo of Glitch University"
|
||
→ gnommoweb IN repo IN dim:glitch_university (is_isa=true)
|
||
|
||
"Michigan is a state of USA"
|
||
→ michigan IN state IN dim:usa (is_isa=true)
|
||
```
|
||
|
||
Without the `of {Z}` modifier, the dimension defaults to `type` for ISA patterns and the most appropriate seed dimension for ISPART patterns (inferred by the cloud LLM during the write step, or defaulting to `membership`).
|
||
|
||
**Cue-triggered writes bypass the saliency threshold.** Explicit relationship cues are treated as high-confidence signals regardless of how many times a concept has previously appeared. The extracted triple goes directly into the write queue with `source: inferred` and a confidence score assigned by the pattern type (exact keyword cues score higher than positional inference).
|
||
|
||
---
|
||
|
||
### Writing — cloud-triggered, async
|
||
|
||
When a concept's saliency crosses the **write threshold**, Middleman queues it for background processing (cloud LLM calls must not block the prompt response path):
|
||
|
||
1. Call cloud LLM: "What is `{concept}`?" with the current dimension taxonomy as a closed list and a structured output prompt requesting `(concept, parent, dimension, is_isa)` triples
|
||
2. Parse response into IN table INSERT statements
|
||
3. Attempt inserts; route constraint violations to the conflict resolver
|
||
4. Update `novelty` score in SOAS for the concept
|
||
|
||
### Reading — spontaneous prompt enrichment
|
||
|
||
On every intercepted prompt:
|
||
|
||
1. Tokenise the full prompt string, extract tokens >4 chars, normalise
|
||
2. Look up each token in SOAS, update encounter counts and saliency
|
||
3. For tokens above the **read threshold**, query the IN table for all edges involving the concept
|
||
4. Traverse each chain upward (configurable max depth)
|
||
5. Format as a `<recollection>` block and prepend to the prompt before forwarding to Ollama
|
||
|
||
Example output:
|
||
|
||
```
|
||
<recollection>
|
||
gnommoweb: [type] repo → software-artifact
|
||
[membership] Glitch-University → Agent0-infrastructure
|
||
[tech] FastAPI → Python
|
||
glitch.university: [type] platform → web-service
|
||
[membership] Agent0 → Glitch-Hunter-project
|
||
</recollection>
|
||
```
|
||
|
||
Only edges above the confidence threshold and within the recency window are included. The agent does not search for these — they appear spontaneously.
|
||
|
||
**Read and write thresholds are separate and independently tunable.** Reading (DB lookup) is cheap; writing (cloud LLM call) is expensive. The write threshold should be meaningfully higher than the read threshold.
|
||
|
||
---
|
||
|
||
## Conflict Resolution — The Nightly Therapy Model
|
||
|
||
The IN table is rigorous and autistic: it cannot hold contradictions. Any collision is immediately routed to the **resolution queue** — a separate table — where it waits for the nightly resolution job to process it. During this period the world model stands unchanged; the old fact continues to be served in recollections, marked with a `?` to signal pending dissonance.
|
||
|
||
### On collision (immediate, synchronous)
|
||
|
||
1. Classify the collision type by reading `is_isa` on both rows: ISA+ISA, ISPART+ISPART, or misclassification
|
||
2. Insert the rejected fact into the **resolution queue** with full context: existing edge, incoming edge, dimension, collision type, timestamp
|
||
3. Return normally — the proxy response is not blocked
|
||
|
||
### During the day (recollection engine)
|
||
|
||
Concepts with entries in the resolution queue are rendered with a `?` marker:
|
||
|
||
```
|
||
<recollection>
|
||
gnommoweb: [type] container
|
||
[type?] repo — pending resolution
|
||
</recollection>
|
||
```
|
||
|
||
The agent sees that a fact is contested. The world model is not wrong — it is incomplete. The `?` marker disappears after the nightly job resolves or dismisses the conflict.
|
||
|
||
### Nightly resolution job
|
||
|
||
Runs as a background thread inside the Festinger proxy process on the schedule set in the `config` table (`resolution_schedule`). Can also be triggered manually via `POST /resolve/run` — a corresponding button is exposed in the Festinger admin UI. Uses the model configured in `resolve_model_id`.
|
||
|
||
For each item in the queue, calls the configured LLM with both facts and the collision type, receives a structured decision, and applies it:
|
||
|
||
**ISA+ISA collision (dimension too coarse):**
|
||
- LLM outcome A — **decompose**: suggest two new dimension names. System creates new SOAS entries and root nodes, inserts both facts into their respective new dimensions, marks queue item resolved.
|
||
- LLM outcome B — **dismiss**: the incoming fact is noise or wrong. Queue item marked dismissed. World model stands.
|
||
|
||
**ISPART+ISPART collision (factual contradiction):**
|
||
- LLM outcome A — **update**: the incoming fact is more current. System removes old IN edge, inserts new one, marks queue item resolved.
|
||
- LLM outcome B — **dismiss**: the existing fact is still correct. Queue item marked dismissed.
|
||
|
||
**Misclassification (ISA+ISPART in same dimension):**
|
||
- LLM suggests the correct dimension for the incoming fact. System inserts it in the corrected dimension (no collision), marks queue item resolved.
|
||
|
||
### Resolution queue schema
|
||
|
||
| Column | Type | Notes |
|
||
|--------|------|-------|
|
||
| id | INT PK | auto-increment |
|
||
| concept_id | INT FK | references SOAS |
|
||
| existing_parent_id | INT FK | the parent currently in the IN table |
|
||
| incoming_parent_id | INT FK | the rejected parent |
|
||
| dimension_id | INT FK | the dimension where the collision occurred |
|
||
| collision_type | ENUM | `isa_isa`, `ispart_ispart`, `misclassification` |
|
||
| status | ENUM | `pending`, `resolved`, `dismissed` |
|
||
| resolution | TEXT | JSON record of what the nightly job decided and did |
|
||
| created_at | TIMESTAMP | when the collision occurred |
|
||
| resolved_at | TIMESTAMP | when the nightly job processed it |
|
||
|
||
### Properties of this model
|
||
|
||
- **IN table is always consistent** — agents never receive contradictory recollections from confirmed facts
|
||
- **Resolution is deliberate, not reactive** — the nightly job processes dissonance calmly, with full context, not under prompt-response time pressure
|
||
- **Dismissal is a first-class outcome** — not every collision is a real problem; the LLM can decide the world model is correct and the incoming fact was noise
|
||
- **The `?` marker is the fuzziness adjunct** — it surfaces uncertainty to agents without compromising the graph's integrity
|
||
- **`/conflicts` endpoint** — exposes the full queue (pending and recently resolved) for human inspection and override
|
||
|
||
---
|
||
|
||
## Graph Properties
|
||
|
||
- **Acyclic**: the unique index on `(concept_id, dimension_id)` enforces single-parent per concept per dimension, making each dimension a forest of trees — structurally acyclic without any runtime check
|
||
- **Shared**: one graph for all agents — recollections represent shared facts about the project, not per-agent beliefs. Agents already have per-agent memory in Agent Zero's own FAISS layer
|
||
- **Contradiction-resistant**: the index makes it structurally impossible to store conflicting facts in the same dimension. Contradictions surface as insert failures, not silent overwrites
|
||
- **Self-organising**: dimension taxonomy starts with a small seed list and decomposes on demand. No human needs to pre-define the full ontology
|
||
|
||
---
|
||
|
||
## Saliency Decay
|
||
|
||
Two decay mechanisms, operating independently:
|
||
|
||
- **SOAS recency** (`last_seen` timestamp): concepts not seen for a long time are deprioritised for recollection injection but not deleted. If gnommoweb reappears in a prompt, `last_seen` updates and its recollections resurface immediately
|
||
- **IN edge recency** (`last_confirmed` timestamp): edges not corroborated by recent prompts are given lower weight in recollection injection, making the recollection block favour currently-relevant facts
|
||
|
||
Decay does not delete knowledge — it adjusts injection priority. The graph remains intact.
|
||
|
||
---
|
||
|
||
## Dimension Taxonomy — Seed List
|
||
|
||
A small, orthogonal set of initial dimensions. Each dimension answers a specific question about a concept. New dimensions emerge through decomposition; this list is the starting skeleton.
|
||
|
||
| Dimension | Question | is_isa |
|
||
|-----------|----------|--------|
|
||
| `type` | What kind of thing is this? | true |
|
||
| `membership` | What system or project does this belong to? | false |
|
||
| `runs-on` | What infrastructure hosts or executes this? | false |
|
||
| `tech` | What technology stack is it built with? | false |
|
||
| `owned-by` | Who is responsible for this? | false |
|
||
| `geography` | Where is this spatially or organisationally located? | false |
|
||
|
||
Root nodes for each dimension are seeded at bootstrap. The `type` dimension is expected to be the first to decompose as domain-specific concepts accumulate.
|
||
|
||
---
|
||
|
||
## Components
|
||
|
||
| Component | Description | State |
|
||
|-----------|-------------|-------|
|
||
| Proxy core | FastAPI Ollama-compatible HTTP proxy | ✅ built |
|
||
| Loop detector | Session-scoped repeat detection + mitigations | ✅ built |
|
||
| Config system | Hot-reloading YAML config | ✅ built |
|
||
| SOAS store | Concept vocabulary + saliency DB table | ✅ built |
|
||
| IN table store | Acyclic concept graph with correct indexes | ✅ built |
|
||
| Dictionary bootstrap | Pre-seed SOAS with common English at saliency 0 | ✅ built |
|
||
| Dimension bootstrap | Seed root nodes for initial dimension taxonomy | ✅ built |
|
||
| Saliency engine | Tokenise prompt, score tokens, update SOAS counts | ✅ built |
|
||
| Recollection engine | Query IN table, traverse chains, format + inject block | ✅ built |
|
||
| Memory writer | Write-threshold trigger → async cloud LLM → NL→IN parse → insert | ✅ built |
|
||
| Conflict resolver | On collision: classify type, insert into resolution queue immediately | ✅ built |
|
||
| Resolution queue | Pending/resolved/dismissed conflicts with full context | ✅ built |
|
||
| Nightly resolution job | Drain queue via cloud LLM; apply decompose/update/dismiss decisions | ✅ built |
|
||
| `/conflicts` endpoint | Expose queue (pending + recent) for human inspection and override | ✅ built |
|
||
| Persistence | Postgres from day one; English dictionary pre-loaded into SOAS at init | ✅ built |
|
||
|
||
---
|
||
|
||
## Task Breakdown
|
||
|
||
### Phase 1 — Foundation (complete)
|
||
- [x] **T01** Proxy core: FastAPI Ollama-compatible server forwarding `/api/chat` and `/api/generate`
|
||
- [x] **T02** Loop detector: session-scoped exact-match repetition detection
|
||
- [x] **T03** Mitigations: temperature boost, forbidden action injection, history truncation, circuit breaker
|
||
- [x] **T04** Hot-reload YAML config
|
||
- [x] **T05** Docker container + docker-compose service entry
|
||
|
||
### Phase 2 — Persistence Layer
|
||
- [x] **T06** Postgres service: add `postgres` container to docker-compose; connection config in `config.yaml`
|
||
- [x] **T07** `models` table: `id` SERIAL PK, `provider` VARCHAR, `model_name` VARCHAR, `api_key` VARCHAR, `created_at` TIMESTAMPTZ
|
||
- [x] **T08** `config` table: `key` VARCHAR PK, `value` TEXT, `updated_at` TIMESTAMPTZ; seed with default values for all config keys
|
||
- [x] **T09** SOAS table: `id` SERIAL PK, `token` VARCHAR UNIQUE, `encounter_count` INT default 0, `last_seen` TIMESTAMPTZ, `saliency` FLOAT default 0, `novelty` FLOAT default 0. All tokens lowercase. Unique index on `token`.
|
||
- [x] **T10** URD table: `id` INT FK → soas, `parent_id` INT FK → soas, `dim_id` INT FK → soas, `is_isa` BOOLEAN, `confidence` FLOAT, `last_confirmed` TIMESTAMPTZ, `source` VARCHAR. PK `(id, parent_id, dim_id)`. Unique index `(id, dim_id)`.
|
||
- [x] **T11** Resolution queue table: `id` SERIAL PK, `concept_id` INT FK → soas, `existing_parent_id` INT FK → soas, `incoming_parent_id` INT FK → soas, `dim_id` INT FK → soas, `collision_type` VARCHAR, `status` VARCHAR default 'pending', `resolution` JSONB, `created_at` TIMESTAMPTZ, `resolved_at` TIMESTAMPTZ
|
||
- [x] **T12** English dictionary bootstrap: bulk-load word list into SOAS with `saliency=0`, `novelty=0`, `encounter_count=0` on container init. Skip existing tokens.
|
||
- [x] **T13** Dimension bootstrap: insert SOAS entries and self-referential URD root nodes (`id = parent_id = dim_id`) for the 6 seed dimensions
|
||
|
||
### Phase 3 — Saliency Engine and Prompt Parsing
|
||
- [x] **T14** Tokeniser: split on whitespace/punctuation; apply compound token rule (consecutive capitalised tokens → single underscore-joined lowercase token); no minimum length; strip punctuation; lowercase all
|
||
- [x] **T15** Relationship cue scanner: regex/pattern scan for ISA and ISPART cue patterns; extract `(subject, parent, dimension_modifier, is_isa)` triples; handle `of {Z}` dimension modifier
|
||
- [x] **T16** SOAS lookup + update: increment `encounter_count`, update `last_seen`, recalculate `saliency` (log scale) for all extracted tokens; read thresholds from `config` table
|
||
- [x] **T17** Threshold evaluation: read `saliency_read_threshold` and `saliency_write_threshold` from config; cue-extracted triples bypass write-threshold entirely
|
||
|
||
### Phase 4 — Recollection Engine (Read Path)
|
||
- [x] **T16** URD query: for each above-read-threshold token, execute the canonical recollection query; filter by `confidence` floor and `last_confirmed` recency window
|
||
- [x] **T17** Recollection formatter — hit path: enumerate query rows; group by concept; render each edge as `[dim_token] parent_token`; append `?` for edges with a pending resolution queue entry
|
||
- [x] **T18** Recollection formatter — zero-hit path: for salient concepts with no URD rows, emit the `? concept: no recollection` prompt block including the `gutask iknowthat` usage hint
|
||
- [x] **T19** Prompt injection: prepend recollection block before forwarding to Ollama
|
||
- [x] **T20** Recollection config: max concepts per block, confidence floor, recency window, injection position
|
||
|
||
### Phase 5 — Memory Writer (Write Path)
|
||
- [x] **T21** `POST /iknowthat` endpoint: accept `{fact: string}`, parse `-isa`/`-ispart` flags and `in context of` clause, run tokeniser, upsert SOAS, insert into URD with `confidence=1.0, source=gutask`; route collisions to resolution queue with priority flag
|
||
- [x] **T22** `gutask iknowthat` command (in `/Users/jenstandstad/Projects/gutasktool`): parse fact string, POST to Festinger `/iknowthat`, surface result to agent
|
||
- [x] **T23** Write queue: async background queue for concepts crossing the write threshold; cue-extracted triples enter directly regardless of threshold
|
||
- [x] **T24** LLM client: support `claude` and `openai` providers; load provider/model/key from `models` table via `write_model_id` config key; structured prompt requesting `(concept, parent, dimension, is_isa, confidence)` triples as JSON
|
||
- [x] **T25** NL→IN parser: validate triples against SOAS and known dimensions; create new SOAS entries for unknown tokens; apply compound token rule
|
||
- [x] **T26** URD insert pipeline: check `urd_by_concept_dim` in-memory first; on miss attempt Postgres insert; on hit or UniqueViolation route to conflict resolver; set `source` field per write path
|
||
|
||
### Phase 6 — Conflict Resolution
|
||
- [x] **T25** Collision handler: on unique constraint violation, classify type via `is_isa` flags, insert immediately into resolution queue
|
||
- [x] **T26** Recollection engine update: check resolution queue for pending items per concept; render pending edges with `[dim?]` marker
|
||
- [x] **T27** Nightly resolution job: background thread, schedule from `config` table (`resolution_schedule`); for each `pending` queue item, call LLM configured in `resolve_model_id` with both facts and collision type, receive JSON decision (`decompose` / `update` / `dismiss`)
|
||
- [x] **T28** Resolution applicator — decompose: create new dimension SOAS entries and root nodes; insert both facts in respective new dimensions; mark queue item resolved
|
||
- [x] **T29** Resolution applicator — update: remove old IN edge, insert new fact, mark queue item resolved
|
||
- [x] **T30** Resolution applicator — dismiss: mark queue item dismissed; world model unchanged; the `[dim?]` marker disappears from recollections
|
||
- [x] **T31** `/conflicts` endpoint: list pending and recently resolved/dismissed items with full context; support human override (force-dismiss, force-resolve)
|
||
- [x] **T32** `POST /resolve/run` endpoint: manually trigger the nightly resolution job outside of its schedule
|
||
- [x] **T33** Admin UI: minimal HTML page served by Festinger at `/admin`; shows pending conflicts count, last resolution run timestamp, and a "Run resolution now" button wired to `POST /resolve/run`
|
||
|
||
### Phase 7 — Hardening
|
||
- [ ] **T32** Latency guard: tokenisation + saliency lookup + recollection query must not add >50ms to prompt round-trip; use connection pooling
|
||
- [ ] **T33** Write path fully async: all cloud LLM calls, IN inserts, and queue operations run in background workers; proxy response never waits
|
||
- [ ] **T34** Integration test: plain prompt → compound token extraction → saliency update → recollection injection → Ollama round trip
|
||
- [ ] **T35** Integration test: cue pattern in prompt ("gnommoweb is a repo of Glitch University") → extracted triple bypasses threshold → IN insert → recollection on next prompt
|
||
- [ ] **T36** Integration test: write threshold → cloud LLM write → IN insert → recollection on next prompt
|
||
- [ ] **T37** Integration test: ISA+ISA collision → immediate queue insert → `[type?]` marker in recollection → nightly job → decompose → clean recollection across two dimensions
|
||
- [ ] **T38** Integration test: ISPART+ISPART collision → queue insert → nightly job → dismiss → world model unchanged, marker gone
|
||
|
||
---
|
||
|
||
## Resolved Design Decisions
|
||
|
||
| Question | Decision |
|
||
|----------|----------|
|
||
| Single relation or ISA+ISPART tables? | Single IN operator; `is_isa` boolean flag outside the index annotates edge type |
|
||
| ISA vs ISPART bleed | Resolved by dimension: `type` dimension = ISA; all other dimensions = ISPART |
|
||
| Shared vs per-agent graph | Shared — recollections are project-wide facts; per-agent memory remains in Agent Zero's FAISS layer |
|
||
| Saliency decay | Dual mechanism: SOAS `last_seen` for concept recency; IN `last_confirmed` for edge recency. Decay adjusts injection priority, never deletes knowledge |
|
||
| Recollection depth | Depth = 1, no chain traversal. Single flat query against URD for direct edges of each salient concept. Property-loop problem is dissolved by design — self-referential chains cannot arise without traversal. |
|
||
| Write path blocking | Fully async — cloud LLM calls queued in background, never block proxy response |
|
||
| Dimension taxonomy | Seed list of 6 orthogonal dimensions; decomposes on demand via nightly resolution job when ISA+ISA collision is queued |
|
||
| Collision semantics | ISA+ISA → dimension too coarse → decompose. ISPART+ISPART → factual contradiction → arbitrate. ISA+ISPART → misclassification → flag |
|
||
| Contradiction resistance | Structural — unique index on `(concept_id, dimension_id)` makes conflicting facts physically uninsertable in the same dimension |
|
||
| New dimensions | Emerge as SOAS tokens; no schema change needed; root nodes created at decomposition time |
|
||
| Conflict resolution timing | Immediate queue insert on collision; nightly job drains the queue via cloud LLM; outcomes are decompose / update / dismiss |
|
||
| Database | Postgres from day one — no SQLite POC. English dictionary bulk-loaded into SOAS at container init. |
|
||
| Tokenisation | Tokens ≥5 chars, lowercased. Consecutive capitalised tokens merged into single underscore-joined token (`Glitch University` → `glitch_university`). |
|
||
| Relationship cue parsing | ISA and ISPART keyword patterns in intercepted text trigger direct triple extraction, bypassing saliency threshold. `of {Z}` modifier sets the dimension. |
|
||
| Cue-triggered writes | Explicit cue patterns are high-confidence signals; extracted triples go straight to the write queue with `source: inferred`, no threshold gate. |
|
||
| Zero-hit recollection | Salient concepts with no URD entries emit a `? concept: no recollection` prompt block instructing the agent to clarify or store the fact via `gutask iknowthat`. |
|
||
| Manual write path | `gutask iknowthat 'X -isa/-ispart Y in context of Z'` inserts directly into URD with `confidence=1.0, source=manual`, bypassing saliency threshold and cloud LLM. |
|
||
| In-memory layer | SOAS and URD cached in Python dicts at startup. Reads are zero-network. IDs always originate from Postgres. Collision detection uses `dict[(concept_id, dim_id)]` mirroring the Postgres unique index. Postgres is the safety net for race conditions. |
|
||
| Token length | No minimum — all tokens indexed. Frequency and novelty determine whether they surface. Optimize later if needed. |
|
||
| Nightly job execution | Background thread inside proxy process. Triggered by cron schedule (config table) or manually via `POST /resolve/run` + admin UI button. |
|
||
| Recollection injection | Prepended to existing system message content. If no system message exists, a new one is inserted at position 0. System message position provides the strongest grounding anchor for instruction-tuned models. |
|
||
| LLM configuration | `models` table (provider, model_name, api_key). `config` table keys `write_model_id` and `resolve_model_id` select which model each purpose uses. Supports `claude` and `openai` providers. |
|
||
| Source values | `cloud_llm` (saliency write path), `inferred` (cue pattern extraction), `festinger` (nightly resolution job), `gutask` (gutask iknowthat command). |
|
||
| gutask iknowthat interface | gutasktool at `/Users/jenstandstad/Projects/gutasktool`. Command POSTs to Festinger's `/iknowthat` endpoint — no direct Postgres access from gutasktool. |
|
||
|
||
---
|
||
|
||
## Test Cases
|
||
|
||
The in-memory cache layer enables full unit testing without a live database. Tests pre-populate `soas_by_token`, `soas_by_id`, `urd_by_concept`, `urd_by_concept_dim`, and `pending_conflicts` directly, then exercise the logic under test and assert on the resulting state.
|
||
|
||
---
|
||
|
||
### Test A — Prompt includes a concept not in the cache
|
||
|
||
**Scenario:** An agent sends a prompt referencing `gnommoweb`. The concept exists nowhere in SOAS or URD.
|
||
|
||
**Setup:**
|
||
```python
|
||
soas_by_token = {} # empty — concept is completely unknown
|
||
urd_by_concept = {}
|
||
urd_by_concept_dim = {}
|
||
pending_conflicts = set()
|
||
```
|
||
|
||
**Input prompt:** `"Please update gnommoweb to use FastAPI instead"`
|
||
|
||
**Expected behaviour:**
|
||
1. Tokeniser extracts: `gnommoweb` (7 chars ✓), `fastapi` (7 chars ✓), `please` (6 chars but common English → saliency 0), `update` (6 chars, common → saliency 0), `instead` (7 chars, common → saliency 0)
|
||
2. `gnommoweb` and `fastapi` not in `soas_by_token` → both are new tokens. In a test with Postgres mocked, the mock returns `id=101` for `gnommoweb` and `id=102` for `fastapi`. Both added to SOAS dicts.
|
||
3. Saliency for both: `log(1)` — first encounter, below read threshold
|
||
4. URD lookup: skipped (below threshold) — no recollection block emitted for these concepts
|
||
|
||
**Edge case variant — above threshold:** pre-seed `soas_by_token["gnommoweb"]` with `encounter_count=50, saliency=0.9` (above read threshold) but keep `urd_by_concept` empty.
|
||
|
||
**Expected behaviour (variant):**
|
||
1. Saliency lookup: above read threshold ✓
|
||
2. URD lookup: `urd_by_concept.get(101, [])` → empty list
|
||
3. Zero-hit path triggered
|
||
4. Recollection block contains:
|
||
```
|
||
? gnommoweb: no recollection. If not a typo, store it before proceeding:
|
||
gutask iknowthat 'gnommoweb -isa <parent> in context of <dimension>'
|
||
gutask iknowthat 'gnommoweb -ispart <system> in context of <dimension>'
|
||
```
|
||
|
||
**Assertions:**
|
||
- `pending_conflicts` unchanged (no collision occurred)
|
||
- Resolution queue empty
|
||
- Recollection block contains `? gnommoweb`
|
||
- Prompt forwarded to Ollama with recollection block prepended
|
||
|
||
---
|
||
|
||
### Test B — Prompt includes "A ISA B" conflicting with existing world model
|
||
|
||
**Scenario:** The world model already holds `gnommoweb ISA repo` in the `type` dimension. A new prompt contains the explicit cue `"gnommoweb is a container"`, creating an ISA+ISA collision.
|
||
|
||
**Setup:**
|
||
```python
|
||
# SOAS
|
||
soas_by_token = {
|
||
"gnommoweb": SoasRow(id=101, saliency=1.2, novelty=1.0),
|
||
"repo": SoasRow(id=201, saliency=0.8, novelty=0.5),
|
||
"container": SoasRow(id=202, saliency=0.6, novelty=0.4),
|
||
"type": SoasRow(id=1, saliency=0.0, novelty=0.0), # seed dimension
|
||
}
|
||
soas_by_id = {101: "gnommoweb", 201: "repo", 202: "container", 1: "type"}
|
||
|
||
# URD — gnommoweb ISA repo IN type (existing confirmed fact)
|
||
existing_edge = UrdEdge(concept_id=101, parent_id=201, dim_id=1,
|
||
is_isa=True, confidence=0.9, source="cloud_llm")
|
||
urd_by_concept = {101: [existing_edge]}
|
||
urd_by_concept_dim = {(101, 1): existing_edge}
|
||
pending_conflicts = set()
|
||
```
|
||
|
||
**Input prompt:** `"gnommoweb is a container deployed on Docker"`
|
||
|
||
**Expected behaviour:**
|
||
1. Cue scanner matches `"is a"` pattern: extracts triple `(gnommoweb, container, type, is_isa=True)`
|
||
2. Collision check: `(101, 1)` found in `urd_by_concept_dim` → collision
|
||
3. Existing edge `is_isa=True`, incoming `is_isa=True` → ISA+ISA collision type
|
||
4. Resolution queue insert (mocked Postgres): `{concept_id=101, existing_parent_id=201, incoming_parent_id=202, dim_id=1, collision_type="isa_isa", status="pending"}`
|
||
5. `pending_conflicts.add(101)`
|
||
6. No URD modification — world model unchanged
|
||
|
||
**Recollection block for this prompt:**
|
||
```
|
||
<recollection>
|
||
gnommoweb: [type?] repo — conflict pending
|
||
</recollection>
|
||
```
|
||
|
||
**Assertions:**
|
||
- `urd_by_concept_dim[(101, 1)]` still points to the original `repo` edge (unchanged)
|
||
- `pending_conflicts == {101}`
|
||
- Resolution queue has exactly one entry with `collision_type="isa_isa"` and `status="pending"`
|
||
- No Postgres URD insert attempted
|
||
- Recollection renders `[type?]` marker for `gnommoweb`
|
||
|
||
---
|
||
|
||
### Test C — Nightly job: queue processing and dimension decomposition
|
||
|
||
**Scenario:** The resolution queue contains the ISA+ISA collision from Test B. The nightly job runs, calls the cloud LLM (mocked), receives a decomposition decision, and updates the world model.
|
||
|
||
**Setup:** state as at end of Test B, plus:
|
||
```python
|
||
resolution_queue = [
|
||
QueueEntry(id=1, concept_id=101, existing_parent_id=201, incoming_parent_id=202,
|
||
dim_id=1, collision_type="isa_isa", status="pending")
|
||
]
|
||
```
|
||
|
||
**Mocked cloud LLM response:**
|
||
```json
|
||
{
|
||
"decision": "decompose",
|
||
"existing_dimension": "artifact-type",
|
||
"new_dimension": "deployment-type",
|
||
"reasoning": "repo describes what gnommoweb is as a software artifact; container describes how it is deployed"
|
||
}
|
||
```
|
||
|
||
**Expected behaviour:**
|
||
1. Nightly job fetches pending queue entries from Postgres
|
||
2. For entry id=1, calls cloud LLM → receives decompose decision
|
||
3. **New dimensions:** insert `artifact-type` into Postgres SOAS → returns `id=401`; insert `deployment-type` → returns `id=402`. Add both to SOAS dicts.
|
||
4. **Root nodes:** insert `(401,401,401)` and `(402,402,402)` into Postgres URD (self-referential dimension roots)
|
||
5. **Migrate existing edge:** delete `(101, 201, 1)` from URD, insert `(101, 201, 401)` — gnommoweb ISA repo IN artifact-type
|
||
6. **Insert new edge:** insert `(101, 202, 402)` — gnommoweb ISA container IN deployment-type
|
||
7. **Update in-memory cache:**
|
||
- `soas_by_token["artifact-type"] = SoasRow(id=401, ...)`
|
||
- `soas_by_token["deployment-type"] = SoasRow(id=402, ...)`
|
||
- Remove `urd_by_concept_dim[(101, 1)]`, add `[(101, 401)]` and `[(101, 402)]`
|
||
- Update `urd_by_concept[101]` to two new edges
|
||
- `pending_conflicts.discard(101)`
|
||
8. Mark queue entry resolved with resolution JSON
|
||
9. Signal proxy `/reload` (or nightly job updates cache directly if in-process)
|
||
|
||
**Final recollection block for gnommoweb:**
|
||
```
|
||
<recollection>
|
||
gnommoweb: [artifact-type] repo [deployment-type] container
|
||
</recollection>
|
||
```
|
||
|
||
**Assertions:**
|
||
- `urd_by_concept_dim` contains `(101, 401)` and `(101, 402)`, not `(101, 1)`
|
||
- `urd_by_concept[101]` has exactly two edges
|
||
- `pending_conflicts` does not contain `101`
|
||
- `soas_by_token` contains `artifact-type` and `deployment-type` with ids from Postgres
|
||
- Resolution queue entry has `status="resolved"` and non-null `resolution` JSON
|
||
- No `[type?]` marker in subsequent recollection for gnommoweb
|