gupi-osint-board/docs/story-graph.md

# Story flow graph: the mystery editor

Status: design proposal for discussion. Supersedes the slot/chapter progression
model (`mystery_chapters` + `cutscenes.slot` + `selectPendingCutscene`). Builds on
the NPC/dialogue/cutscene work in [narrative-todo.md](narrative-todo.md).

## Concept

A mystery is authored as a **directed flow graph** of nodes. The author drags
nodes on a canvas and connects an **output terminal** of one node to another node
with a directed, curved edge. A playthrough is a walk of that graph.

- It is a **graph, not a tree**: gates may route back to earlier nodes ("not
  convincing → keep investigating"), so cycles are expected and intended. The
  runtime only re-traverses a node when the player acts again, so loops never spin
  on their own.
- Every mystery has exactly **one entrypoint node**, stored as a foreign key on
  the mystery (`mysteries.entry_node_id`) so "at most one" is structural.

## Node types

Five types, one uniform node+terminal shape. Terminals mean different things per
type, but the wiring (terminal → edge → node) is identical everywhere.

| type | what it does | terminals |
|---|---|---|
| `cutscene` | Renders a **custom presentation component** (title card, video, 3D) chosen from a frontend registry. Deliberately opaque to the graph so bespoke set-pieces don't clutter the dialogue tree. | usually 1 (`continue`), N allowed |
| `dialogue` | The standard NPC dialogue box: a sequence of steps, optionally ending in player **choices**. | 1 (linear) or 1 per choice |
| `level` | Instantiates a playable board from a level template version and hands control to the investigation. | 1 (`report_back`) |
| `det_gate` | Deterministic gate: evaluates conditions on player/board state and routes to a matching terminal. | 1 per condition branch (+ else) |
| `llm_gate` | LLM-powered gate: the model reads allowlisted player state and returns one of the terminal keys (constrained output → reliable routing). | 1 per verdict |

A **cutscene** and a **dialogue** are different because a cutscene is arbitrary
custom UI (its component decides its own presentation and when it completes),
while a dialogue is the shared, data-driven NPC box. Keeping them separate means a
one-off React set-piece never has to pretend to be a dialogue sequence.



## Cutscenes, Dialogue node and  "utterances" 
  A node has zero to many utterances.
  [ ] Cutscene : That react can use potential **utterances** such that for example, it can play 
  an ordered sequence utterance that brief the player: `(speaker NPC, pose, text)`. 
  This is relevant for the linear cutscene component. (No user actions)
  A node can be marked has_utterances which permits the admin user to add utterances in order.

  [ ] Dialogue. This is another type of node that always has utterances. It invokes the standard NPC dialogue component.  The node consist of a graph where utterances either are spoken by the NPC or available for selection.
  Example :  if the NPC utters "Are you ready?" this has two child utterances marked "player" which could be "yes" and no. The user may select these. "No" could in principle point back to the same utterance and "yes" to the next. If an utterance has a non-NULL terminal id reference, then the game advances to the node pointed to by that terminal. Available terminals are only those who have the current dialogue node as it parent.

# Gates
[ ] There exists "det_gate" nodes and "llm_gate" nodes. We begin with the deterministic gate only. The end result of a previous node is sent to a "det_gate". For dialogue nodes, the entire "dialogue" array (chosen utterances and spoken NPC utterance) are sent. For levels, typically the case report is sent. The det gate can inspect the output of previous and determine if the story should advance through one of its output terminals. For now, we implement a particularly dumb det-gate, which always returns first terminal leading to the mystery being solved.

## Data model

Core graph (three tables), plus one typed subtype table per node type — no
untyped `config` blob, consistent with the exhibit model.

```sql
CREATE TABLE osint.story_nodes (
  id UUID PRIMARY KEY,
  mystery_id UUID NOT NULL REFERENCES osint.mysteries(id) ON DELETE CASCADE,
  node_type TEXT NOT NULL CHECK (node_type IN ('cutscene','dialogue','level','det_gate','llm_gate')),
  label TEXT NOT NULL DEFAULT '',
  has_utterances BOOLEAN NOT NULL DEFAULT FALSE,
  xpos DOUBLE PRECISION NOT NULL,
  ypos DOUBLE PRECISION NOT NULL,
  -- Folded, type-specific scalars (nullable, kept honest by the CHECKs below).
  level_template_version_id UUID REFERENCES osint.level_template_versions(id),
  component_key TEXT,  -- cutscene: frontend React component; gate: backend gate function
  CHECK (node_type <> 'level' OR level_template_version_id IS NOT NULL),
  CHECK (level_template_version_id IS NULL OR node_type = 'level'),
  CHECK (node_type NOT IN ('cutscene','det_gate','llm_gate') OR component_key IS NOT NULL),
  CHECK (component_key IS NULL OR node_type IN ('cutscene','det_gate','llm_gate'))
);

-- Output ports. A node has ONE implicit input (many edges may converge on it)
-- and N explicit output terminals.
CREATE TABLE osint.story_node_terminals (
  id UUID PRIMARY KEY,
  parent_node_id UUID NOT NULL REFERENCES osint.story_nodes(id) ON DELETE CASCADE,
  terminal_key TEXT NOT NULL,
  label TEXT NOT NULL DEFAULT '',
  -- The wire out of this port. NULL = unwired (authoring in progress, or a
  -- deliberate end). SET NULL keeps the port when its target node is deleted.
  to_node_id UUID REFERENCES osint.story_nodes(id) ON DELETE SET NULL,
  sort_order INTEGER NOT NULL DEFAULT 0,
  UNIQUE (parent_node_id, terminal_key)
);
-- A deferred constraint trigger enforces that to_node_id shares parent_node_id's
-- mystery (same pattern as the existing same-board checks).

-- entry_node_id must be nullable: nodes are inserted first, then the entrypoint
-- is set, avoiding a chicken-and-egg with story_nodes.mystery_id.
ALTER TABLE osint.mysteries ADD COLUMN entry_node_id UUID REFERENCES osint.story_nodes(id);
```

A terminal carries its own single outgoing wire (`to_node_id`), so there is no
separate edges table. Many terminals can still converge on one node.

### Subtype tables

```sql



-- One table for both NPC lines and player choices, distinguished by `utterer`.
-- Belongs directly to a story_node (dialogue or utterance-bearing cutscene).
CREATE TABLE osint.utterances (
  id UUID PRIMARY KEY,
  node_id UUID NOT NULL REFERENCES osint.story_nodes(id) ON DELETE CASCADE,
  utterer TEXT NOT NULL DEFAULT 'npc' CHECK (utterer IN ('npc','player')),
  npc_id UUID REFERENCES osint.npcs(id),   -- speaker for NPC lines; NULL for player choices
  pose_key TEXT,                            -- resolved with the usual pose fallback
  text TEXT NOT NULL,
  -- Intra-node conversation graph (WITHIN one dialogue node):
  parent_utterance_id UUID REFERENCES osint.utterances(id) ON DELETE CASCADE,       -- player options hang under the NPC prompt they answer
  advances_to_utterance_id UUID REFERENCES osint.utterances(id) ON DELETE SET NULL, -- next line; may loop back to the same one ("No")
  -- Inter-node exit: if set, choosing/finishing this utterance leaves the node.
  terminal_id UUID REFERENCES osint.story_node_terminals(id) ON DELETE SET NULL,
  effect TEXT,                              -- optional authored side-effect hook (vocabulary TBD)
  xpos DOUBLE PRECISION NOT NULL DEFAULT 0, -- position in the node's utterance sub-canvas
  ypos DOUBLE PRECISION NOT NULL DEFAULT 0,
  sort_order INTEGER NOT NULL DEFAULT 0
);

```
Several **choices may share one terminal. 
Example : Player selects an utterance that points to a terminal, which in turn points to another node of type "det_gate". The code loads the component_key "det_gate_lvl_1" which is is a small and passes the outcome
of the previous node to it. 
The det_gate script  counts the number of correct assertions and determines the correct output
terminal; the correct answer → a `proceed` terminal). A dialogue node with **no
choices** has a single default terminal, followed when the sequence ends — which
is how today's linear intro/debrief become plain dialogue nodes.

### Cutscene component registry (frontend)

Like the exhibit registry: `component_key → React component`. Each component owns
its own presentation and signals completion with an optional terminal key:

```ts
type CutsceneComponent = React.FC<{ onComplete: (terminalKey?: string) => void }>
// registry: { 'glass-harbour-diversion': GlassHarbourDiversion, ... }
```

For a single-terminal cutscene, `onComplete()` follows the only terminal.

## Editing UX: nested canvases

The same canvas engine (drag, pan, zoom, curved wires) is reused at two levels, so
each view stays uncluttered:

1. **Mystery graph** — `story_nodes` as cards, their output terminals as ports;
   wires are `terminal.to_node_id`.
2. **Utterance graph** — opening a dialogue (or utterance-bearing cutscene) node
   drills into *its* utterances. Utterances are draggable cards (`xpos`/`ypos`);
   the parent node's output terminals appear as **docked exit sinks** along one
   edge (no coordinates of their own — their home is the port in the mystery
   graph).

Authoring a dialogue is: add/move utterance cards, wire them, and wire the
branch-ends to the exit sinks. A single "draw a wire" gesture maps to storage by
what it connects:

- NPC line → player option ⇒ option grouping (`parent_utterance_id`)
- utterance → next NPC line ⇒ flow (`advances_to_utterance_id`), and a wire back
  to the same card is a loop ("No" → re-ask)
- utterance → an exit sink ⇒ leave the node (`terminal_id`)

A cutscene's utterance sub-canvas is the same editor, just linear (NPC-only, no
player branches).

## Runtime traversal (Phase 2)

The playthrough tracks a **current node** instead of a chapter:

```sql
-- replaces current_chapter_index; current_level_id stays (set while on a level node)
ALTER TABLE osint.playthroughs ADD COLUMN current_node_id UUID REFERENCES osint.story_nodes(id);
```

Walking the graph:

- **cutscene** → render its component; on `onComplete(key?)` follow that terminal's edge.
- **dialogue** → play steps; if it ends in choices, the picked choice's `terminal_id` is followed; otherwise the single default terminal.
- **level** → instantiate the board, set `current_level_id`, hand off to the investigation; the `report_back` terminal fires when the player reports back.
- **det_gate** → evaluate terminals in `sort_order`, first satisfied wins (NULL condition = else); follow it. No player-facing UI.
- **llm_gate** → gather allowlisted state, call the latest Claude model constrained to return one terminal key, follow it. Falls back to a designated terminal on error/timeout.
- **no outgoing edge** → the mystery ends.

Progress (visited nodes / seen dialogue) continues to reference stable authored
ids, exactly as `seen_dialogue` does today.

## Worked example — the Glass Harbour POC

The exact anatomy to build:

```
[cutscene: "glass-harbour-diversion"]        (title card, fades in/out on black)
    └─(continue)──▶ [dialogue: Almira briefing]
                        ├─(correct choice)──▶ [level: glass-harbor board]
                        └─(wrong choices)───▶ (retry terminal → loops back)   ← optional
                                                   │
                                          (report_back)
                                                   ▼
                                        [dialogue: Almira debrief]
                                            ├─(good)──▶ (end)
                                            └─(back)──▶ loops to the level
```

So: one cutscene node (custom component) → one dialogue node (Almira, choices) →
one level node → one dialogue node (debrief). The entrypoint is the cutscene node.

## Migration from the slot model

A clean cutover (only Glass Harbour exists), in the spirit of migration 006:

- Retire `mystery_chapters`, `cutscenes.slot`, `cutscenes.chapter_index`, and the
  slot-selection logic (`selectPendingCutscene`, `getSceneBySlot`, `getPendingCutscene`).
- What is today a `cutscene` row (a `dialogue_steps` sequence) becomes a
  **dialogue node**; `dialogue_steps` repoints to `dialogue_node_id`.
- The reserved `dialogue_choices` becomes real and gains `terminal_id`.
- `playthroughs.current_chapter_index` → `current_node_id`.
- Re-author Glass Harbour as the graph above through the editor.

## Validation rules

- Exactly one `entry_node_id`; recommended (soft) that it be a `cutscene`.
- Per-type terminal rules: `cutscene` ≥1, `dialogue` ≥1, `level` exactly 1,
  gates ≥1.
- Every terminal's `to_node_id` belongs to the same mystery as its parent node.
- Every utterance's `terminal_id` belongs to that utterance's node; its
  `parent_utterance_id` / `advances_to_utterance_id` stay within the same node.
- Warn (don't block) on unreachable nodes and terminals with no edge (deliberate
  ends are allowed).

## Phasing

1. **Schema + editor, authoring-only.** Graph tables + subtype stubs, CRUD API,
   and the draggable node canvas with curved edges, port wiring, and entrypoint
   marking. Reuses the board's screen↔canvas coordinate conversion, drag, and
   pan/zoom. Runtime still plays Glass Harbour via the current slot model.
2. **Runtime cutover.** Graph traversal replaces slot selection; re-author Glass
   Harbour as a graph; retire slots/chapters.
3. **Gates.** `det_gate` first (structured conditions), then `llm_gate`
   (constrained terminal-key output), wiring the cognitive-shim hint loop.

## Open questions

- **Cutscene params:** do custom components need authored parameters, or is the
  `component_key` plus its own assets enough for now? (Deferred; add typed params
  if a component needs them.)
- **det_gate conditions:** the initial condition vocabulary
  (`all_concepts_classified`, `min_claims`, `claim_exists`, …) — lock when we
  build Phase 3.
- **Node input ports:** single implicit input assumed. Revisit only if a node ever
  needs to distinguish where it was entered from.