gupi-osint-board/docs/exhibit-data-model.md

# GUPI OSINT Board: canonical exhibit data model

Status: accepted design foundation; the core schema, template lifecycle, frontend registry, Event workflow, brief concepts, and Party subtypes are implemented. Interactive Party-to-Party relationships remain roadmap work.

## Vocabulary

- An **exhibit** is a durable investigation-domain object stored in PostgreSQL.
- An **exhibit type** describes its domain behavior: folder, document, clipping, note, event, party, or conclusion.
- A **widget** is the frontend visualization and interaction implementation selected for an exhibit type.
- A **document type** specializes a document exhibit: image, PDF, web capture, email, article, filing, price list, text, or generic file.
- A **board** is a neutral container for exhibits. Both mutable levels and immutable template versions own boards.
- A **level** is a mutable board copy used for either play or authoring.
- A **level template version** is an immutable board snapshot.

The frontend registry maps `exhibit_type` (and, for documents, `document_type`) to a widget component. Changing a widget must never change the meaning or storage of its exhibit.

## Lifecycle

Templates are never edited in place.

1. A template family has one or more immutable versions.
2. Starting work creates a new level and clones the selected template-version board into it.
3. Every cloned exhibit receives a new ID, belongs to the new level board, and retains provenance through `origin_exhibit_id`.
4. Playing and editing write to exactly the same level and exhibit tables.
5. “Save as template” clones the current level board into a new immutable template version.
6. Binary assets are immutable and shared; document exhibits are copied, asset bytes are not.

No long-term `playthrough_*` overlay is required. A level is already the user's isolated working copy.

## Core structure

```sql
CREATE TABLE osint.boards (
  id UUID PRIMARY KEY,
  board_kind TEXT NOT NULL CHECK (board_kind IN ('level', 'template_version')),
  revision BIGINT NOT NULL DEFAULT 0,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE TABLE osint.level_templates (
  id UUID PRIMARY KEY,
  slug TEXT NOT NULL UNIQUE,
  name TEXT NOT NULL,
  current_version_id UUID,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE TABLE osint.level_template_versions (
  id UUID PRIMARY KEY,
  template_id UUID NOT NULL REFERENCES osint.level_templates(id),
  version INTEGER NOT NULL,
  board_id UUID NOT NULL UNIQUE REFERENCES osint.boards(id),
  title TEXT NOT NULL,
  subtitle TEXT NOT NULL DEFAULT '',
  created_from_level_id UUID,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  UNIQUE (template_id, version)
);

CREATE TABLE osint.levels (
  id UUID PRIMARY KEY,
  slug TEXT NOT NULL UNIQUE,
  board_id UUID NOT NULL UNIQUE REFERENCES osint.boards(id),
  source_template_version_id UUID REFERENCES osint.level_template_versions(id),
  title TEXT NOT NULL,
  subtitle TEXT NOT NULL DEFAULT '',
  status TEXT NOT NULL,
  viewport_x DOUBLE PRECISION NOT NULL DEFAULT 0,
  viewport_y DOUBLE PRECISION NOT NULL DEFAULT 28,
  viewport_zoom DOUBLE PRECISION NOT NULL DEFAULT 0.7,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```

`boards.revision` is incremented transactionally and used for optimistic concurrency control.

## Exhibits

Common spatial and lifecycle properties belong to the base exhibit, not to its widget or subtype.

```sql
CREATE TABLE osint.exhibit_types (
  id TEXT PRIMARY KEY,
  name TEXT NOT NULL UNIQUE,
  is_spatial BOOLEAN NOT NULL DEFAULT TRUE
);

CREATE TABLE osint.exhibits (
  id UUID PRIMARY KEY,
  board_id UUID NOT NULL REFERENCES osint.boards(id) ON DELETE CASCADE,
  exhibit_type_id TEXT NOT NULL REFERENCES osint.exhibit_types(id),
  origin_exhibit_id UUID REFERENCES osint.exhibits(id) ON DELETE SET NULL,
  xpos DOUBLE PRECISION NOT NULL,
  ypos DOUBLE PRECISION NOT NULL,
  width DOUBLE PRECISION NOT NULL,
  height DOUBLE PRECISION NOT NULL,
  rotation DOUBLE PRECISION NOT NULL DEFAULT 0,
  z_index INTEGER NOT NULL DEFAULT 0,
  hidden BOOLEAN NOT NULL DEFAULT FALSE,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  UNIQUE (board_id, id)
);
```

The common table is extended with one-to-one subtype tables. Domain fields do not live in an unstructured `config` document.

```sql
CREATE TABLE osint.folder_exhibits (
  exhibit_id UUID PRIMARY KEY REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  title TEXT NOT NULL,
  label_text TEXT NOT NULL DEFAULT '',
  is_open BOOLEAN NOT NULL DEFAULT FALSE
);

CREATE TABLE osint.document_types (
  id TEXT PRIMARY KEY,
  name TEXT NOT NULL UNIQUE
);

CREATE TABLE osint.document_exhibits (
  exhibit_id UUID PRIMARY KEY REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  document_type_id TEXT NOT NULL REFERENCES osint.document_types(id),
  asset_id UUID REFERENCES osint.assets(id),
  title TEXT NOT NULL,
  published_at TIMESTAMPTZ,
  captured_at TIMESTAMPTZ,
  source_uri TEXT
);

CREATE TABLE osint.image_documents (
  exhibit_id UUID PRIMARY KEY REFERENCES osint.document_exhibits(exhibit_id) ON DELETE CASCADE,
  pixel_width INTEGER CHECK (pixel_width > 0),
  pixel_height INTEGER CHECK (pixel_height > 0),
  alt_text TEXT NOT NULL DEFAULT ''
);

CREATE TABLE osint.note_exhibits (
  exhibit_id UUID PRIMARY KEY REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  title TEXT NOT NULL,
  note_text TEXT NOT NULL
);

CREATE TABLE osint.event_exhibits (
  exhibit_id UUID PRIMARY KEY REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  title TEXT NOT NULL,
  narrative_text TEXT NOT NULL,
  occurred_at TIMESTAMPTZ NOT NULL
);
```

The service validates that every base exhibit has exactly one subtype row matching `exhibit_type_id`.

### Event semantics

An event is an investigator-authored assertion: **“this happened.”** It is not source evidence and must not silently inherit a document's publication time.

- `narrative_text` states what the investigator believes happened.
- `occurred_at` places that assertion in reconstructed time.
- One event may cite several supporting exhibits.
- One exhibit may support several events.
- Events ordered by `occurred_at` form the emerging case narrative; there is no duplicated story-text record.

Supporting evidence is an explicit normalized relationship:

```sql
CREATE TABLE osint.event_evidence (
  event_exhibit_id UUID NOT NULL REFERENCES osint.event_exhibits(exhibit_id) ON DELETE CASCADE,
  evidence_exhibit_id UUID NOT NULL REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  sort_order INTEGER NOT NULL DEFAULT 0,
  note TEXT,
  PRIMARY KEY (event_exhibit_id, evidence_exhibit_id),
  CHECK (event_exhibit_id <> evidence_exhibit_id)
);
```

A deferred constraint trigger verifies that both exhibits belong to the same board and that the evidence endpoint is not itself the same event. The optional `note` explains why that exhibit supports the event; it does not replace the event's narrative.

In the frontend, the Event widget shows its occurrence time, narrative text, and evidence count. Opening or selecting it reveals its supporting exhibits. Lines between an event and its evidence visualize `event_evidence`; they are not ordinary folder containment bands.

### Party and brief-concept semantics

A party is a person or organization that participates in the investigation. Businesses are organizations. A party is a first-class exhibit, not a special folder: identity and evidence association must not be represented as file ownership.

- A **person** has a display name and may have aliases and other structured identity fields.
- An **organization** has a display name and an organization kind such as business, public body, association, or informal group.
- A party may be linked to several supporting exhibits, and one exhibit may concern several parties.
- Relationships between parties—employment, ownership, membership, control, representation, or an investigator-defined connection—are explicit typed relationships rather than containment.

The intended normalized shape is a `party_exhibits` supertype with one-to-one `person_parties` and `organization_parties` subtype tables, plus a many-to-many `party_evidence` association. Aliases and party-to-party relationships use child tables rather than arrays or widget configuration JSON.

The frontend provides distinct Person and Organization widgets through the exhibit registry. They may visually behave like dossiers—opening one can reveal associated evidence—but that interaction is derived from `party_evidence`; it does not turn the party into a folder or cause evidence to be owned by or disappear into the party.

A name appearing in the level brief begins as a normalized `brief_concept`, not as an exhibit. The author may record the expected Party classification, which is omitted from play-mode API responses. When the investigator classifies a concept as Person or Organization, the level creates a Party exhibit and records it in `resolved_party_exhibit_id`. This keeps the reasoning action explicit: the game does not pre-create a correctly typed party and merely hide its widget.

## Assets and document content

`assets` stores immutable uploaded bytes, checksum, MIME type, original filename, and size. Multiple cloned document exhibits may reference one asset.

Document text and extractable regions are normalized separately:

```sql
CREATE TABLE osint.document_content_blocks (
  id UUID PRIMARY KEY,
  document_exhibit_id UUID NOT NULL REFERENCES osint.document_exhibits(exhibit_id) ON DELETE CASCADE,
  sort_order INTEGER NOT NULL,
  content TEXT NOT NULL,
  UNIQUE (document_exhibit_id, sort_order)
);

CREATE TABLE osint.document_regions (
  id UUID PRIMARY KEY,
  document_exhibit_id UUID NOT NULL REFERENCES osint.document_exhibits(exhibit_id) ON DELETE CASCADE,
  region_key TEXT NOT NULL,
  label TEXT NOT NULL,
  excerpt TEXT NOT NULL,
  occurred_at TIMESTAMPTZ,
  sort_order INTEGER NOT NULL DEFAULT 0,
  UNIQUE (document_exhibit_id, region_key)
);
```

## Folder ownership

Folder containment is a domain relationship, not a generic visual connection.

```sql
CREATE TABLE osint.folder_memberships (
  board_id UUID NOT NULL REFERENCES osint.boards(id) ON DELETE CASCADE,
  folder_exhibit_id UUID NOT NULL REFERENCES osint.folder_exhibits(exhibit_id) ON DELETE CASCADE,
  child_exhibit_id UUID NOT NULL REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  sort_order INTEGER NOT NULL DEFAULT 0,
  PRIMARY KEY (folder_exhibit_id, child_exhibit_id),
  UNIQUE (board_id, child_exhibit_id),
  CHECK (folder_exhibit_id <> child_exhibit_id)
);
```

For the POC, one exhibit has at most one owning folder. This makes open/close behavior deterministic. If the same uploaded image must appear in two folders, two document exhibits reference the same immutable asset.

Composite foreign keys or deferred constraint triggers enforce that folder, child, and membership all belong to the same board and that folders cannot contain themselves or form containment cycles.

The pale red folder band is derived from this table. Its geometry is not stored.

## Investigative connections and provenance

```sql
CREATE TABLE osint.connection_types (
  id TEXT PRIMARY KEY,
  name TEXT NOT NULL,
  directed BOOLEAN NOT NULL DEFAULT FALSE
);

CREATE TABLE osint.exhibit_connections (
  id UUID PRIMARY KEY,
  board_id UUID NOT NULL REFERENCES osint.boards(id) ON DELETE CASCADE,
  connection_type_id TEXT NOT NULL REFERENCES osint.connection_types(id),
  from_exhibit_id UUID NOT NULL REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  to_exhibit_id UUID NOT NULL REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  label TEXT,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  CHECK (from_exhibit_id <> to_exhibit_id)
);

CREATE TABLE osint.exhibit_sources (
  exhibit_id UUID PRIMARY KEY REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  source_document_exhibit_id UUID NOT NULL REFERENCES osint.document_exhibits(exhibit_id),
  source_region_id UUID REFERENCES osint.document_regions(id),
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```

Red investigative thread is derived from `exhibit_connections`. Extraction provenance is stored in `exhibit_sources`; it is not represented as a visual thread.

## Document metadata

Known, semantically important values remain real columns: `published_at`, `captured_at`, and `source_uri`. Truly author-defined fields use typed metadata definitions and values rather than JSONB or one untyped EAV table.

```sql
CREATE TABLE osint.metadata_fields (
  id UUID PRIMARY KEY,
  key TEXT NOT NULL UNIQUE,
  label TEXT NOT NULL,
  value_type TEXT NOT NULL CHECK (value_type IN ('text', 'timestamp', 'number', 'boolean'))
);

CREATE TABLE osint.document_metadata_text_values (
  document_exhibit_id UUID NOT NULL REFERENCES osint.document_exhibits(exhibit_id) ON DELETE CASCADE,
  field_id UUID NOT NULL REFERENCES osint.metadata_fields(id),
  value TEXT NOT NULL,
  PRIMARY KEY (document_exhibit_id, field_id)
);
```

Timestamp, number, and boolean values use equivalent type-specific tables. This lets PostgreSQL enforce actual value types and avoids nullable multi-type value columns.

## Derived presentation

The following are computed and must not become duplicate source-of-truth tables:

- Grey timeline projection: document exhibit position to `published_at` on the timeline.
- Pale red containment band: folder position to contained exhibit position.
- Folder flight animation: closed folder position to the exhibit's stored `xpos` and `ypos`.
- Widget selection: `exhibit_type` plus optional `document_type` mapped through the frontend registry.

## Transactional operations

`instantiate_template_version(source_version_id)` performs one database transaction:

1. Create the level board and level row.
2. Clone all exhibits and build an old-to-new ID map.
3. Clone each subtype row.
4. Clone content, typed metadata, and provenance through the map.
5. Clone folder memberships, event-evidence links, and exhibit connections through the map.
6. Reuse asset IDs.
7. Commit only after all constraints pass.

`save_level_as_template(level_id)` runs the same clone operation into a new immutable template-version board.

## Implemented cutover

Migration 006 made this the sole persistence model. Because the POC database contained no canonical or legacy content worth preserving, the cutover intentionally dropped the JSON case store, `widgets`, `widget_relations`, and `playthrough_*` tables without a backfill period. Template save, version selection, instantiation, and reset use one transactional board-cloning service. Exhibit and document types resolve through the typed frontend registry; the remaining model work is the planned exhibit families and richer behavior.