gupi-osint-board/docs/exhibit-data-model.md

GUPI OSINT Board: canonical exhibit data model

Status: accepted design foundation.

Vocabulary

• An exhibit is a durable investigation-domain object stored in PostgreSQL.
• An exhibit type describes its domain behavior: folder, document, clipping, note, event, party, or conclusion.
• A widget is the frontend visualization and interaction implementation selected for an exhibit type.
• A document type specializes a document exhibit: image, PDF, web capture, email, article, filing, price list, text, or generic file.
• A board is a neutral container for exhibits. Both mutable levels and immutable template versions own boards.
• A level is a mutable board copy used for either play or authoring.
• A level template version is an immutable board snapshot.

The frontend registry maps exhibit_type (and, for documents, document_type) to a widget component. Changing a widget must never change the meaning or storage of its exhibit.

Lifecycle

Templates are never edited in place.

1. A template family has one or more immutable versions.
2. Starting work creates a new level and clones the selected template-version board into it.
3. Every cloned exhibit receives a new ID, belongs to the new level board, and retains provenance through origin_exhibit_id.
4. Playing and editing write to exactly the same level and exhibit tables.
5. “Save as template” clones the current level board into a new immutable template version.
6. Binary assets are immutable and shared; document exhibits are copied, asset bytes are not.

No long-term playthrough_* overlay is required. A level is already the user's isolated working copy.

Core structure

CREATE TABLE osint.boards (
  id UUID PRIMARY KEY,
  board_kind TEXT NOT NULL CHECK (board_kind IN ('level', 'template_version')),
  revision BIGINT NOT NULL DEFAULT 0,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE TABLE osint.level_templates (
  id UUID PRIMARY KEY,
  slug TEXT NOT NULL UNIQUE,
  name TEXT NOT NULL,
  current_version_id UUID,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE TABLE osint.level_template_versions (
  id UUID PRIMARY KEY,
  template_id UUID NOT NULL REFERENCES osint.level_templates(id),
  version INTEGER NOT NULL,
  board_id UUID NOT NULL UNIQUE REFERENCES osint.boards(id),
  title TEXT NOT NULL,
  subtitle TEXT NOT NULL DEFAULT '',
  created_from_level_id UUID,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  UNIQUE (template_id, version)
);

CREATE TABLE osint.levels (
  id UUID PRIMARY KEY,
  slug TEXT NOT NULL UNIQUE,
  board_id UUID NOT NULL UNIQUE REFERENCES osint.boards(id),
  source_template_version_id UUID REFERENCES osint.level_template_versions(id),
  title TEXT NOT NULL,
  subtitle TEXT NOT NULL DEFAULT '',
  status TEXT NOT NULL,
  viewport_x DOUBLE PRECISION NOT NULL DEFAULT 0,
  viewport_y DOUBLE PRECISION NOT NULL DEFAULT 28,
  viewport_zoom DOUBLE PRECISION NOT NULL DEFAULT 0.7,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

boards.revision is incremented transactionally and used for optimistic concurrency control.

Exhibits

Common spatial and lifecycle properties belong to the base exhibit, not to its widget or subtype.

CREATE TABLE osint.exhibit_types (
  id TEXT PRIMARY KEY,
  name TEXT NOT NULL UNIQUE,
  is_spatial BOOLEAN NOT NULL DEFAULT TRUE
);

CREATE TABLE osint.exhibits (
  id UUID PRIMARY KEY,
  board_id UUID NOT NULL REFERENCES osint.boards(id) ON DELETE CASCADE,
  exhibit_type_id TEXT NOT NULL REFERENCES osint.exhibit_types(id),
  origin_exhibit_id UUID REFERENCES osint.exhibits(id) ON DELETE SET NULL,
  xpos DOUBLE PRECISION NOT NULL,
  ypos DOUBLE PRECISION NOT NULL,
  width DOUBLE PRECISION NOT NULL,
  height DOUBLE PRECISION NOT NULL,
  rotation DOUBLE PRECISION NOT NULL DEFAULT 0,
  z_index INTEGER NOT NULL DEFAULT 0,
  hidden BOOLEAN NOT NULL DEFAULT FALSE,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  UNIQUE (board_id, id)
);

The common table is extended with one-to-one subtype tables. Domain fields do not live in an unstructured config document.

CREATE TABLE osint.folder_exhibits (
  exhibit_id UUID PRIMARY KEY REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  title TEXT NOT NULL,
  label_text TEXT NOT NULL DEFAULT '',
  is_open BOOLEAN NOT NULL DEFAULT FALSE
);

CREATE TABLE osint.document_types (
  id TEXT PRIMARY KEY,
  name TEXT NOT NULL UNIQUE
);

CREATE TABLE osint.document_exhibits (
  exhibit_id UUID PRIMARY KEY REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  document_type_id TEXT NOT NULL REFERENCES osint.document_types(id),
  asset_id UUID REFERENCES osint.assets(id),
  title TEXT NOT NULL,
  published_at TIMESTAMPTZ,
  captured_at TIMESTAMPTZ,
  source_uri TEXT
);

CREATE TABLE osint.image_documents (
  exhibit_id UUID PRIMARY KEY REFERENCES osint.document_exhibits(exhibit_id) ON DELETE CASCADE,
  pixel_width INTEGER CHECK (pixel_width > 0),
  pixel_height INTEGER CHECK (pixel_height > 0),
  alt_text TEXT NOT NULL DEFAULT ''
);

CREATE TABLE osint.note_exhibits (
  exhibit_id UUID PRIMARY KEY REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  title TEXT NOT NULL,
  note_text TEXT NOT NULL
);

CREATE TABLE osint.event_exhibits (
  exhibit_id UUID PRIMARY KEY REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  title TEXT NOT NULL,
  narrative_text TEXT NOT NULL,
  occurred_at TIMESTAMPTZ NOT NULL
);

The service validates that every base exhibit has exactly one subtype row matching exhibit_type_id.

Event semantics

An event is an investigator-authored assertion: “this happened.” It is not source evidence and must not silently inherit a document's publication time.

• narrative_text states what the investigator believes happened.
• occurred_at places that assertion in reconstructed time.
• One event may cite several supporting exhibits.
• One exhibit may support several events.
• Events ordered by occurred_at form the emerging case narrative; there is no duplicated story-text record.

Supporting evidence is an explicit normalized relationship:

CREATE TABLE osint.event_evidence (
  event_exhibit_id UUID NOT NULL REFERENCES osint.event_exhibits(exhibit_id) ON DELETE CASCADE,
  evidence_exhibit_id UUID NOT NULL REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  sort_order INTEGER NOT NULL DEFAULT 0,
  note TEXT,
  PRIMARY KEY (event_exhibit_id, evidence_exhibit_id),
  CHECK (event_exhibit_id <> evidence_exhibit_id)
);

A deferred constraint trigger verifies that both exhibits belong to the same board and that the evidence endpoint is not itself the same event. The optional note explains why that exhibit supports the event; it does not replace the event's narrative.

In the frontend, the Event widget shows its occurrence time, narrative text, and evidence count. Opening or selecting it reveals its supporting exhibits. Lines between an event and its evidence visualize event_evidence; they are not ordinary folder containment bands.

Party semantics (planned)

A party is a person or organization that participates in the investigation. Businesses are organizations. A party is a first-class exhibit, not a special folder: identity and evidence association must not be represented as file ownership.

• A person has a display name and may have aliases and other structured identity fields.
• An organization has a display name and an organization kind such as business, public body, association, or informal group.
• A party may be linked to several supporting exhibits, and one exhibit may concern several parties.
• Relationships between parties—employment, ownership, membership, control, representation, or an investigator-defined connection—are explicit typed relationships rather than containment.

The intended normalized shape is a party_exhibits supertype with one-to-one person_parties and organization_parties subtype tables, plus a many-to-many party_evidence association. Aliases and party-to-party relationships use child tables rather than arrays or widget configuration JSON.

The frontend provides distinct Person and Organization widgets through the exhibit registry. They may visually behave like dossiers—opening one can reveal associated evidence—but that interaction is derived from party_evidence; it does not turn the party into a folder or cause evidence to be owned by or disappear into the party.

Assets and document content

assets stores immutable uploaded bytes, checksum, MIME type, original filename, and size. Multiple cloned document exhibits may reference one asset.

Document text and extractable regions are normalized separately:

CREATE TABLE osint.document_content_blocks (
  id UUID PRIMARY KEY,
  document_exhibit_id UUID NOT NULL REFERENCES osint.document_exhibits(exhibit_id) ON DELETE CASCADE,
  sort_order INTEGER NOT NULL,
  content TEXT NOT NULL,
  UNIQUE (document_exhibit_id, sort_order)
);

CREATE TABLE osint.document_regions (
  id UUID PRIMARY KEY,
  document_exhibit_id UUID NOT NULL REFERENCES osint.document_exhibits(exhibit_id) ON DELETE CASCADE,
  region_key TEXT NOT NULL,
  label TEXT NOT NULL,
  excerpt TEXT NOT NULL,
  occurred_at TIMESTAMPTZ,
  sort_order INTEGER NOT NULL DEFAULT 0,
  UNIQUE (document_exhibit_id, region_key)
);

Folder ownership

Folder containment is a domain relationship, not a generic visual connection.

CREATE TABLE osint.folder_memberships (
  board_id UUID NOT NULL REFERENCES osint.boards(id) ON DELETE CASCADE,
  folder_exhibit_id UUID NOT NULL REFERENCES osint.folder_exhibits(exhibit_id) ON DELETE CASCADE,
  child_exhibit_id UUID NOT NULL REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  sort_order INTEGER NOT NULL DEFAULT 0,
  PRIMARY KEY (folder_exhibit_id, child_exhibit_id),
  UNIQUE (board_id, child_exhibit_id),
  CHECK (folder_exhibit_id <> child_exhibit_id)
);

For the POC, one exhibit has at most one owning folder. This makes open/close behavior deterministic. If the same uploaded image must appear in two folders, two document exhibits reference the same immutable asset.

Composite foreign keys or deferred constraint triggers enforce that folder, child, and membership all belong to the same board and that folders cannot contain themselves or form containment cycles.

The pale red folder band is derived from this table. Its geometry is not stored.

Investigative connections and provenance

CREATE TABLE osint.connection_types (
  id TEXT PRIMARY KEY,
  name TEXT NOT NULL,
  directed BOOLEAN NOT NULL DEFAULT FALSE
);

CREATE TABLE osint.exhibit_connections (
  id UUID PRIMARY KEY,
  board_id UUID NOT NULL REFERENCES osint.boards(id) ON DELETE CASCADE,
  connection_type_id TEXT NOT NULL REFERENCES osint.connection_types(id),
  from_exhibit_id UUID NOT NULL REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  to_exhibit_id UUID NOT NULL REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  label TEXT,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  CHECK (from_exhibit_id <> to_exhibit_id)
);

CREATE TABLE osint.exhibit_sources (
  exhibit_id UUID PRIMARY KEY REFERENCES osint.exhibits(id) ON DELETE CASCADE,
  source_document_exhibit_id UUID NOT NULL REFERENCES osint.document_exhibits(exhibit_id),
  source_region_id UUID REFERENCES osint.document_regions(id),
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

Red investigative thread is derived from exhibit_connections. Extraction provenance is stored in exhibit_sources; it is not represented as a visual thread.

Document metadata

Known, semantically important values remain real columns: published_at, captured_at, and source_uri. Truly author-defined fields use typed metadata definitions and values rather than JSONB or one untyped EAV table.

CREATE TABLE osint.metadata_fields (
  id UUID PRIMARY KEY,
  key TEXT NOT NULL UNIQUE,
  label TEXT NOT NULL,
  value_type TEXT NOT NULL CHECK (value_type IN ('text', 'timestamp', 'number', 'boolean'))
);

CREATE TABLE osint.document_metadata_text_values (
  document_exhibit_id UUID NOT NULL REFERENCES osint.document_exhibits(exhibit_id) ON DELETE CASCADE,
  field_id UUID NOT NULL REFERENCES osint.metadata_fields(id),
  value TEXT NOT NULL,
  PRIMARY KEY (document_exhibit_id, field_id)
);

Timestamp, number, and boolean values use equivalent type-specific tables. This lets PostgreSQL enforce actual value types and avoids nullable multi-type value columns.

Derived presentation

The following are computed and must not become duplicate source-of-truth tables:

• Grey timeline projection: document exhibit position to published_at on the timeline.
• Pale red containment band: folder position to contained exhibit position.
• Folder flight animation: closed folder position to the exhibit's stored xpos and ypos.
• Widget selection: exhibit_type plus optional document_type mapped through the frontend registry.

Transactional operations

instantiate_template_version(source_version_id) performs one database transaction:

1. Create the level board and level row.
2. Clone all exhibits and build an old-to-new ID map.
3. Clone each subtype row.
4. Clone content, typed metadata, and provenance through the map.
5. Clone folder memberships, event-evidence links, and exhibit connections through the map.
6. Reuse asset IDs.
7. Commit only after all constraints pass.

save_level_as_template(level_id) runs the same clone operation into a new immutable template-version board.

Migration direction

The current widgets, widget_relations, and playthrough_* tables are transitional. The cutover should:

1. Introduce boards, exhibit tables, subtype tables, templates, and typed metadata.
2. Convert existing authored widgets to exhibits.
3. Materialize each existing playthrough as its own cloned mutable level.
4. Move document layout from relation JSON into document exhibit coordinates.
5. Convert contains relations to folder memberships and generic connections to exhibit connections.
6. Update the API to read and write exhibits directly.
7. Remove the transitional widget/playthrough tables only after data equivalence checks pass.