Implement Scene 7 evidence goal flow
This commit is contained in:
@@ -0,0 +1,264 @@
|
||||
# Scene 7: prove Barricelli was an inventor
|
||||
|
||||
Status: **implemented and verified** on `scene-7-evidence-goal`
|
||||
Demo: **GUPI Demo 1 — The Barricelli Files**
|
||||
|
||||
## Outcome
|
||||
|
||||
Scene 7 teaches one idea: a screenshot found during an OSINT search can become
|
||||
source evidence.
|
||||
|
||||
The player opens a minimal OSINT board, reads **“Demonstrate OSINT skill: prove
|
||||
Nils Aall Barricelli was an inventor,”** finds the relevant Google Patents result,
|
||||
and pastes or uploads one screenshot. The board creates a document, extracts its
|
||||
text, recognizes the source, and clears the level. A URL and written report are
|
||||
not required.
|
||||
|
||||
```text
|
||||
paste screenshot -> document appears -> text is read -> source is verified
|
||||
-> Scene 7 complete -> continue to Scene 8
|
||||
```
|
||||
|
||||
## Product decisions
|
||||
|
||||
- One suitable Google Patents screenshot is sufficient.
|
||||
- Clipboard paste and file upload use the same server path.
|
||||
- The image and OCR remain a real source document on the player's board.
|
||||
- A known-source fuzzy match is the fast, deterministic victory path.
|
||||
- A small semantic judge is a fallback for other credible sources and for
|
||||
distinguishing Nils from his father. It cannot overrule the trusted path.
|
||||
- Evidence about Barricelli's father may award an optional discovery but does not
|
||||
clear the assignment unless it also supports the claim about Nils.
|
||||
- The boarding-house fire belongs to the later age/rescue assignment, not Scene 7.
|
||||
- Scene 7 records `scene7.nils_inventor_proved`. Scene 8 owns the ceremony and
|
||||
awards `barricelli_luggage`.
|
||||
- Inconclusive evaluation never deletes or penalizes uploaded evidence.
|
||||
|
||||
## Shared identifiers
|
||||
|
||||
| Purpose | Key |
|
||||
|---|---|
|
||||
| Goal | `barricelli.inventor-proof` |
|
||||
| Scene 7 completion | `scene7.nils_inventor_proved` |
|
||||
| Optional father discovery | `scene7.father_inventor_discovered` |
|
||||
| Scene 8 reward | `barricelli_luggage` |
|
||||
|
||||
All identifiers and content are template data. There must be no Barricelli
|
||||
conditional in React or server business logic.
|
||||
|
||||
## Existing foundation
|
||||
|
||||
- `025_level_document_flags.sql`: clonable document gates and level flags.
|
||||
- `026_achievements.sql`: playthrough achievements.
|
||||
- `027_evidence_text_matching.sql`: asset OCR, board match rules/anchors,
|
||||
level-owned evaluations, and flag provenance.
|
||||
- `028_level_goals.sql`: board-owned goals and normalized flag requirements.
|
||||
- `029_semantic_evidence_judging.sql`: clonable semantic rules, level-owned
|
||||
evaluations, and semantic flag provenance.
|
||||
- `030_evidence_match_source_metadata.sql`: author-only canonical source metadata.
|
||||
- `server/ocr.ts`: plain-text extraction and Tesseract.
|
||||
- `server/evidenceMatching.ts`: OCR-tolerant fuzzy passage matching.
|
||||
- `POST /api/levels/:id/documents`: persistent upload plus OCR and matching.
|
||||
- The story runtime already tracks `current_node_id` and `current_level_id`.
|
||||
|
||||
## Architecture contract
|
||||
|
||||
### Recognition and completion are separate
|
||||
|
||||
Recognition answers what a document supports. A goal answers whether the level's
|
||||
authored requirements have been satisfied. `level_goals` and
|
||||
`level_goal_flag_requirements` clone with a template. Goal completion is derived
|
||||
from level flags; there is no second mutable completion boolean.
|
||||
|
||||
Play mode receives a goal's key, title, instructions, completion copy, status,
|
||||
and completion time. IDs, enabled state, required flags, target text, and judging
|
||||
prompts remain author-only.
|
||||
|
||||
### Known-source fast path
|
||||
|
||||
The Scene 7 template owns an `evidence_match_rule` with distinctive text visible
|
||||
in the real Google Patents result: a combination of patent number/title, inventor
|
||||
name, and invention language. A name alone is too generic. When enough anchors
|
||||
match, the existing matcher awards `scene7.nils_inventor_proved` in the upload
|
||||
transaction and the goal becomes complete immediately.
|
||||
|
||||
Reference OCR, thresholds, source metadata, and copy live in the manifest/database,
|
||||
not TypeScript constants. The expected text is never returned to play mode.
|
||||
|
||||
### Semantic fallback
|
||||
|
||||
A provider-neutral `EvidenceJudge` receives only allowlisted goal data and OCR
|
||||
text. It returns strictly validated structured data:
|
||||
|
||||
```ts
|
||||
type EvidenceVerdict = {
|
||||
subject: 'target' | 'related' | 'ambiguous' | 'neither'
|
||||
supportsClaim: boolean
|
||||
evidenceExcerpt: string
|
||||
confidence: number
|
||||
}
|
||||
```
|
||||
|
||||
OCR is untrusted quoted material. The judge prompt explicitly ignores instructions
|
||||
inside it. Provider/model, timeout, input limit, and confidence threshold are
|
||||
environment configuration. Semantic configuration clones with the board;
|
||||
evaluation history belongs to the level/document/extraction and records evaluator
|
||||
version, provider/model, verdict, excerpt, confidence, timestamps, and sanitized
|
||||
failure state.
|
||||
|
||||
| Verdict | Mutation |
|
||||
|---|---|
|
||||
| Target + assertion supported at threshold | authored success flag (`scene7.nils_inventor_proved`) |
|
||||
| Related subject only + assertion supported | authored related flag (`scene7.father_inventor_discovered`) |
|
||||
| Ambiguous, unsupported, or below threshold | none |
|
||||
| Provider failure | none; retryable |
|
||||
|
||||
The known-source pass avoids an LLM call. After a deterministic miss, the client
|
||||
automatically calls an idempotent semantic endpoint. This keeps document upload
|
||||
durable even if an external provider times out, without requiring a demo job queue.
|
||||
|
||||
### Story progression
|
||||
|
||||
Advancing from a level is server-authoritative. The server verifies the JWT user,
|
||||
active playthrough/current level, and enabled goal state. It promotes the Scene 7
|
||||
completion fact idempotently and only then follows the level terminal. The browser
|
||||
cannot grant its own achievement.
|
||||
|
||||
The player must see the verification result before navigation. Success exposes a
|
||||
single Continue action to Scene 8. Scene 6 only wires into the Scene 7 level node;
|
||||
Scene 8 owns the luggage reward.
|
||||
|
||||
## Work packages
|
||||
|
||||
### S7-A — Goal model and cloning
|
||||
|
||||
- [x] Add normalized board goals and flag requirements in migration `028`.
|
||||
- [x] Clone goals during freeze, instantiation, and reset.
|
||||
- [x] Derive pending/complete state from level flags.
|
||||
- [x] Hide goal requirements from play payloads.
|
||||
- [x] Return newly completed goals from document upload.
|
||||
- [x] Add admin CRUD repository/API contracts.
|
||||
- [x] Test migration, cloning, deterministic completion, and isolation.
|
||||
- [x] Clear level flags/reveal state when resetting from a template.
|
||||
|
||||
### S7-B — Barricelli content and trusted recognition
|
||||
|
||||
- [x] Add goals and evidence-match rules to the normal mystery manifest importer.
|
||||
- [x] Create the Scene 7 template with its assignment and no solution-bearing
|
||||
starting document.
|
||||
- [x] Add a reproducible screenshot-paste acceptance path that runs through local OCR.
|
||||
- [x] Author three distinctive patent anchors.
|
||||
- [x] Tune against the target and unrelated/father-only negative fixtures.
|
||||
- [x] Award `scene7.nils_inventor_proved` and require it for the level goal.
|
||||
- [x] Store canonical source metadata for administrators; keep player URL optional.
|
||||
|
||||
### S7-C — Semantic judge
|
||||
|
||||
- [x] Add provider-neutral interface and strict verdict validation.
|
||||
- [x] Add clonable semantic rule configuration and level-owned evaluations.
|
||||
- [x] Add semantic evaluation provenance to level flags.
|
||||
- [x] Add provider/model/timeout/input/confidence environment configuration.
|
||||
- [x] Send OCR text rather than raw image bytes.
|
||||
- [x] Add ownership-checked, idempotent judge endpoint.
|
||||
- [x] Route target and father verdicts to their authored flags.
|
||||
- [x] Make disabled provider, timeout, quota, and malformed output safe/retryable.
|
||||
- [x] Do not log evidence text or secrets.
|
||||
|
||||
### S7-D — Story bridge
|
||||
|
||||
- [x] Require completed enabled goals before a playthrough can leave a level.
|
||||
- [x] Verify the current playthrough belongs to the JWT user and owns the level.
|
||||
- [x] Promote required completion facts exactly once.
|
||||
- [x] Return a useful pending-goal error rather than advancing early.
|
||||
- [x] Return to the generic story runtime after success; the Scene 6/8 graph owner
|
||||
wires the actual neighboring nodes.
|
||||
- [x] Keep the arbitrary achievement-grant route development-only.
|
||||
|
||||
### S7-E — Board experience
|
||||
|
||||
- [x] Present the active goal prominently on Scene 7.
|
||||
- [x] Preserve clipboard paste, drag/drop, and file picker equivalence.
|
||||
- [x] Show source import/OCR and semantic checking stages.
|
||||
- [x] Highlight the accepted document and show **SOURCE VERIFIED — NILS AALL
|
||||
BARRICELLI: INVENTOR**.
|
||||
- [x] Show Continue only after server-confirmed completion.
|
||||
- [x] Acknowledge father-only discovery while asking for evidence about Nils.
|
||||
- [x] Keep inconclusive evidence and offer neutral guidance.
|
||||
- [x] Respect reduced-motion and mobile layouts.
|
||||
|
||||
### S7-F — Verification
|
||||
|
||||
- [x] Add a legally safe derived OCR fixture.
|
||||
- [x] Test line breaks, punctuation, cropping, name hyphenation, unrelated patent,
|
||||
father-only text, empty OCR, and prompt-injection-like text.
|
||||
- [x] Contract-test with a fake semantic provider; CI never calls a paid model.
|
||||
- [x] Integration-test target upload -> document -> flag -> completed goal.
|
||||
- [x] Test duplicate evaluation, provider failure/retry, two-user isolation, and
|
||||
story progression before/after completion.
|
||||
- [x] Browser-test clipboard image paste through OCR and verification. Continue
|
||||
is covered by the generic story-gate integration until the Scene 8 node lands.
|
||||
- [x] Run migrations on an empty database and one currently at `027`.
|
||||
- [x] Run unit, integration, build, and Docker smoke tests.
|
||||
|
||||
## API shape
|
||||
|
||||
```ts
|
||||
type LevelGoal = {
|
||||
key: string
|
||||
title: string
|
||||
instructions: string
|
||||
completionMessage: string
|
||||
status: 'pending' | 'complete'
|
||||
completedAt?: string
|
||||
newlyCompleted: boolean
|
||||
}
|
||||
|
||||
type DocumentAnalysis = {
|
||||
extractionStatus: 'succeeded' | 'unsupported' | 'failed'
|
||||
matchedFlags: string[]
|
||||
awardedFlags: string[]
|
||||
goals: LevelGoal[]
|
||||
}
|
||||
```
|
||||
|
||||
`newlyCompleted` is response-local: a reload returns complete with
|
||||
`newlyCompleted: false`.
|
||||
|
||||
## Security and cost limits
|
||||
|
||||
- Require identity and level ownership on player mutations.
|
||||
- Limit upload bytes, OCR/model characters, output tokens, duration, and retries.
|
||||
- Never expose reference anchors or judge instructions to play mode.
|
||||
- Treat filenames, MIME declarations, OCR, and model output as untrusted.
|
||||
- Validate model output and confidence before mutating flags.
|
||||
- Persist enough provenance to explain completion without storing unnecessary raw
|
||||
provider payloads.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Scenes 1–6, Scene 8's ceremony/3D model, and Scene 9's fire mystery.
|
||||
- Knowledge graph, Case Report, claims, red-thread reasoning, or multi-document
|
||||
synthesis.
|
||||
- Web crawling, URL fetching, or requiring a URL for victory.
|
||||
- Custom model training or the general story graph `llm_gate`.
|
||||
- Deleting irrelevant evidence.
|
||||
|
||||
## Merge guidance
|
||||
|
||||
Prefer new modules and narrow glue commits. High-conflict files are
|
||||
`server/index.ts`, `server/narrativeRepository.ts`, `src/App.tsx`, `src/main.tsx`,
|
||||
and `src/play.tsx`; one integrator should own their final changes.
|
||||
|
||||
Suggested order: S7-A schema -> S7-B deterministic content -> S7-C judge -> S7-D
|
||||
story bridge -> S7-E UI -> S7-F hardening. Reserve migration numbers before
|
||||
parallel schema work and never renumber an already-shared migration silently.
|
||||
|
||||
## Definition of done
|
||||
|
||||
From a fresh playthrough, the player reaches Scene 7 and pastes one accepted
|
||||
Google Patents screenshot. One source document appears; MinIO asset, OCR, match
|
||||
evaluation, and `scene7.nils_inventor_proved` are persisted; the goal completes;
|
||||
the UI explains what was proved and offers Continue; and the story enters Scene 8.
|
||||
Reload and duplicate evaluation are idempotent, another player's level is
|
||||
unaffected, no URL was required, and `barricelli_luggage` is not awarded until
|
||||
Scene 8.
|
||||
Reference in New Issue
Block a user