Adding fixes to the render system
This commit is contained in:
@@ -0,0 +1,165 @@
|
||||
# Atomic Events — Design Spec
|
||||
|
||||
Status: **Stage A + Stage B implemented (2026-07-27).** Motivated by a future **Glitch
|
||||
Studio GUI** that edits each video occurrence as a self-contained object.
|
||||
|
||||
Implemented:
|
||||
- Per-occurrence presentation resolves via `transformer.resolve_video_presentation`
|
||||
(precedence: inline/GUI override > shorthand prefix > videos.json > default; video
|
||||
`end_on` default = `next_video`, `[narration:]` runs to end).
|
||||
- events.json is materialized/atomic: `derive_events` writes `handle/cutout/layer/end_on/
|
||||
take`; `events_to_marker_timings` round-trips them as overrides.
|
||||
- Inline grammar `[prefix:handle, key=value, …]` (`parser.parse_marker`), threaded through
|
||||
alignment into `MarkerTiming.overrides`. Supported inline keys: **cutout, layer, end_on,
|
||||
take** (the fully-wired per-event fields). Unknown keys are ignored.
|
||||
- The key-reuse collision validator hard-error was removed (reuse is legal now).
|
||||
|
||||
Deferred (follow-ups): inline override of the *global* params (skip/zoom/volume/
|
||||
use_audio_channels/pause_narration) — the renderer reads these from `video_source` in ~13
|
||||
places, so wiring them per-event is a separate change; stripping the moved fields from
|
||||
videos.json (kept as fallback defaults for now); a validator warning for unknown/unwired
|
||||
inline keys.
|
||||
|
||||
## Problem
|
||||
|
||||
Presentation/timing properties (`cutout`, `layer`, `end_on`, `take`, `pause_narration`)
|
||||
live on the **videos.json handle**, but they are really properties of *where a clip is
|
||||
used*, not of the file. The shorthand prefix (`vst:` = square/above, `vsb:` =
|
||||
square/below) is per-marker, but `_project_markers_to_videos` collapses it onto the
|
||||
single handle record (last-wins). So one handle used two ways collides:
|
||||
|
||||
- `[vst:glitch_ccd_binning]` (above) and `[vsb:glitch_ccd_binning]` (below) → videos.json
|
||||
can only store `layer: below`, so the first occurrence renders under the slide (hidden).
|
||||
- video5 has 5 such collisions today (glitch_ccd_binning, pexels/12471039…,
|
||||
mainvideopart1, shotnoiseacc, slide_periodogram).
|
||||
|
||||
A stopgap validator hard-error (`validate_project`, gnommo/validator.py) currently blocks
|
||||
render on these. This spec removes the *cause* so that guard is no longer needed.
|
||||
|
||||
The naive fixes are both rejected: copying the file/handle (duplication on disk), and a
|
||||
"hybrid override + materialize" layer (too much indirection). Instead: **the per-occurrence
|
||||
properties move onto the event.**
|
||||
|
||||
## Field homes
|
||||
|
||||
**videos.json — asset + global defaults (one value per handle):**
|
||||
`source_file`, `output_file`/`processed_file`, `filter`, `has_audio`, `is_shared`,
|
||||
`src_mtime`, `duration` (probed; asset-only, never per-event), and the globals
|
||||
`zoom`, `skip`, `volume`, `use_audio_channels`.
|
||||
|
||||
**events.json — per-occurrence (one value per event):**
|
||||
`handle` (the video id, **prefix-free**), `cutout`, `layer`, `end_on`, `take`,
|
||||
`pause_narration`.
|
||||
|
||||
**Resolution order for a rendered clip:** event field (if set) → videos.json value (for the
|
||||
globals) → config default. The per-occurrence fields have no videos.json fallback — they
|
||||
are always materialized onto the event at build time.
|
||||
|
||||
Notes:
|
||||
- `end_on` **defaults to `next_video`** for videos when unspecified (was implicitly
|
||||
`next_slide`). Existing videos.json `end_on` values are migrated onto events explicitly,
|
||||
so current projects keep their behavior; only *new* unspecified markers get the new default.
|
||||
- `take` is the event-level cut length, only meaningful when `end_on=take`; otherwise the
|
||||
end is implicit from `end_on` and `take` stays null.
|
||||
- `skip` stays a global (asset trim-in) while `take` is per-event — a deliberate asymmetry:
|
||||
"where this asset generally starts" vs. "how long this occurrence plays."
|
||||
- `zoom`/`volume`/`use_audio_channels` stay global but are inline-overridable per event
|
||||
(below), so they can diverge without a videos.json copy.
|
||||
|
||||
## Authoring: shorthand + inline overloads
|
||||
|
||||
The manuscript stays the compact authoring surface. The shorthand letters encode
|
||||
`cutout`+`layer` (and `pause_narration` via the `…p:` variants). Anything the letters
|
||||
don't encode — chiefly `end_on`, and any per-event override of a global — is given as
|
||||
inline **`key=value`** pairs (simplified from the earlier `{"json":"form"}`):
|
||||
|
||||
```
|
||||
[vsb:glitch_ccd_binning2] # square/below, end_on defaults to next_video
|
||||
[vsb:glitch_ccd_binning2, end_on=next_video] # + explicit end_on
|
||||
[vsb:glitch_ccd_binning2, take=5, volume=0.5] # + per-event overrides of globals
|
||||
[video:glitch_ccd_binning2, cutout=square, layer=below] # generic; equivalent to [vsb:…]
|
||||
```
|
||||
|
||||
Rules:
|
||||
- The first token inside `[]` is `prefix:handle` (handle may contain `/`, e.g. `pexels/123`).
|
||||
- Remaining comma-separated tokens are `key=value`. Values are type-inferred: numeric →
|
||||
float, `true`/`false` → bool, else string. Allowed keys: `cutout`, `layer`, `end_on`,
|
||||
`take`, `skip`, `zoom`, `volume`, `use_audio_channels`, `pause_narration`,
|
||||
`always_visible`.
|
||||
- An inline key overrides whatever the shorthand implied (e.g. `[vst:x, layer=below]` →
|
||||
above from the prefix, then below from the override). Last-writer-wins, prefix first.
|
||||
- `[video:handle, …]` is the fully-explicit form the GUI round-trips: no prefix magic, every
|
||||
presentation field named.
|
||||
|
||||
Why `key=value` over JSON: no braces/quotes to escape inside `[]`, one obvious separator,
|
||||
and it reads cleanly in a script. The GUI still stores the resolved values as real JSON
|
||||
fields on the event — the manuscript form is just sugar that populates them.
|
||||
|
||||
## Build-time materialization
|
||||
|
||||
At build (`build_render_plan` / scaffold construction), each video marker resolves to an
|
||||
atomic event dict:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "video",
|
||||
"handle": "glitch_ccd_binning",
|
||||
"cutout": "square",
|
||||
"layer": "above",
|
||||
"end_on": "next_video",
|
||||
"take": null,
|
||||
"pause_narration": 0.0,
|
||||
"narration_time": 0.0, "adjustment": 0.0, "final_time": 0.0,
|
||||
"mapping": "exact", "confidence": 1.0, "context": "…"
|
||||
}
|
||||
```
|
||||
|
||||
`id` (currently `"vst:glitch_ccd_binning"`) is replaced by `handle` + explicit fields. The
|
||||
render pass reads presentation straight off the event and no longer consults the prefix or
|
||||
the videos.json presentation fields. `merge_events` must preserve manual event edits (the
|
||||
GUI's writes) across rebuilds, the same way it preserves `adjustment` today.
|
||||
|
||||
## Code touchpoints
|
||||
|
||||
- **models.py** — `VideoSource` sheds `cutout`/`layer`/`end_on`/`take`/`pause_narration`
|
||||
(or they become defaults-only); `VideoEvent` already carries `cutout`/`layer`/`end_on` —
|
||||
extend to `take`/`pause_narration` sourced from the event, not the handle.
|
||||
- **parser.py `parse_manuscript`** — extend the marker grammar to accept
|
||||
`prefix:handle, key=value, …`; update the malformed-marker detector (which today flags
|
||||
spaces/commas inside `[]`).
|
||||
- **transformer.py `_extract_video_events`** — resolve `cutout/layer/end_on/take/
|
||||
pause_narration` from (prefix ∪ inline overrides), not from `video_source`.
|
||||
- **scaffold.py** — event schema: `handle` + presentation fields; `merge_events` preserves
|
||||
GUI edits; migration for existing events.json.
|
||||
- **cli.py** — retire `_project_markers_to_videos` and `_writeback_video_metadata` (they
|
||||
project/writeback per-handle presentation) in favor of seeding event fields.
|
||||
- **validator.py** — **remove** the key-reuse collision hard-error (reuse is legal now).
|
||||
- **renderer.py** — read presentation from the event (mostly already does via `VideoEvent`).
|
||||
|
||||
## Migration
|
||||
|
||||
Existing projects (video0–video6, …) have presentation on the handle and prefixed `id`s in
|
||||
events.json. A one-shot migration, run on build:
|
||||
|
||||
1. For each video event, split the prefixed `id` into `handle` + implied `cutout`/`layer`.
|
||||
2. Fill `end_on`/`take`/`pause_narration` from the handle's current videos.json values
|
||||
(preserving today's behavior — including handles that explicitly set `next_slide`).
|
||||
3. Strip the moved fields from videos.json handles (leave the globals).
|
||||
4. Idempotent: a second run is a no-op once events carry `handle`.
|
||||
|
||||
## Staging
|
||||
|
||||
- **Stage A** — schema split + per-event resolution from the shorthand prefix, migration,
|
||||
remove the collision guard. Shorthand-only authoring keeps working; the 5 video5
|
||||
collisions resolve. (This is the part that fixes the bug.)
|
||||
- **Stage B** — the inline `key=value` overload grammar + malformed-marker updates.
|
||||
|
||||
Keep the validator collision hard-error in place **until Stage A lands** — removing it
|
||||
earlier would let the hidden-overlay bug back in on video5.
|
||||
|
||||
## Open questions
|
||||
|
||||
- `always_visible`, `use_audio_channels`: confirmed as inline-overridable globals — do any
|
||||
need to become fully per-event?
|
||||
- Does the GUI want events fully flattened (every field present) or sparse (only overrides,
|
||||
inherit the rest)? Affects whether the build writes defaults explicitly.
|
||||
+13
-15
@@ -707,19 +707,16 @@ def cmd_import(project_path: Path, force: bool, verbose: bool) -> int:
|
||||
if len(keynote_files) > 1:
|
||||
print(f" Warning: Multiple .key files found, using {keynote_file.name}")
|
||||
_import_presenter_notes(project_path, keynote_file, verbose, config)
|
||||
elif verbose:
|
||||
print(" No .key file found, skipping presenter notes import")
|
||||
|
||||
# Generate slides.json for each slide directory (after Keynote export)
|
||||
slides_base = project_path / "media" / "slides"
|
||||
slides_dirs = (
|
||||
[d for d in slides_base.glob("*/") if d.is_dir()]
|
||||
if slides_base.exists()
|
||||
else []
|
||||
)
|
||||
for slides_dir in slides_dirs:
|
||||
_generate_slides_json(slides_dir, verbose)
|
||||
else:
|
||||
if verbose:
|
||||
print(" No .key file found, skipping presenter notes import")
|
||||
# Generate slides.json for this project's slide export dir only. Slides always
|
||||
# export to media/slides/<project>/ (consistent across projects); sibling folders
|
||||
# like spec/, prompt/, render/ are auxiliary AI-workflow files, not slide images,
|
||||
# so scanning them just produced spurious "No image files" warnings.
|
||||
project_slides_dir = project_path / "media" / "slides" / project_path.name.lower()
|
||||
if project_slides_dir.is_dir():
|
||||
_generate_slides_json(project_slides_dir, verbose)
|
||||
|
||||
# Import shared assets (pexels, etc.) from shared_assets directory
|
||||
# Look for shared_assets relative to project or in parent directories
|
||||
@@ -1317,8 +1314,9 @@ def _generate_slides_json(directory: Path, verbose: bool) -> None:
|
||||
# each new videos.json entry (and backfilled onto existing ones) so nothing is
|
||||
# an invisible optional. Auto-managed fields (output_file, duration, has_audio,
|
||||
# is_shared, attribution) are intentionally omitted — they're set by the tool.
|
||||
# end_on defaults to null = "use the marker-type default" (video → next_slide,
|
||||
# narration → play to end); set it to "next_slide", "end", or "take" explicitly.
|
||||
# end_on defaults to null = "use the marker-type default" (video → next_video,
|
||||
# narration → play to end); set it to "next_slide", "next_video", "loop", "end", or
|
||||
# "take" explicitly (resolved per-event by transformer.resolve_video_presentation).
|
||||
_VIDEO_DEFAULTS = {
|
||||
"cutout": "square", # named zone from project.json cutouts
|
||||
"layer": "above", # above | mid | below (relative to slides/narrator)
|
||||
@@ -1330,7 +1328,7 @@ _VIDEO_DEFAULTS = {
|
||||
"use_audio_channels": "both", # both | left | right
|
||||
"always_visible": False, # always on screen (like the talking head)
|
||||
"pause_narration": 0.0, # seconds to freeze narration for a cutscene
|
||||
"end_on": None, # null | next_slide | end | take
|
||||
"end_on": None, # null | next_slide | next_video | loop | end | take
|
||||
}
|
||||
|
||||
|
||||
|
||||
+55
-3
@@ -54,6 +54,55 @@ def _resolve_case_insensitive(path: Path) -> Path:
|
||||
return resolved
|
||||
|
||||
|
||||
# Inline marker-override keys honored at build time. These are the per-occurrence
|
||||
# presentation fields materialized onto events.json and resolved per-event
|
||||
# (transformer.resolve_video_presentation). Global params (skip/zoom/volume/…) are not
|
||||
# yet overridable inline; unknown keys are ignored.
|
||||
_MARKER_OVERRIDE_KEYS = frozenset({"cutout", "layer", "end_on", "take"})
|
||||
|
||||
|
||||
def _coerce_marker_value(key: str, raw: str):
|
||||
"""Coerce an inline override value: `take` → float; everything else → string."""
|
||||
v = raw.strip().strip('"').strip("'")
|
||||
if key == "take":
|
||||
try:
|
||||
return float(v)
|
||||
except ValueError:
|
||||
return None
|
||||
return v
|
||||
|
||||
|
||||
def parse_marker(raw: str) -> "tuple[str, Optional[dict]]":
|
||||
"""Split a bracket's contents into (marker_id, overrides).
|
||||
|
||||
Supports the inline-overload grammar `[prefix:handle, key=value, key=value]`:
|
||||
the text before the first comma is the marker id (prefix:handle); the rest are
|
||||
comma-separated key=value overrides. Only _MARKER_OVERRIDE_KEYS are kept (others
|
||||
ignored). A plain marker (no comma) returns (marker_id, None).
|
||||
|
||||
Examples:
|
||||
"vsb:clip" -> ("vsb:clip", None)
|
||||
"vsb:clip, end_on=next_video" -> ("vsb:clip", {"end_on": "next_video"})
|
||||
"video:clip, cutout=square, layer=below"
|
||||
-> ("video:clip", {"cutout": "square", "layer": "below"})
|
||||
"""
|
||||
if "," not in raw:
|
||||
return raw.strip(), None
|
||||
head, _, tail = raw.partition(",")
|
||||
marker_id = head.strip()
|
||||
overrides: dict = {}
|
||||
for tok in tail.split(","):
|
||||
if "=" not in tok:
|
||||
continue
|
||||
key, _, val = tok.partition("=")
|
||||
key = key.strip().lower()
|
||||
if key in _MARKER_OVERRIDE_KEYS:
|
||||
coerced = _coerce_marker_value(key, val)
|
||||
if coerced is not None:
|
||||
overrides[key] = coerced
|
||||
return marker_id, (overrides or None)
|
||||
|
||||
|
||||
def parse_manuscript(
|
||||
project_path: Path,
|
||||
) -> tuple[str, list[str], list[tuple[int, str]], list[Citation]]:
|
||||
@@ -86,9 +135,12 @@ def parse_manuscript(
|
||||
text = re.sub(r"\[pause\]", "", text)
|
||||
text = re.sub(r"\[stop\]", "", text)
|
||||
|
||||
# Extract all valid markers like [S1], [video:demo], [vf2m:pexels/clip-name], etc.
|
||||
# Include / and - to capture pexels/library video IDs; . to catch file extensions in markers.
|
||||
markers = re.findall(r"\[([A-Za-z0-9_:./\-]+)\]", text)
|
||||
# Extract all valid markers like [S1], [video:demo], [vf2m:pexels/clip-name], and
|
||||
# inline-override forms like [vsb:clip, end_on=next_video]. Include / and - for
|
||||
# pexels/library video IDs; . for file extensions; an optional ",…" tail carries
|
||||
# per-event overrides (parsed out by parse_marker, so `markers` holds bare ids).
|
||||
raw_markers = re.findall(r"\[([A-Za-z0-9_:./\-]+(?:,[^\]\n]*)?)\]", text)
|
||||
markers = [parse_marker(m)[0] for m in raw_markers]
|
||||
|
||||
# Find malformed markers (missing brackets, extra spaces, etc.)
|
||||
malformed: list[tuple[int, str]] = []
|
||||
|
||||
+42
-2
@@ -28,7 +28,11 @@ from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from .models import CAMERA_PRESETS
|
||||
from .transformer import MarkerTiming
|
||||
from .transformer import MarkerTiming, resolve_video_presentation
|
||||
|
||||
# Per-occurrence presentation fields materialized onto video events (atomic events.json,
|
||||
# GUI-ready). Round-tripped as overrides so a stored value drives render over the default.
|
||||
_PRESENTATION_KEYS = ("cutout", "layer", "end_on", "take")
|
||||
|
||||
EVENTS_FILE = "events.json"
|
||||
SCAFFOLD_FILE = "scaffold.json"
|
||||
@@ -97,6 +101,16 @@ _PAUSE_MARKER_PREFIXES = (
|
||||
)
|
||||
|
||||
|
||||
def _lookup_video(marker_id: str, videos: dict):
|
||||
"""Case-insensitive videos.json lookup for a video marker (prefix stripped)."""
|
||||
if not videos:
|
||||
return None
|
||||
handle = marker_id.split(":", 1)[1].lower() if ":" in marker_id else marker_id.lower()
|
||||
return videos.get(handle) or next(
|
||||
(v for k, v in videos.items() if k.lower() == handle), None
|
||||
)
|
||||
|
||||
|
||||
def _pause_duration(marker_id: str, videos: dict) -> float:
|
||||
"""Seconds a pause-variant video marker freezes the narration for, else 0."""
|
||||
if not marker_id.startswith(_PAUSE_MARKER_PREFIXES):
|
||||
@@ -138,8 +152,9 @@ def derive_events(
|
||||
if (placed and t.confidence >= _EXACT_THRESHOLD and not after_prev)
|
||||
else MAPPING_INTERPOLATED
|
||||
)
|
||||
etype = marker_type(t.marker_id, slides, videos, audio)
|
||||
e = {
|
||||
"type": marker_type(t.marker_id, slides, videos, audio),
|
||||
"type": etype,
|
||||
"id": t.marker_id,
|
||||
"narration_time": round(t.timestamp, 3) if placed else None,
|
||||
"adjustment": 0.0,
|
||||
@@ -148,6 +163,27 @@ def derive_events(
|
||||
"confidence": round(t.confidence, 3),
|
||||
"context": (t.context or "")[:80],
|
||||
}
|
||||
# Materialize per-occurrence presentation onto video events so events.json is
|
||||
# atomic (each occurrence self-contained, GUI-editable) rather than depending on
|
||||
# the handle's videos.json entry. Resolved from the shorthand prefix + any prior
|
||||
# override the marker carried.
|
||||
if etype == "video":
|
||||
vs = _lookup_video(t.marker_id, videos)
|
||||
if vs is not None:
|
||||
pres = resolve_video_presentation(
|
||||
t.marker_id,
|
||||
vs,
|
||||
t.overrides,
|
||||
default_end_on=(
|
||||
None if t.marker_id.startswith("narration:") else "next_video"
|
||||
),
|
||||
)
|
||||
e["handle"] = pres["handle"]
|
||||
e["cutout"] = pres["cutout"]
|
||||
e["layer"] = pres["layer"]
|
||||
e["end_on"] = pres["end_on"]
|
||||
if pres["take"] is not None:
|
||||
e["take"] = pres["take"]
|
||||
pd = _pause_duration(t.marker_id, videos)
|
||||
if pd:
|
||||
e["pause_duration"] = pd
|
||||
@@ -289,12 +325,16 @@ def events_to_marker_timings(events: list[dict]) -> list[MarkerTiming]:
|
||||
for e in events:
|
||||
n = e.get("narration_time")
|
||||
eff = (n + e.get("adjustment", 0.0)) if n is not None else -1.0
|
||||
# Carry any stored presentation as overrides so render honors the atomic event
|
||||
# (e.g. a GUI edit) over the shorthand/videos.json default. Absent on old events.
|
||||
overrides = {k: e[k] for k in _PRESENTATION_KEYS if k in e} or None
|
||||
timings.append(
|
||||
MarkerTiming(
|
||||
marker_id=e["id"],
|
||||
timestamp=eff,
|
||||
context=e.get("context", ""),
|
||||
confidence=float(e.get("confidence", 1.0)),
|
||||
overrides=overrides,
|
||||
)
|
||||
)
|
||||
return timings
|
||||
|
||||
+131
-42
@@ -22,7 +22,7 @@ from .models import (
|
||||
VideoEvent,
|
||||
VideoSource,
|
||||
)
|
||||
from .parser import get_video_duration, resolve_missing_videos
|
||||
from .parser import get_video_duration, resolve_missing_videos, parse_marker
|
||||
from .transcriber import TranscribedWord
|
||||
|
||||
# Audio trigger offset: play sound this many seconds before the marker
|
||||
@@ -54,6 +54,59 @@ _SHORTHAND_PREFIXES: dict[str, tuple] = {
|
||||
"vsmp:": ("square", "mid"),
|
||||
}
|
||||
|
||||
# Cutout zone the narration talking-head defaults to when a segment/source has none.
|
||||
# Matches the convention import uses (cli._import_narration_segments writes this).
|
||||
_NARRATION_CUTOUT = "talkinghead"
|
||||
|
||||
|
||||
def resolve_video_presentation(
|
||||
marker_id: str,
|
||||
video_source,
|
||||
overrides: Optional[dict] = None,
|
||||
default_end_on: Optional[str] = "next_video",
|
||||
) -> dict:
|
||||
"""Resolve a video marker's per-occurrence presentation to an atomic dict.
|
||||
|
||||
Single source of truth for how a marker becomes an event, shared by the render
|
||||
transformer (_extract_video_events) and the scaffold that materializes events.json.
|
||||
Presentation is per-occurrence: the shorthand prefix (vst: → square/above) decides
|
||||
cutout+layer for THIS marker, so one handle can appear as vst: and vsb: without a
|
||||
videos.json collision.
|
||||
|
||||
Precedence per field: explicit event override > shorthand prefix > videos.json
|
||||
default > built-in default. `default_end_on` is "next_video" for video triggers and
|
||||
None for [narration:] (which runs to the end).
|
||||
|
||||
Returns {handle, cutout, layer, end_on, take, pause_narration}.
|
||||
"""
|
||||
overrides = overrides or {}
|
||||
|
||||
prefix = next((p for p in _SHORTHAND_PREFIXES if marker_id.startswith(p)), None)
|
||||
if prefix is not None:
|
||||
handle = marker_id[len(prefix):].lower()
|
||||
impl_cutout, impl_layer = _SHORTHAND_PREFIXES[prefix]
|
||||
else:
|
||||
# Legacy [video:X] / [narration:X] — strip the generic prefix if present.
|
||||
handle = (marker_id.split(":", 1)[1] if ":" in marker_id else marker_id).lower()
|
||||
impl_cutout = impl_layer = None
|
||||
|
||||
cutout = overrides.get("cutout") or impl_cutout or video_source.cutout
|
||||
layer = overrides.get("layer") or impl_layer or video_source.layer
|
||||
end_on = overrides.get("end_on") or video_source.end_on or default_end_on
|
||||
take = overrides["take"] if "take" in overrides else video_source.take
|
||||
pause_narration = overrides.get(
|
||||
"pause_narration", video_source.pause_narration or 0.0
|
||||
)
|
||||
|
||||
return {
|
||||
"handle": handle,
|
||||
"cutout": cutout,
|
||||
"layer": layer,
|
||||
"end_on": end_on,
|
||||
"take": take,
|
||||
"pause_narration": float(pause_narration or 0.0),
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class MarkerTiming:
|
||||
@@ -63,6 +116,9 @@ class MarkerTiming:
|
||||
timestamp: float # -1 if not found
|
||||
context: str # the text following the marker
|
||||
confidence: float # 0-1, how confident the match is
|
||||
# Per-occurrence presentation overrides carried from events.json (GUI edits) so
|
||||
# render honors them over the shorthand/videos.json defaults. None on fresh align.
|
||||
overrides: Optional[dict] = None
|
||||
|
||||
|
||||
def _normalize_text(text: str) -> str:
|
||||
@@ -226,7 +282,10 @@ def _extract_marker_contexts(
|
||||
|
||||
raw_contexts = []
|
||||
for i in range(1, len(parts), 2):
|
||||
marker_id = parts[i]
|
||||
# Split the bracket into a bare marker id and any inline overrides
|
||||
# ([vsb:clip, end_on=next_video]); the id drives alignment, the overrides ride
|
||||
# along to the event.
|
||||
marker_id, overrides = parse_marker(parts[i])
|
||||
|
||||
if not _is_known_marker(marker_id, slides, videos, audio):
|
||||
continue
|
||||
@@ -240,7 +299,7 @@ def _extract_marker_contexts(
|
||||
j += 1
|
||||
if j >= len(parts):
|
||||
break
|
||||
if _is_known_marker(parts[j], slides, videos, audio):
|
||||
if _is_known_marker(parse_marker(parts[j])[0], slides, videos, audio):
|
||||
break
|
||||
j += 1
|
||||
|
||||
@@ -248,22 +307,22 @@ def _extract_marker_contexts(
|
||||
following_text = " ".join(following_text.split())
|
||||
following_text = _strip_unknown_markers(following_text, slides, videos, audio)
|
||||
following_text = " ".join(following_text.split())
|
||||
raw_contexts.append((marker_id, following_text))
|
||||
raw_contexts.append((marker_id, following_text, overrides))
|
||||
|
||||
contexts = []
|
||||
for i, (marker_id, following_text) in enumerate(raw_contexts):
|
||||
for i, (marker_id, following_text, overrides) in enumerate(raw_contexts):
|
||||
if following_text:
|
||||
words = following_text.split()[:10]
|
||||
contexts.append((marker_id, " ".join(words), False, "before"))
|
||||
contexts.append((marker_id, " ".join(words), False, "before", overrides))
|
||||
else:
|
||||
borrowed = False
|
||||
for j in range(i + 1, len(raw_contexts)):
|
||||
next_marker_id, next_text = raw_contexts[j]
|
||||
next_marker_id, next_text, _ = raw_contexts[j]
|
||||
if next_text:
|
||||
if next_marker_id in (slides or {}):
|
||||
break
|
||||
words = next_text.split()[:10]
|
||||
contexts.append((marker_id, " ".join(words), True, "before"))
|
||||
contexts.append((marker_id, " ".join(words), True, "before", overrides))
|
||||
borrowed = True
|
||||
break
|
||||
if not borrowed:
|
||||
@@ -278,9 +337,9 @@ def _extract_marker_contexts(
|
||||
if preceding_text:
|
||||
words = preceding_text.split()
|
||||
tail = " ".join(words[-6:])
|
||||
contexts.append((marker_id, tail, False, "after"))
|
||||
contexts.append((marker_id, tail, False, "after", overrides))
|
||||
else:
|
||||
contexts.append((marker_id, "", False, "before"))
|
||||
contexts.append((marker_id, "", False, "before", overrides))
|
||||
|
||||
return contexts
|
||||
|
||||
@@ -484,7 +543,7 @@ def align_markers_to_transcription(
|
||||
last_idx = 0
|
||||
last_end_time = 0.0
|
||||
|
||||
for marker_id, anchor_text, is_borrowed, anchor_type in contexts:
|
||||
for marker_id, anchor_text, is_borrowed, anchor_type, overrides in contexts:
|
||||
if not anchor_text.strip():
|
||||
marker_time = last_end_time + 1.0
|
||||
timings.append(
|
||||
@@ -493,6 +552,7 @@ def align_markers_to_transcription(
|
||||
timestamp=marker_time,
|
||||
context="(after previous)",
|
||||
confidence=1.0,
|
||||
overrides=overrides,
|
||||
)
|
||||
)
|
||||
last_end_time = marker_time
|
||||
@@ -517,6 +577,7 @@ def align_markers_to_transcription(
|
||||
timestamp=marker_time,
|
||||
context=f"(end of: {anchor_text[:40]})",
|
||||
confidence=confidence,
|
||||
overrides=overrides,
|
||||
)
|
||||
)
|
||||
last_idx = match_end_idx
|
||||
@@ -529,6 +590,7 @@ def align_markers_to_transcription(
|
||||
timestamp=adjusted_time,
|
||||
context=anchor_text[:50],
|
||||
confidence=confidence,
|
||||
overrides=overrides,
|
||||
)
|
||||
)
|
||||
if not is_borrowed:
|
||||
@@ -544,6 +606,7 @@ def align_markers_to_transcription(
|
||||
timestamp=-1.0,
|
||||
context=anchor_text[:50],
|
||||
confidence=0.0,
|
||||
overrides=overrides,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -557,7 +620,7 @@ def align_markers_to_transcription(
|
||||
if timing.timestamp >= 0:
|
||||
continue
|
||||
|
||||
marker_id, anchor_text, is_borrowed, anchor_type = contexts[i]
|
||||
marker_id, anchor_text, is_borrowed, anchor_type, overrides = contexts[i]
|
||||
if not anchor_text.strip():
|
||||
continue
|
||||
|
||||
@@ -612,6 +675,7 @@ def align_markers_to_transcription(
|
||||
timestamp=marker_time,
|
||||
context=f"(repaired: {anchor_text[:40]})",
|
||||
confidence=confidence,
|
||||
overrides=overrides,
|
||||
)
|
||||
|
||||
# Deduplicate slide markers. The manuscript pattern [SN]\n\n[SN] text... is
|
||||
@@ -733,8 +797,9 @@ def build_render_plan(
|
||||
# sum of the segment durations and skip is already baked into each segment.
|
||||
if narration_schedule:
|
||||
narration_video_id = "narration"
|
||||
# cutout left unset — resolved to the talkinghead zone below (single source).
|
||||
narration_video = narration_source or VideoSource(
|
||||
source_file="", cutout=config.default_slide_type, always_visible=True
|
||||
source_file="", always_visible=True
|
||||
)
|
||||
narration_skip = 0.0
|
||||
full_duration = sum(seg.duration for seg in narration_schedule)
|
||||
@@ -770,7 +835,20 @@ def build_render_plan(
|
||||
if timing.timestamp >= 0:
|
||||
marker_times[timing.marker_id] = timing.timestamp
|
||||
|
||||
cutout = config.cutouts[narration_video.cutout]
|
||||
# Narration talking-head cutout. Narration segments carry no cutout of their own,
|
||||
# so default to the dedicated "talkinghead" zone (the convention import uses when it
|
||||
# creates narration entries — see cli._import_narration_segments), falling back to
|
||||
# the project's default slide type only if that zone isn't defined. Guard the lookup
|
||||
# so a misconfigured cutout gives a clear error instead of a bare KeyError.
|
||||
narration_cutout_name = narration_video.cutout or (
|
||||
_NARRATION_CUTOUT if _NARRATION_CUTOUT in config.cutouts else config.default_slide_type
|
||||
)
|
||||
if narration_cutout_name not in config.cutouts:
|
||||
raise ValueError(
|
||||
f"Narration cutout '{narration_cutout_name}' not found in project cutouts "
|
||||
f"{list(config.cutouts)}"
|
||||
)
|
||||
cutout = config.cutouts[narration_cutout_name]
|
||||
# Adjust duration for skip (content starts at skip, so effective duration is less)
|
||||
effective_duration = full_duration - narration_skip
|
||||
narration_videos: list[tuple[str, VideoSource, CutoutDefinition]] = [
|
||||
@@ -1194,14 +1272,10 @@ def _extract_video_events(
|
||||
marker_timings, slides, total_duration
|
||||
)
|
||||
|
||||
# Pause-variant prefixes — the only thing the render pass still needs from
|
||||
# shorthand markers at event-build time (pause_narration is per-event, not stored in videos.json).
|
||||
_PAUSE_PREFIXES = {"vftp:", "vfbp:", "vfmp:", "vf2tp:", "vf2bp:", "vf2mp:", "vstp:", "vsbp:", "vsmp:"}
|
||||
|
||||
# Collect video markers: (time, video_id, event_type, pause_narration)
|
||||
# video_markers: (timestamp, video_id, marker_type, pause_narration)
|
||||
# cutout and layer are read from videos.json (projected there by _project_markers_to_videos)
|
||||
video_markers: list[tuple[float, str, str, bool]] = []
|
||||
# Collect video markers. Carry the full marker_id (so presentation resolves from
|
||||
# its shorthand prefix PER occurrence) and any per-event overrides from events.json.
|
||||
# video_markers: (timestamp, marker_id, handle, trigger_type, overrides)
|
||||
video_markers: list[tuple[float, str, str, str, Optional[dict]]] = []
|
||||
|
||||
for timing in marker_timings:
|
||||
if timing.timestamp < 0:
|
||||
@@ -1229,8 +1303,9 @@ def _extract_video_events(
|
||||
f"run render once to project values, or set cutout manually."
|
||||
)
|
||||
continue
|
||||
pause_narration = shorthand_match in _PAUSE_PREFIXES
|
||||
video_markers.append((timing.timestamp, video_id, "video", pause_narration))
|
||||
video_markers.append(
|
||||
(timing.timestamp, mid, video_id, "video", timing.overrides)
|
||||
)
|
||||
continue
|
||||
|
||||
# --- legacy [video:xxx] ---
|
||||
@@ -1247,7 +1322,9 @@ def _extract_video_events(
|
||||
f"[video:{video_id}] has no valid cutout in videos.json — skipped."
|
||||
)
|
||||
continue
|
||||
video_markers.append((timing.timestamp, video_id, "video", False))
|
||||
video_markers.append(
|
||||
(timing.timestamp, mid, video_id, "video", timing.overrides)
|
||||
)
|
||||
continue
|
||||
|
||||
# --- [narration:xxx] ---
|
||||
@@ -1264,29 +1341,41 @@ def _extract_video_events(
|
||||
f"[narration:{video_id}] has no valid cutout in videos.json — skipped."
|
||||
)
|
||||
continue
|
||||
video_markers.append((timing.timestamp, video_id, "narration", False))
|
||||
video_markers.append(
|
||||
(timing.timestamp, mid, video_id, "narration", timing.overrides)
|
||||
)
|
||||
|
||||
# Sorted start times of all video markers — used by end_on="next_video" to cap
|
||||
# a clip when the next video begins, so videos never overlap.
|
||||
video_start_times = sorted(t for t, _, _, _ in video_markers)
|
||||
video_start_times = sorted(t for t, *_ in video_markers)
|
||||
|
||||
events: list[VideoEvent] = []
|
||||
for start_time, video_id, marker_type, pause_narration in video_markers:
|
||||
for start_time, marker_id, video_id, trigger_type, overrides in video_markers:
|
||||
video_source = videos[video_id]
|
||||
|
||||
# Read cutout and layer directly from videos.json (projected by ETL)
|
||||
cutout_name = video_source.cutout
|
||||
# Resolve presentation per-occurrence: shorthand prefix (and any events.json
|
||||
# override) wins over the videos.json default, so one handle can render above
|
||||
# in one place and below in another. [narration:] runs to the end by default.
|
||||
pres = resolve_video_presentation(
|
||||
marker_id,
|
||||
video_source,
|
||||
overrides,
|
||||
default_end_on=(None if trigger_type == "narration" else "next_video"),
|
||||
)
|
||||
cutout_name = pres["cutout"]
|
||||
cutout = cutouts[cutout_name]
|
||||
layer = video_source.layer
|
||||
layer = pres["layer"]
|
||||
end_on = pres["end_on"]
|
||||
take = pres["take"]
|
||||
pause_narration = pres["pause_narration"]
|
||||
|
||||
end_on = video_source.end_on
|
||||
if end_on == "take" and video_source.take is not None:
|
||||
end_time = start_time + video_source.take
|
||||
if end_on == "take" and take is not None:
|
||||
end_time = start_time + take
|
||||
elif end_on == "end":
|
||||
# Play the clip once through its natural length, then stop — no looping.
|
||||
# Natural length = explicit take, else the file's own duration past skip.
|
||||
if video_source.take is not None:
|
||||
natural = video_source.take
|
||||
if take is not None:
|
||||
natural = take
|
||||
elif video_source.duration is not None:
|
||||
natural = max(0.0, video_source.duration - (video_source.skip or 0.0))
|
||||
else:
|
||||
@@ -1304,9 +1393,9 @@ def _extract_video_events(
|
||||
end_time = vt
|
||||
break
|
||||
# A pause-narration video must stay for at least the pause it holds.
|
||||
if video_source.pause_narration:
|
||||
end_time = max(end_time, start_time + video_source.pause_narration)
|
||||
elif end_on in ("next_slide", "slide") or (end_on is None and marker_type == "video"):
|
||||
if pause_narration:
|
||||
end_time = max(end_time, start_time + pause_narration)
|
||||
elif end_on in ("next_slide", "slide"):
|
||||
# End at next slide marker ("slide" is a recognised alias for "next_slide")
|
||||
end_time = total_duration
|
||||
for slide_time in slide_times:
|
||||
@@ -1315,10 +1404,10 @@ def _extract_video_events(
|
||||
break
|
||||
# pause_narration videos must stay visible for the full pause duration —
|
||||
# the narration is held for that long, so the overlay should match.
|
||||
if video_source.pause_narration:
|
||||
end_time = max(end_time, start_time + video_source.pause_narration)
|
||||
if pause_narration:
|
||||
end_time = max(end_time, start_time + pause_narration)
|
||||
else:
|
||||
# end_on is None and marker_type == "narration": runs to end
|
||||
# end_on None ([narration:] with no explicit end) — runs to end.
|
||||
end_time = total_duration
|
||||
|
||||
# Filter by time range — CHUNKING v2 (docs/chunking_v2.md).
|
||||
|
||||
@@ -58,6 +58,10 @@ def validate_project(
|
||||
elif marker.startswith("narration:"):
|
||||
referenced_video_ids.add(marker[10:].lower())
|
||||
|
||||
# (Key-reuse across cutout/layer is legal: presentation is resolved per-occurrence
|
||||
# from the shorthand prefix now — see transformer.resolve_video_presentation — so
|
||||
# one handle can appear as vst: (above) and vsb: (below) without colliding.)
|
||||
|
||||
# Check for malformed markers first (these are likely typos)
|
||||
if malformed_markers:
|
||||
for line_num, marker_text in malformed_markers:
|
||||
|
||||
Reference in New Issue
Block a user