Compare commits
3
Commits
4e1bfe03e2
...
0c8f662bee
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0c8f662bee | ||
|
|
cdda2e9024 | ||
|
|
a8aab55bd2 |
@@ -0,0 +1,165 @@
|
||||
# Atomic Events — Design Spec
|
||||
|
||||
Status: **Stage A + Stage B implemented (2026-07-27).** Motivated by a future **Glitch
|
||||
Studio GUI** that edits each video occurrence as a self-contained object.
|
||||
|
||||
Implemented:
|
||||
- Per-occurrence presentation resolves via `transformer.resolve_video_presentation`
|
||||
(precedence: inline/GUI override > shorthand prefix > videos.json > default; video
|
||||
`end_on` default = `next_video`, `[narration:]` runs to end).
|
||||
- events.json is materialized/atomic: `derive_events` writes `handle/cutout/layer/end_on/
|
||||
take`; `events_to_marker_timings` round-trips them as overrides.
|
||||
- Inline grammar `[prefix:handle, key=value, …]` (`parser.parse_marker`), threaded through
|
||||
alignment into `MarkerTiming.overrides`. Supported inline keys: **cutout, layer, end_on,
|
||||
take** (the fully-wired per-event fields). Unknown keys are ignored.
|
||||
- The key-reuse collision validator hard-error was removed (reuse is legal now).
|
||||
|
||||
Deferred (follow-ups): inline override of the *global* params (skip/zoom/volume/
|
||||
use_audio_channels/pause_narration) — the renderer reads these from `video_source` in ~13
|
||||
places, so wiring them per-event is a separate change; stripping the moved fields from
|
||||
videos.json (kept as fallback defaults for now); a validator warning for unknown/unwired
|
||||
inline keys.
|
||||
|
||||
## Problem
|
||||
|
||||
Presentation/timing properties (`cutout`, `layer`, `end_on`, `take`, `pause_narration`)
|
||||
live on the **videos.json handle**, but they are really properties of *where a clip is
|
||||
used*, not of the file. The shorthand prefix (`vst:` = square/above, `vsb:` =
|
||||
square/below) is per-marker, but `_project_markers_to_videos` collapses it onto the
|
||||
single handle record (last-wins). So one handle used two ways collides:
|
||||
|
||||
- `[vst:glitch_ccd_binning]` (above) and `[vsb:glitch_ccd_binning]` (below) → videos.json
|
||||
can only store `layer: below`, so the first occurrence renders under the slide (hidden).
|
||||
- video5 has 5 such collisions today (glitch_ccd_binning, pexels/12471039…,
|
||||
mainvideopart1, shotnoiseacc, slide_periodogram).
|
||||
|
||||
A stopgap validator hard-error (`validate_project`, gnommo/validator.py) currently blocks
|
||||
render on these. This spec removes the *cause* so that guard is no longer needed.
|
||||
|
||||
The naive fixes are both rejected: copying the file/handle (duplication on disk), and a
|
||||
"hybrid override + materialize" layer (too much indirection). Instead: **the per-occurrence
|
||||
properties move onto the event.**
|
||||
|
||||
## Field homes
|
||||
|
||||
**videos.json — asset + global defaults (one value per handle):**
|
||||
`source_file`, `output_file`/`processed_file`, `filter`, `has_audio`, `is_shared`,
|
||||
`src_mtime`, `duration` (probed; asset-only, never per-event), and the globals
|
||||
`zoom`, `skip`, `volume`, `use_audio_channels`.
|
||||
|
||||
**events.json — per-occurrence (one value per event):**
|
||||
`handle` (the video id, **prefix-free**), `cutout`, `layer`, `end_on`, `take`,
|
||||
`pause_narration`.
|
||||
|
||||
**Resolution order for a rendered clip:** event field (if set) → videos.json value (for the
|
||||
globals) → config default. The per-occurrence fields have no videos.json fallback — they
|
||||
are always materialized onto the event at build time.
|
||||
|
||||
Notes:
|
||||
- `end_on` **defaults to `next_video`** for videos when unspecified (was implicitly
|
||||
`next_slide`). Existing videos.json `end_on` values are migrated onto events explicitly,
|
||||
so current projects keep their behavior; only *new* unspecified markers get the new default.
|
||||
- `take` is the event-level cut length, only meaningful when `end_on=take`; otherwise the
|
||||
end is implicit from `end_on` and `take` stays null.
|
||||
- `skip` stays a global (asset trim-in) while `take` is per-event — a deliberate asymmetry:
|
||||
"where this asset generally starts" vs. "how long this occurrence plays."
|
||||
- `zoom`/`volume`/`use_audio_channels` stay global but are inline-overridable per event
|
||||
(below), so they can diverge without a videos.json copy.
|
||||
|
||||
## Authoring: shorthand + inline overloads
|
||||
|
||||
The manuscript stays the compact authoring surface. The shorthand letters encode
|
||||
`cutout`+`layer` (and `pause_narration` via the `…p:` variants). Anything the letters
|
||||
don't encode — chiefly `end_on`, and any per-event override of a global — is given as
|
||||
inline **`key=value`** pairs (simplified from the earlier `{"json":"form"}`):
|
||||
|
||||
```
|
||||
[vsb:glitch_ccd_binning2] # square/below, end_on defaults to next_video
|
||||
[vsb:glitch_ccd_binning2, end_on=next_video] # + explicit end_on
|
||||
[vsb:glitch_ccd_binning2, take=5, volume=0.5] # + per-event overrides of globals
|
||||
[video:glitch_ccd_binning2, cutout=square, layer=below] # generic; equivalent to [vsb:…]
|
||||
```
|
||||
|
||||
Rules:
|
||||
- The first token inside `[]` is `prefix:handle` (handle may contain `/`, e.g. `pexels/123`).
|
||||
- Remaining comma-separated tokens are `key=value`. Values are type-inferred: numeric →
|
||||
float, `true`/`false` → bool, else string. Allowed keys: `cutout`, `layer`, `end_on`,
|
||||
`take`, `skip`, `zoom`, `volume`, `use_audio_channels`, `pause_narration`,
|
||||
`always_visible`.
|
||||
- An inline key overrides whatever the shorthand implied (e.g. `[vst:x, layer=below]` →
|
||||
above from the prefix, then below from the override). Last-writer-wins, prefix first.
|
||||
- `[video:handle, …]` is the fully-explicit form the GUI round-trips: no prefix magic, every
|
||||
presentation field named.
|
||||
|
||||
Why `key=value` over JSON: no braces/quotes to escape inside `[]`, one obvious separator,
|
||||
and it reads cleanly in a script. The GUI still stores the resolved values as real JSON
|
||||
fields on the event — the manuscript form is just sugar that populates them.
|
||||
|
||||
## Build-time materialization
|
||||
|
||||
At build (`build_render_plan` / scaffold construction), each video marker resolves to an
|
||||
atomic event dict:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "video",
|
||||
"handle": "glitch_ccd_binning",
|
||||
"cutout": "square",
|
||||
"layer": "above",
|
||||
"end_on": "next_video",
|
||||
"take": null,
|
||||
"pause_narration": 0.0,
|
||||
"narration_time": 0.0, "adjustment": 0.0, "final_time": 0.0,
|
||||
"mapping": "exact", "confidence": 1.0, "context": "…"
|
||||
}
|
||||
```
|
||||
|
||||
`id` (currently `"vst:glitch_ccd_binning"`) is replaced by `handle` + explicit fields. The
|
||||
render pass reads presentation straight off the event and no longer consults the prefix or
|
||||
the videos.json presentation fields. `merge_events` must preserve manual event edits (the
|
||||
GUI's writes) across rebuilds, the same way it preserves `adjustment` today.
|
||||
|
||||
## Code touchpoints
|
||||
|
||||
- **models.py** — `VideoSource` sheds `cutout`/`layer`/`end_on`/`take`/`pause_narration`
|
||||
(or they become defaults-only); `VideoEvent` already carries `cutout`/`layer`/`end_on` —
|
||||
extend to `take`/`pause_narration` sourced from the event, not the handle.
|
||||
- **parser.py `parse_manuscript`** — extend the marker grammar to accept
|
||||
`prefix:handle, key=value, …`; update the malformed-marker detector (which today flags
|
||||
spaces/commas inside `[]`).
|
||||
- **transformer.py `_extract_video_events`** — resolve `cutout/layer/end_on/take/
|
||||
pause_narration` from (prefix ∪ inline overrides), not from `video_source`.
|
||||
- **scaffold.py** — event schema: `handle` + presentation fields; `merge_events` preserves
|
||||
GUI edits; migration for existing events.json.
|
||||
- **cli.py** — retire `_project_markers_to_videos` and `_writeback_video_metadata` (they
|
||||
project/writeback per-handle presentation) in favor of seeding event fields.
|
||||
- **validator.py** — **remove** the key-reuse collision hard-error (reuse is legal now).
|
||||
- **renderer.py** — read presentation from the event (mostly already does via `VideoEvent`).
|
||||
|
||||
## Migration
|
||||
|
||||
Existing projects (video0–video6, …) have presentation on the handle and prefixed `id`s in
|
||||
events.json. A one-shot migration, run on build:
|
||||
|
||||
1. For each video event, split the prefixed `id` into `handle` + implied `cutout`/`layer`.
|
||||
2. Fill `end_on`/`take`/`pause_narration` from the handle's current videos.json values
|
||||
(preserving today's behavior — including handles that explicitly set `next_slide`).
|
||||
3. Strip the moved fields from videos.json handles (leave the globals).
|
||||
4. Idempotent: a second run is a no-op once events carry `handle`.
|
||||
|
||||
## Staging
|
||||
|
||||
- **Stage A** — schema split + per-event resolution from the shorthand prefix, migration,
|
||||
remove the collision guard. Shorthand-only authoring keeps working; the 5 video5
|
||||
collisions resolve. (This is the part that fixes the bug.)
|
||||
- **Stage B** — the inline `key=value` overload grammar + malformed-marker updates.
|
||||
|
||||
Keep the validator collision hard-error in place **until Stage A lands** — removing it
|
||||
earlier would let the hidden-overlay bug back in on video5.
|
||||
|
||||
## Open questions
|
||||
|
||||
- `always_visible`, `use_audio_channels`: confirmed as inline-overridable globals — do any
|
||||
need to become fully per-event?
|
||||
- Does the GUI want events fully flattened (every field present) or sparse (only overrides,
|
||||
inherit the rest)? Affects whether the build writes defaults explicitly.
|
||||
+25
-12
@@ -1,6 +1,9 @@
|
||||
# Chunked Rendering v2 — Design Spec
|
||||
|
||||
Status: **planned** (v1 shipped; v1 boundary limitation is detected + warned, not yet fixed)
|
||||
Status: **implemented on branch `chunking-v2`, pending render-seam validation on the rig.**
|
||||
Plan-level logic is covered by `tests/test_chunking_v2.py` (all green). What remains
|
||||
is confirming the ffmpeg concat seam is frame/phase-accurate on a real render — see
|
||||
"Concat-seam correctness" below.
|
||||
|
||||
## Why chunking exists
|
||||
|
||||
@@ -113,16 +116,26 @@ via `skip_override`. Validation plan:
|
||||
|
||||
## Work items
|
||||
|
||||
- [ ] `_extract_video_events`: overlap test + `skip_override` (loop-aware).
|
||||
- [ ] `_extract_audio_events`: add `end_time`/`skip_override`, overlap + seek.
|
||||
- [ ] `AudioEvent`: `skip_override` field; renderer audio path honors it.
|
||||
- [ ] `OutroEvent`: same treatment.
|
||||
- [ ] Frame/audio seam-diff test (chunked vs full) in the test suite.
|
||||
- [ ] Flip `_chunk_boundary_span_warnings` from "will be dropped" to a debug-only
|
||||
assertion once v2 is the default.
|
||||
- [x] `_extract_video_events`: overlap test + `skip_override` (loop-aware).
|
||||
- [x] `_extract_audio_events`: overlap + `src_offset` seek (loop phase / linear).
|
||||
- [x] `AudioEvent`: `src_offset` field; renderer audio paths (loop-with-pauses,
|
||||
standard loop, one-shot) honor it.
|
||||
- [x] `VideoEvent.skip_override`: renderer video input `-ss` honors it; the clip's
|
||||
embedded audio (`tvaud`) is seeked automatically by the same input seek.
|
||||
- [x] `_chunk_boundary_span_warnings`: downgraded from the "will be dropped" v1
|
||||
warning to an informational note (logged; terminal only under `--verbose`).
|
||||
- [x] Plan-level tests: `tests/test_chunking_v2.py`.
|
||||
- [ ] **Render-seam validation on the rig** (chunked-vs-full frame/audio diff) — the
|
||||
remaining gate before making v2 the trusted default.
|
||||
- [ ] Crossfade-loop audio (`_build_crossfade_loop_filter`) does not yet apply
|
||||
`src_offset` — a crossfaded looping bed restarts phase at the seam. Standard
|
||||
(non-crossfade) loops and one-shots are handled. Low priority.
|
||||
- [ ] `OutroEvent`: not needed — outros are extracted for the last chunk only
|
||||
(`config.outro if is_last_chunk`), so they never split across a seam.
|
||||
|
||||
## Seams already in place (v2 preparations, shipped)
|
||||
## Not affected (verified)
|
||||
|
||||
- `VideoEvent.skip_override` (models.py) — inert; renderer honors it when set.
|
||||
- `cli._chunk_boundary_span_warnings` — detection + warning.
|
||||
- v1-limitation comments at both filter sites in `transformer.py`.
|
||||
- Slides: `_extract_slide_events` already used overlap+clamp — no change.
|
||||
- Full-screen `plan.background`: a separate always-included input.
|
||||
- Full (non-chunked) render: `time_range=None` path leaves `skip_override`/
|
||||
`src_offset` at their defaults, so output is byte-identical to before.
|
||||
|
||||
+37
-47
@@ -707,20 +707,17 @@ def cmd_import(project_path: Path, force: bool, verbose: bool) -> int:
|
||||
if len(keynote_files) > 1:
|
||||
print(f" Warning: Multiple .key files found, using {keynote_file.name}")
|
||||
_import_presenter_notes(project_path, keynote_file, verbose, config)
|
||||
|
||||
# Generate slides.json for each slide directory (after Keynote export)
|
||||
slides_base = project_path / "media" / "slides"
|
||||
slides_dirs = (
|
||||
[d for d in slides_base.glob("*/") if d.is_dir()]
|
||||
if slides_base.exists()
|
||||
else []
|
||||
)
|
||||
for slides_dir in slides_dirs:
|
||||
_generate_slides_json(slides_dir, verbose)
|
||||
else:
|
||||
if verbose:
|
||||
elif verbose:
|
||||
print(" No .key file found, skipping presenter notes import")
|
||||
|
||||
# Generate slides.json for this project's slide export dir only. Slides always
|
||||
# export to media/slides/<project>/ (consistent across projects); sibling folders
|
||||
# like spec/, prompt/, render/ are auxiliary AI-workflow files, not slide images,
|
||||
# so scanning them just produced spurious "No image files" warnings.
|
||||
project_slides_dir = project_path / "media" / "slides" / project_path.name.lower()
|
||||
if project_slides_dir.is_dir():
|
||||
_generate_slides_json(project_slides_dir, verbose)
|
||||
|
||||
# Import shared assets (pexels, etc.) from shared_assets directory
|
||||
# Look for shared_assets relative to project or in parent directories
|
||||
shared_assets_dir = _find_shared_assets(project_path)
|
||||
@@ -1317,8 +1314,9 @@ def _generate_slides_json(directory: Path, verbose: bool) -> None:
|
||||
# each new videos.json entry (and backfilled onto existing ones) so nothing is
|
||||
# an invisible optional. Auto-managed fields (output_file, duration, has_audio,
|
||||
# is_shared, attribution) are intentionally omitted — they're set by the tool.
|
||||
# end_on defaults to null = "use the marker-type default" (video → next_slide,
|
||||
# narration → play to end); set it to "next_slide", "end", or "take" explicitly.
|
||||
# end_on defaults to null = "use the marker-type default" (video → next_video,
|
||||
# narration → play to end); set it to "next_slide", "next_video", "loop", "end", or
|
||||
# "take" explicitly (resolved per-event by transformer.resolve_video_presentation).
|
||||
_VIDEO_DEFAULTS = {
|
||||
"cutout": "square", # named zone from project.json cutouts
|
||||
"layer": "above", # above | mid | below (relative to slides/narrator)
|
||||
@@ -1330,7 +1328,7 @@ _VIDEO_DEFAULTS = {
|
||||
"use_audio_channels": "both", # both | left | right
|
||||
"always_visible": False, # always on screen (like the talking head)
|
||||
"pause_narration": 0.0, # seconds to freeze narration for a cutscene
|
||||
"end_on": None, # null | next_slide | end | take
|
||||
"end_on": None, # null | next_slide | next_video | loop | end | take
|
||||
}
|
||||
|
||||
|
||||
@@ -4137,16 +4135,14 @@ def _writeback_video_metadata(plan, project_path, config) -> None:
|
||||
|
||||
|
||||
def _chunk_boundary_span_warnings(plan, groups) -> list[str]:
|
||||
"""Detect events that span a chunk boundary.
|
||||
"""Report clips that span a chunk boundary.
|
||||
|
||||
KNOWN LIMITATION (chunking v1): the render's time-range filter keeps only events
|
||||
whose *start* falls inside a chunk's window (transformer.py `_extract_video_events`
|
||||
/ `_extract_audio_events`: `if start_time < range_start: continue`). An overlay
|
||||
video or audio clip that began in an earlier chunk and is still playing across the
|
||||
boundary is therefore DROPPED from every later chunk it overlaps — so the chunked
|
||||
output silently diverges from a full render. This detector makes that non-silent.
|
||||
The real fix (overlap-inclusion + per-event seek) is chunking v2 — see
|
||||
docs/chunking_v2.md.
|
||||
Chunking v2 (docs/chunking_v2.md) INCLUDES these in the later chunk and seeks
|
||||
into them (VideoEvent.skip_override / AudioEvent.src_offset) so they resume
|
||||
mid-clip instead of being dropped. The seam then relies on `-c copy` joining
|
||||
frame-aligned chunks, which is the one thing worth eyeballing — so this stays as
|
||||
an informational list (logged; shown on the terminal only with --verbose), not
|
||||
the hard "will be dropped" warning of v1.
|
||||
"""
|
||||
slide_start = {e.slide_id: e.start_time for e in plan.slide_events}
|
||||
boundaries = [] # (slide_id, output_time) at each chunk seam after the first
|
||||
@@ -4166,8 +4162,8 @@ def _chunk_boundary_span_warnings(plan, groups) -> list[str]:
|
||||
extra = f" (and {len(crossed) - 1} more)" if len(crossed) > 1 else ""
|
||||
warnings.append(
|
||||
f"{kind} '{label}' plays {_format_time(start)}–{_format_time(end)} and "
|
||||
f"crosses the chunk boundary at {sid} ({_format_time(bt)}){extra}; it will "
|
||||
f"be DROPPED from the chunk(s) after the boundary."
|
||||
f"crosses the chunk boundary at {sid} ({_format_time(bt)}){extra}; v2 "
|
||||
f"seeks into it so it continues across the seam."
|
||||
)
|
||||
|
||||
for e in plan.video_events:
|
||||
@@ -4211,29 +4207,23 @@ def _chunked_render(
|
||||
f"\n Auto-chunking: {len(slide_ids)} slides → {len(groups)} chunks of ≤{chunk_size}"
|
||||
)
|
||||
|
||||
# Warn about events that span a chunk boundary — chunking v1 drops these from
|
||||
# the later chunk, so the output would silently differ from a full render.
|
||||
# Report clips that span a chunk boundary. v2 seeks into them so they continue
|
||||
# across the seam (no longer dropped); this is informational — logged always,
|
||||
# and echoed to the terminal only under --verbose — with a nudge to eyeball the
|
||||
# seam since concat uses -c copy.
|
||||
if plan is not None:
|
||||
_span_warnings = _chunk_boundary_span_warnings(plan, groups)
|
||||
if _span_warnings:
|
||||
banner = " " + "!" * 64
|
||||
print(f"\n{banner}", file=sys.stderr)
|
||||
print(" CHUNK-BOUNDARY WARNING — chunked output will differ from a full render:",
|
||||
file=sys.stderr)
|
||||
for w in _span_warnings:
|
||||
print(f" - {w}", file=sys.stderr)
|
||||
print(" These overlay video/audio clips span a chunk seam. Chunking v1 keeps",
|
||||
file=sys.stderr)
|
||||
print(" only clips that START inside a chunk, so the ones above vanish after",
|
||||
file=sys.stderr)
|
||||
print(" the boundary. Options: render this project WITHOUT chunking (lower",
|
||||
file=sys.stderr)
|
||||
print(" cpu_limit instead), or move the chunk size so no seam splits them.",
|
||||
file=sys.stderr)
|
||||
print(f"{banner}\n", file=sys.stderr)
|
||||
_render_log("CHUNK-BOUNDARY WARNING (v1 drops cross-boundary clips):")
|
||||
for w in _span_warnings:
|
||||
_span_notes = _chunk_boundary_span_warnings(plan, groups)
|
||||
if _span_notes:
|
||||
_render_log(f"chunking v2: {len(_span_notes)} clip(s) span a boundary (seam-seeked):")
|
||||
for w in _span_notes:
|
||||
_render_log(f" - {w}")
|
||||
if verbose:
|
||||
print(f"\n {len(_span_notes)} clip(s) span a chunk boundary — v2 seeks across the seam:",
|
||||
file=sys.stderr)
|
||||
for w in _span_notes:
|
||||
print(f" - {w}", file=sys.stderr)
|
||||
print(" If a seam looks off, verify frame alignment (concat uses -c copy).",
|
||||
file=sys.stderr)
|
||||
|
||||
chunks_dir = out_dir / "chunks"
|
||||
chunks_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@@ -416,6 +416,11 @@ class AudioEvent:
|
||||
audio_id: str
|
||||
start_time: float # When to start playing (marker time - offset)
|
||||
audio_def: AudioDefinition
|
||||
# Chunking v2 (docs/chunking_v2.md): when a clip began before this chunk's
|
||||
# window it must resume mid-track, not restart at the seam. src_offset is the
|
||||
# position (seconds) into the source stream to begin at — the loop phase for
|
||||
# looping music, or a linear seek for one-shots. 0.0 = play from the start (v1).
|
||||
src_offset: float = 0.0
|
||||
|
||||
|
||||
@dataclass
|
||||
|
||||
+80
-5
@@ -54,6 +54,55 @@ def _resolve_case_insensitive(path: Path) -> Path:
|
||||
return resolved
|
||||
|
||||
|
||||
# Inline marker-override keys honored at build time. These are the per-occurrence
|
||||
# presentation fields materialized onto events.json and resolved per-event
|
||||
# (transformer.resolve_video_presentation). Global params (skip/zoom/volume/…) are not
|
||||
# yet overridable inline; unknown keys are ignored.
|
||||
_MARKER_OVERRIDE_KEYS = frozenset({"cutout", "layer", "end_on", "take"})
|
||||
|
||||
|
||||
def _coerce_marker_value(key: str, raw: str):
|
||||
"""Coerce an inline override value: `take` → float; everything else → string."""
|
||||
v = raw.strip().strip('"').strip("'")
|
||||
if key == "take":
|
||||
try:
|
||||
return float(v)
|
||||
except ValueError:
|
||||
return None
|
||||
return v
|
||||
|
||||
|
||||
def parse_marker(raw: str) -> "tuple[str, Optional[dict]]":
|
||||
"""Split a bracket's contents into (marker_id, overrides).
|
||||
|
||||
Supports the inline-overload grammar `[prefix:handle, key=value, key=value]`:
|
||||
the text before the first comma is the marker id (prefix:handle); the rest are
|
||||
comma-separated key=value overrides. Only _MARKER_OVERRIDE_KEYS are kept (others
|
||||
ignored). A plain marker (no comma) returns (marker_id, None).
|
||||
|
||||
Examples:
|
||||
"vsb:clip" -> ("vsb:clip", None)
|
||||
"vsb:clip, end_on=next_video" -> ("vsb:clip", {"end_on": "next_video"})
|
||||
"video:clip, cutout=square, layer=below"
|
||||
-> ("video:clip", {"cutout": "square", "layer": "below"})
|
||||
"""
|
||||
if "," not in raw:
|
||||
return raw.strip(), None
|
||||
head, _, tail = raw.partition(",")
|
||||
marker_id = head.strip()
|
||||
overrides: dict = {}
|
||||
for tok in tail.split(","):
|
||||
if "=" not in tok:
|
||||
continue
|
||||
key, _, val = tok.partition("=")
|
||||
key = key.strip().lower()
|
||||
if key in _MARKER_OVERRIDE_KEYS:
|
||||
coerced = _coerce_marker_value(key, val)
|
||||
if coerced is not None:
|
||||
overrides[key] = coerced
|
||||
return marker_id, (overrides or None)
|
||||
|
||||
|
||||
def parse_manuscript(
|
||||
project_path: Path,
|
||||
) -> tuple[str, list[str], list[tuple[int, str]], list[Citation]]:
|
||||
@@ -86,9 +135,12 @@ def parse_manuscript(
|
||||
text = re.sub(r"\[pause\]", "", text)
|
||||
text = re.sub(r"\[stop\]", "", text)
|
||||
|
||||
# Extract all valid markers like [S1], [video:demo], [vf2m:pexels/clip-name], etc.
|
||||
# Include / and - to capture pexels/library video IDs; . to catch file extensions in markers.
|
||||
markers = re.findall(r"\[([A-Za-z0-9_:./\-]+)\]", text)
|
||||
# Extract all valid markers like [S1], [video:demo], [vf2m:pexels/clip-name], and
|
||||
# inline-override forms like [vsb:clip, end_on=next_video]. Include / and - for
|
||||
# pexels/library video IDs; . for file extensions; an optional ",…" tail carries
|
||||
# per-event overrides (parsed out by parse_marker, so `markers` holds bare ids).
|
||||
raw_markers = re.findall(r"\[([A-Za-z0-9_:./\-]+(?:,[^\]\n]*)?)\]", text)
|
||||
markers = [parse_marker(m)[0] for m in raw_markers]
|
||||
|
||||
# Find malformed markers (missing brackets, extra spaces, etc.)
|
||||
malformed: list[tuple[int, str]] = []
|
||||
@@ -625,12 +677,35 @@ def parse_narration(
|
||||
default_filters = config.default_filters if config else {}
|
||||
|
||||
narration = {}
|
||||
_narr_video_exts = {".mov", ".mp4", ".webm", ".avi", ".mkv", ".m4v"}
|
||||
for segment_id, segment_data in data.items():
|
||||
if "source_file" not in segment_data:
|
||||
if not segment_data.get("source_file"):
|
||||
# source_file can drift out of an entry (case/sync churn between
|
||||
# machines). Recover the way import/prune/trim do: find the raw
|
||||
# recording whose stem matches the segment id, case-insensitively.
|
||||
recovered = None
|
||||
for sub in ("raw_mov", "processed"):
|
||||
sub_dir = narration_dir / sub
|
||||
if not sub_dir.is_dir():
|
||||
continue
|
||||
for f in sorted(sub_dir.iterdir()):
|
||||
if (
|
||||
f.is_file()
|
||||
and f.suffix.lower() in _narr_video_exts
|
||||
and f.stem.lower() == segment_id.lower()
|
||||
):
|
||||
recovered = f"{sub}/{f.name}"
|
||||
break
|
||||
if recovered:
|
||||
break
|
||||
if recovered is None:
|
||||
raise ParseError(
|
||||
f"Narration segment '{segment_id}' missing required field 'source_file'",
|
||||
f"Narration segment '{segment_id}' missing required field 'source_file' "
|
||||
f"and no matching recording was found in raw_mov/ or processed/. "
|
||||
f"Add a 'source_file' or run 'import' to repair narration.json.",
|
||||
narration_path,
|
||||
)
|
||||
segment_data = {**segment_data, "source_file": recovered}
|
||||
|
||||
# Resolve filter - can be a list or a string reference to default_filters
|
||||
filter_value = segment_data.get("filter", [])
|
||||
|
||||
+12
-5
@@ -1473,7 +1473,9 @@ def build_filter_complex(
|
||||
for p in plan.narration_pauses
|
||||
if p.output_time > event.start_time
|
||||
]
|
||||
src_pos = 0.0
|
||||
# Chunking v2: start partway into the looped stream when the
|
||||
# clip began in an earlier chunk (docs/chunking_v2.md).
|
||||
src_pos = getattr(event, "src_offset", 0.0)
|
||||
seg_start = event.start_time
|
||||
seg_count = 0
|
||||
|
||||
@@ -1530,21 +1532,26 @@ def build_filter_complex(
|
||||
)
|
||||
filters.extend(crossfade_filters)
|
||||
else:
|
||||
# Standard loop without crossfade
|
||||
# Standard loop without crossfade. Chunking v2: seek to
|
||||
# the loop phase when the clip began in an earlier chunk.
|
||||
_off = getattr(event, "src_offset", 0.0)
|
||||
filters.append(
|
||||
f"[{audio_idx}:a]aloop=loop=-1:size=2e+09,"
|
||||
f"atrim=0:{remaining:.3f},"
|
||||
f"atrim={_off:.3f}:{_off + remaining:.3f},"
|
||||
f"asetpts=PTS-STARTPTS,"
|
||||
f"adelay={delay_ms}|{delay_ms},"
|
||||
f"volume={volume:.2f}[{label}]"
|
||||
)
|
||||
audio_labels_to_mix.append(f"[{label}]")
|
||||
else:
|
||||
# One-shot audio: delay to trigger time
|
||||
# One-shot audio: delay to trigger time. Chunking v2: seek in if
|
||||
# the clip began in an earlier chunk (docs/chunking_v2.md).
|
||||
label = f"aud{i}"
|
||||
delay_ms = int(event.start_time * 1000)
|
||||
_off = getattr(event, "src_offset", 0.0)
|
||||
_seek = f"atrim={_off:.3f},asetpts=PTS-STARTPTS," if _off > 0 else ""
|
||||
filters.append(
|
||||
f"[{audio_idx}:a]adelay={delay_ms}|{delay_ms},volume={volume:.2f}[{label}]"
|
||||
f"[{audio_idx}:a]{_seek}adelay={delay_ms}|{delay_ms},volume={volume:.2f}[{label}]"
|
||||
)
|
||||
audio_labels_to_mix.append(f"[{label}]")
|
||||
|
||||
|
||||
+42
-2
@@ -28,7 +28,11 @@ from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from .models import CAMERA_PRESETS
|
||||
from .transformer import MarkerTiming
|
||||
from .transformer import MarkerTiming, resolve_video_presentation
|
||||
|
||||
# Per-occurrence presentation fields materialized onto video events (atomic events.json,
|
||||
# GUI-ready). Round-tripped as overrides so a stored value drives render over the default.
|
||||
_PRESENTATION_KEYS = ("cutout", "layer", "end_on", "take")
|
||||
|
||||
EVENTS_FILE = "events.json"
|
||||
SCAFFOLD_FILE = "scaffold.json"
|
||||
@@ -97,6 +101,16 @@ _PAUSE_MARKER_PREFIXES = (
|
||||
)
|
||||
|
||||
|
||||
def _lookup_video(marker_id: str, videos: dict):
|
||||
"""Case-insensitive videos.json lookup for a video marker (prefix stripped)."""
|
||||
if not videos:
|
||||
return None
|
||||
handle = marker_id.split(":", 1)[1].lower() if ":" in marker_id else marker_id.lower()
|
||||
return videos.get(handle) or next(
|
||||
(v for k, v in videos.items() if k.lower() == handle), None
|
||||
)
|
||||
|
||||
|
||||
def _pause_duration(marker_id: str, videos: dict) -> float:
|
||||
"""Seconds a pause-variant video marker freezes the narration for, else 0."""
|
||||
if not marker_id.startswith(_PAUSE_MARKER_PREFIXES):
|
||||
@@ -138,8 +152,9 @@ def derive_events(
|
||||
if (placed and t.confidence >= _EXACT_THRESHOLD and not after_prev)
|
||||
else MAPPING_INTERPOLATED
|
||||
)
|
||||
etype = marker_type(t.marker_id, slides, videos, audio)
|
||||
e = {
|
||||
"type": marker_type(t.marker_id, slides, videos, audio),
|
||||
"type": etype,
|
||||
"id": t.marker_id,
|
||||
"narration_time": round(t.timestamp, 3) if placed else None,
|
||||
"adjustment": 0.0,
|
||||
@@ -148,6 +163,27 @@ def derive_events(
|
||||
"confidence": round(t.confidence, 3),
|
||||
"context": (t.context or "")[:80],
|
||||
}
|
||||
# Materialize per-occurrence presentation onto video events so events.json is
|
||||
# atomic (each occurrence self-contained, GUI-editable) rather than depending on
|
||||
# the handle's videos.json entry. Resolved from the shorthand prefix + any prior
|
||||
# override the marker carried.
|
||||
if etype == "video":
|
||||
vs = _lookup_video(t.marker_id, videos)
|
||||
if vs is not None:
|
||||
pres = resolve_video_presentation(
|
||||
t.marker_id,
|
||||
vs,
|
||||
t.overrides,
|
||||
default_end_on=(
|
||||
None if t.marker_id.startswith("narration:") else "next_video"
|
||||
),
|
||||
)
|
||||
e["handle"] = pres["handle"]
|
||||
e["cutout"] = pres["cutout"]
|
||||
e["layer"] = pres["layer"]
|
||||
e["end_on"] = pres["end_on"]
|
||||
if pres["take"] is not None:
|
||||
e["take"] = pres["take"]
|
||||
pd = _pause_duration(t.marker_id, videos)
|
||||
if pd:
|
||||
e["pause_duration"] = pd
|
||||
@@ -289,12 +325,16 @@ def events_to_marker_timings(events: list[dict]) -> list[MarkerTiming]:
|
||||
for e in events:
|
||||
n = e.get("narration_time")
|
||||
eff = (n + e.get("adjustment", 0.0)) if n is not None else -1.0
|
||||
# Carry any stored presentation as overrides so render honors the atomic event
|
||||
# (e.g. a GUI edit) over the shorthand/videos.json default. Absent on old events.
|
||||
overrides = {k: e[k] for k in _PRESENTATION_KEYS if k in e} or None
|
||||
timings.append(
|
||||
MarkerTiming(
|
||||
marker_id=e["id"],
|
||||
timestamp=eff,
|
||||
context=e.get("context", ""),
|
||||
confidence=float(e.get("confidence", 1.0)),
|
||||
overrides=overrides,
|
||||
)
|
||||
)
|
||||
return timings
|
||||
|
||||
+175
-57
@@ -22,7 +22,7 @@ from .models import (
|
||||
VideoEvent,
|
||||
VideoSource,
|
||||
)
|
||||
from .parser import get_video_duration, resolve_missing_videos
|
||||
from .parser import get_video_duration, resolve_missing_videos, parse_marker
|
||||
from .transcriber import TranscribedWord
|
||||
|
||||
# Audio trigger offset: play sound this many seconds before the marker
|
||||
@@ -54,6 +54,59 @@ _SHORTHAND_PREFIXES: dict[str, tuple] = {
|
||||
"vsmp:": ("square", "mid"),
|
||||
}
|
||||
|
||||
# Cutout zone the narration talking-head defaults to when a segment/source has none.
|
||||
# Matches the convention import uses (cli._import_narration_segments writes this).
|
||||
_NARRATION_CUTOUT = "talkinghead"
|
||||
|
||||
|
||||
def resolve_video_presentation(
|
||||
marker_id: str,
|
||||
video_source,
|
||||
overrides: Optional[dict] = None,
|
||||
default_end_on: Optional[str] = "next_video",
|
||||
) -> dict:
|
||||
"""Resolve a video marker's per-occurrence presentation to an atomic dict.
|
||||
|
||||
Single source of truth for how a marker becomes an event, shared by the render
|
||||
transformer (_extract_video_events) and the scaffold that materializes events.json.
|
||||
Presentation is per-occurrence: the shorthand prefix (vst: → square/above) decides
|
||||
cutout+layer for THIS marker, so one handle can appear as vst: and vsb: without a
|
||||
videos.json collision.
|
||||
|
||||
Precedence per field: explicit event override > shorthand prefix > videos.json
|
||||
default > built-in default. `default_end_on` is "next_video" for video triggers and
|
||||
None for [narration:] (which runs to the end).
|
||||
|
||||
Returns {handle, cutout, layer, end_on, take, pause_narration}.
|
||||
"""
|
||||
overrides = overrides or {}
|
||||
|
||||
prefix = next((p for p in _SHORTHAND_PREFIXES if marker_id.startswith(p)), None)
|
||||
if prefix is not None:
|
||||
handle = marker_id[len(prefix):].lower()
|
||||
impl_cutout, impl_layer = _SHORTHAND_PREFIXES[prefix]
|
||||
else:
|
||||
# Legacy [video:X] / [narration:X] — strip the generic prefix if present.
|
||||
handle = (marker_id.split(":", 1)[1] if ":" in marker_id else marker_id).lower()
|
||||
impl_cutout = impl_layer = None
|
||||
|
||||
cutout = overrides.get("cutout") or impl_cutout or video_source.cutout
|
||||
layer = overrides.get("layer") or impl_layer or video_source.layer
|
||||
end_on = overrides.get("end_on") or video_source.end_on or default_end_on
|
||||
take = overrides["take"] if "take" in overrides else video_source.take
|
||||
pause_narration = overrides.get(
|
||||
"pause_narration", video_source.pause_narration or 0.0
|
||||
)
|
||||
|
||||
return {
|
||||
"handle": handle,
|
||||
"cutout": cutout,
|
||||
"layer": layer,
|
||||
"end_on": end_on,
|
||||
"take": take,
|
||||
"pause_narration": float(pause_narration or 0.0),
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class MarkerTiming:
|
||||
@@ -63,6 +116,9 @@ class MarkerTiming:
|
||||
timestamp: float # -1 if not found
|
||||
context: str # the text following the marker
|
||||
confidence: float # 0-1, how confident the match is
|
||||
# Per-occurrence presentation overrides carried from events.json (GUI edits) so
|
||||
# render honors them over the shorthand/videos.json defaults. None on fresh align.
|
||||
overrides: Optional[dict] = None
|
||||
|
||||
|
||||
def _normalize_text(text: str) -> str:
|
||||
@@ -226,7 +282,10 @@ def _extract_marker_contexts(
|
||||
|
||||
raw_contexts = []
|
||||
for i in range(1, len(parts), 2):
|
||||
marker_id = parts[i]
|
||||
# Split the bracket into a bare marker id and any inline overrides
|
||||
# ([vsb:clip, end_on=next_video]); the id drives alignment, the overrides ride
|
||||
# along to the event.
|
||||
marker_id, overrides = parse_marker(parts[i])
|
||||
|
||||
if not _is_known_marker(marker_id, slides, videos, audio):
|
||||
continue
|
||||
@@ -240,7 +299,7 @@ def _extract_marker_contexts(
|
||||
j += 1
|
||||
if j >= len(parts):
|
||||
break
|
||||
if _is_known_marker(parts[j], slides, videos, audio):
|
||||
if _is_known_marker(parse_marker(parts[j])[0], slides, videos, audio):
|
||||
break
|
||||
j += 1
|
||||
|
||||
@@ -248,22 +307,22 @@ def _extract_marker_contexts(
|
||||
following_text = " ".join(following_text.split())
|
||||
following_text = _strip_unknown_markers(following_text, slides, videos, audio)
|
||||
following_text = " ".join(following_text.split())
|
||||
raw_contexts.append((marker_id, following_text))
|
||||
raw_contexts.append((marker_id, following_text, overrides))
|
||||
|
||||
contexts = []
|
||||
for i, (marker_id, following_text) in enumerate(raw_contexts):
|
||||
for i, (marker_id, following_text, overrides) in enumerate(raw_contexts):
|
||||
if following_text:
|
||||
words = following_text.split()[:10]
|
||||
contexts.append((marker_id, " ".join(words), False, "before"))
|
||||
contexts.append((marker_id, " ".join(words), False, "before", overrides))
|
||||
else:
|
||||
borrowed = False
|
||||
for j in range(i + 1, len(raw_contexts)):
|
||||
next_marker_id, next_text = raw_contexts[j]
|
||||
next_marker_id, next_text, _ = raw_contexts[j]
|
||||
if next_text:
|
||||
if next_marker_id in (slides or {}):
|
||||
break
|
||||
words = next_text.split()[:10]
|
||||
contexts.append((marker_id, " ".join(words), True, "before"))
|
||||
contexts.append((marker_id, " ".join(words), True, "before", overrides))
|
||||
borrowed = True
|
||||
break
|
||||
if not borrowed:
|
||||
@@ -278,9 +337,9 @@ def _extract_marker_contexts(
|
||||
if preceding_text:
|
||||
words = preceding_text.split()
|
||||
tail = " ".join(words[-6:])
|
||||
contexts.append((marker_id, tail, False, "after"))
|
||||
contexts.append((marker_id, tail, False, "after", overrides))
|
||||
else:
|
||||
contexts.append((marker_id, "", False, "before"))
|
||||
contexts.append((marker_id, "", False, "before", overrides))
|
||||
|
||||
return contexts
|
||||
|
||||
@@ -484,7 +543,7 @@ def align_markers_to_transcription(
|
||||
last_idx = 0
|
||||
last_end_time = 0.0
|
||||
|
||||
for marker_id, anchor_text, is_borrowed, anchor_type in contexts:
|
||||
for marker_id, anchor_text, is_borrowed, anchor_type, overrides in contexts:
|
||||
if not anchor_text.strip():
|
||||
marker_time = last_end_time + 1.0
|
||||
timings.append(
|
||||
@@ -493,6 +552,7 @@ def align_markers_to_transcription(
|
||||
timestamp=marker_time,
|
||||
context="(after previous)",
|
||||
confidence=1.0,
|
||||
overrides=overrides,
|
||||
)
|
||||
)
|
||||
last_end_time = marker_time
|
||||
@@ -517,6 +577,7 @@ def align_markers_to_transcription(
|
||||
timestamp=marker_time,
|
||||
context=f"(end of: {anchor_text[:40]})",
|
||||
confidence=confidence,
|
||||
overrides=overrides,
|
||||
)
|
||||
)
|
||||
last_idx = match_end_idx
|
||||
@@ -529,6 +590,7 @@ def align_markers_to_transcription(
|
||||
timestamp=adjusted_time,
|
||||
context=anchor_text[:50],
|
||||
confidence=confidence,
|
||||
overrides=overrides,
|
||||
)
|
||||
)
|
||||
if not is_borrowed:
|
||||
@@ -544,6 +606,7 @@ def align_markers_to_transcription(
|
||||
timestamp=-1.0,
|
||||
context=anchor_text[:50],
|
||||
confidence=0.0,
|
||||
overrides=overrides,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -557,7 +620,7 @@ def align_markers_to_transcription(
|
||||
if timing.timestamp >= 0:
|
||||
continue
|
||||
|
||||
marker_id, anchor_text, is_borrowed, anchor_type = contexts[i]
|
||||
marker_id, anchor_text, is_borrowed, anchor_type, overrides = contexts[i]
|
||||
if not anchor_text.strip():
|
||||
continue
|
||||
|
||||
@@ -612,6 +675,7 @@ def align_markers_to_transcription(
|
||||
timestamp=marker_time,
|
||||
context=f"(repaired: {anchor_text[:40]})",
|
||||
confidence=confidence,
|
||||
overrides=overrides,
|
||||
)
|
||||
|
||||
# Deduplicate slide markers. The manuscript pattern [SN]\n\n[SN] text... is
|
||||
@@ -733,8 +797,9 @@ def build_render_plan(
|
||||
# sum of the segment durations and skip is already baked into each segment.
|
||||
if narration_schedule:
|
||||
narration_video_id = "narration"
|
||||
# cutout left unset — resolved to the talkinghead zone below (single source).
|
||||
narration_video = narration_source or VideoSource(
|
||||
source_file="", cutout=config.default_slide_type, always_visible=True
|
||||
source_file="", always_visible=True
|
||||
)
|
||||
narration_skip = 0.0
|
||||
full_duration = sum(seg.duration for seg in narration_schedule)
|
||||
@@ -770,7 +835,20 @@ def build_render_plan(
|
||||
if timing.timestamp >= 0:
|
||||
marker_times[timing.marker_id] = timing.timestamp
|
||||
|
||||
cutout = config.cutouts[narration_video.cutout]
|
||||
# Narration talking-head cutout. Narration segments carry no cutout of their own,
|
||||
# so default to the dedicated "talkinghead" zone (the convention import uses when it
|
||||
# creates narration entries — see cli._import_narration_segments), falling back to
|
||||
# the project's default slide type only if that zone isn't defined. Guard the lookup
|
||||
# so a misconfigured cutout gives a clear error instead of a bare KeyError.
|
||||
narration_cutout_name = narration_video.cutout or (
|
||||
_NARRATION_CUTOUT if _NARRATION_CUTOUT in config.cutouts else config.default_slide_type
|
||||
)
|
||||
if narration_cutout_name not in config.cutouts:
|
||||
raise ValueError(
|
||||
f"Narration cutout '{narration_cutout_name}' not found in project cutouts "
|
||||
f"{list(config.cutouts)}"
|
||||
)
|
||||
cutout = config.cutouts[narration_cutout_name]
|
||||
# Adjust duration for skip (content starts at skip, so effective duration is less)
|
||||
effective_duration = full_duration - narration_skip
|
||||
narration_videos: list[tuple[str, VideoSource, CutoutDefinition]] = [
|
||||
@@ -1194,14 +1272,10 @@ def _extract_video_events(
|
||||
marker_timings, slides, total_duration
|
||||
)
|
||||
|
||||
# Pause-variant prefixes — the only thing the render pass still needs from
|
||||
# shorthand markers at event-build time (pause_narration is per-event, not stored in videos.json).
|
||||
_PAUSE_PREFIXES = {"vftp:", "vfbp:", "vfmp:", "vf2tp:", "vf2bp:", "vf2mp:", "vstp:", "vsbp:", "vsmp:"}
|
||||
|
||||
# Collect video markers: (time, video_id, event_type, pause_narration)
|
||||
# video_markers: (timestamp, video_id, marker_type, pause_narration)
|
||||
# cutout and layer are read from videos.json (projected there by _project_markers_to_videos)
|
||||
video_markers: list[tuple[float, str, str, bool]] = []
|
||||
# Collect video markers. Carry the full marker_id (so presentation resolves from
|
||||
# its shorthand prefix PER occurrence) and any per-event overrides from events.json.
|
||||
# video_markers: (timestamp, marker_id, handle, trigger_type, overrides)
|
||||
video_markers: list[tuple[float, str, str, str, Optional[dict]]] = []
|
||||
|
||||
for timing in marker_timings:
|
||||
if timing.timestamp < 0:
|
||||
@@ -1229,8 +1303,9 @@ def _extract_video_events(
|
||||
f"run render once to project values, or set cutout manually."
|
||||
)
|
||||
continue
|
||||
pause_narration = shorthand_match in _PAUSE_PREFIXES
|
||||
video_markers.append((timing.timestamp, video_id, "video", pause_narration))
|
||||
video_markers.append(
|
||||
(timing.timestamp, mid, video_id, "video", timing.overrides)
|
||||
)
|
||||
continue
|
||||
|
||||
# --- legacy [video:xxx] ---
|
||||
@@ -1247,7 +1322,9 @@ def _extract_video_events(
|
||||
f"[video:{video_id}] has no valid cutout in videos.json — skipped."
|
||||
)
|
||||
continue
|
||||
video_markers.append((timing.timestamp, video_id, "video", False))
|
||||
video_markers.append(
|
||||
(timing.timestamp, mid, video_id, "video", timing.overrides)
|
||||
)
|
||||
continue
|
||||
|
||||
# --- [narration:xxx] ---
|
||||
@@ -1264,29 +1341,41 @@ def _extract_video_events(
|
||||
f"[narration:{video_id}] has no valid cutout in videos.json — skipped."
|
||||
)
|
||||
continue
|
||||
video_markers.append((timing.timestamp, video_id, "narration", False))
|
||||
video_markers.append(
|
||||
(timing.timestamp, mid, video_id, "narration", timing.overrides)
|
||||
)
|
||||
|
||||
# Sorted start times of all video markers — used by end_on="next_video" to cap
|
||||
# a clip when the next video begins, so videos never overlap.
|
||||
video_start_times = sorted(t for t, _, _, _ in video_markers)
|
||||
video_start_times = sorted(t for t, *_ in video_markers)
|
||||
|
||||
events: list[VideoEvent] = []
|
||||
for start_time, video_id, marker_type, pause_narration in video_markers:
|
||||
for start_time, marker_id, video_id, trigger_type, overrides in video_markers:
|
||||
video_source = videos[video_id]
|
||||
|
||||
# Read cutout and layer directly from videos.json (projected by ETL)
|
||||
cutout_name = video_source.cutout
|
||||
# Resolve presentation per-occurrence: shorthand prefix (and any events.json
|
||||
# override) wins over the videos.json default, so one handle can render above
|
||||
# in one place and below in another. [narration:] runs to the end by default.
|
||||
pres = resolve_video_presentation(
|
||||
marker_id,
|
||||
video_source,
|
||||
overrides,
|
||||
default_end_on=(None if trigger_type == "narration" else "next_video"),
|
||||
)
|
||||
cutout_name = pres["cutout"]
|
||||
cutout = cutouts[cutout_name]
|
||||
layer = video_source.layer
|
||||
layer = pres["layer"]
|
||||
end_on = pres["end_on"]
|
||||
take = pres["take"]
|
||||
pause_narration = pres["pause_narration"]
|
||||
|
||||
end_on = video_source.end_on
|
||||
if end_on == "take" and video_source.take is not None:
|
||||
end_time = start_time + video_source.take
|
||||
if end_on == "take" and take is not None:
|
||||
end_time = start_time + take
|
||||
elif end_on == "end":
|
||||
# Play the clip once through its natural length, then stop — no looping.
|
||||
# Natural length = explicit take, else the file's own duration past skip.
|
||||
if video_source.take is not None:
|
||||
natural = video_source.take
|
||||
if take is not None:
|
||||
natural = take
|
||||
elif video_source.duration is not None:
|
||||
natural = max(0.0, video_source.duration - (video_source.skip or 0.0))
|
||||
else:
|
||||
@@ -1304,9 +1393,9 @@ def _extract_video_events(
|
||||
end_time = vt
|
||||
break
|
||||
# A pause-narration video must stay for at least the pause it holds.
|
||||
if video_source.pause_narration:
|
||||
end_time = max(end_time, start_time + video_source.pause_narration)
|
||||
elif end_on in ("next_slide", "slide") or (end_on is None and marker_type == "video"):
|
||||
if pause_narration:
|
||||
end_time = max(end_time, start_time + pause_narration)
|
||||
elif end_on in ("next_slide", "slide"):
|
||||
# End at next slide marker ("slide" is a recognised alias for "next_slide")
|
||||
end_time = total_duration
|
||||
for slide_time in slide_times:
|
||||
@@ -1315,21 +1404,31 @@ def _extract_video_events(
|
||||
break
|
||||
# pause_narration videos must stay visible for the full pause duration —
|
||||
# the narration is held for that long, so the overlay should match.
|
||||
if video_source.pause_narration:
|
||||
end_time = max(end_time, start_time + video_source.pause_narration)
|
||||
if pause_narration:
|
||||
end_time = max(end_time, start_time + pause_narration)
|
||||
else:
|
||||
# end_on is None and marker_type == "narration": runs to end
|
||||
# end_on None ([narration:] with no explicit end) — runs to end.
|
||||
end_time = total_duration
|
||||
|
||||
# Filter by time range.
|
||||
# CHUNKING v1 LIMITATION: this keeps only clips whose START is inside the
|
||||
# window, so a clip that began in an earlier chunk and is still playing
|
||||
# across the boundary is DROPPED here — the chunked output then differs from
|
||||
# a full render. cli._chunk_boundary_span_warnings surfaces this. The v2 fix
|
||||
# is overlap-inclusion + a per-event seek (skip_override); see
|
||||
# docs/chunking_v2.md.
|
||||
if start_time < range_start or start_time >= range_end:
|
||||
# Filter by time range — CHUNKING v2 (docs/chunking_v2.md).
|
||||
# Include any clip that OVERLAPS the window (not just those starting inside
|
||||
# it), so a clip spanning a chunk boundary survives into the later chunk.
|
||||
if end_time <= range_start or start_time >= range_end:
|
||||
continue
|
||||
# A clip that began before this window is already mid-playback at the seam;
|
||||
# seek into it so it resumes at the right frame instead of restarting.
|
||||
skip_override = None
|
||||
if start_time < range_start:
|
||||
into = range_start - start_time # elapsed since the clip started
|
||||
base = video_source.skip or 0.0
|
||||
playable = (video_source.duration - base) if video_source.duration else None
|
||||
if playable and playable > 0 and into >= playable:
|
||||
# the clip has looped by the window start → resume at the loop phase
|
||||
skip_override = base + (into % playable)
|
||||
else:
|
||||
# still within the first play-through (or unknown length) → linear seek
|
||||
skip_override = base + into
|
||||
start_time = range_start # -> 0 after time_offset subtraction
|
||||
end_time = min(end_time, range_end)
|
||||
|
||||
events.append(
|
||||
@@ -1341,6 +1440,7 @@ def _extract_video_events(
|
||||
cutout=cutout,
|
||||
cutout_name=cutout_name,
|
||||
layer=layer,
|
||||
skip_override=skip_override,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -1367,17 +1467,35 @@ def _extract_audio_events(
|
||||
elif marker_id.startswith("audio:"):
|
||||
audio_id = marker_id[6:]
|
||||
if audio_id is not None and audio_id in audio:
|
||||
# CHUNKING v1 LIMITATION (same as video, worse for looping background
|
||||
# music): a clip started before the window is dropped from later chunks.
|
||||
# v2 = overlap-inclusion + audio seek; see docs/chunking_v2.md.
|
||||
if timing.timestamp < range_start or timing.timestamp >= range_end:
|
||||
adef = audio[audio_id]
|
||||
astart = max(0, timing.timestamp - AUDIO_OFFSET_SECONDS)
|
||||
# Effective end of this clip on the output timeline.
|
||||
if adef.loop:
|
||||
aend = range_end # a loop fills to the window/render end
|
||||
elif adef.duration is not None:
|
||||
aend = astart + adef.duration
|
||||
else:
|
||||
aend = float("inf") # unknown one-shot length — assume it may span
|
||||
# CHUNKING v2 (docs/chunking_v2.md): include if it OVERLAPS the window,
|
||||
# and seek into clips that began earlier so they resume mid-track — the
|
||||
# loop phase for looping music, a linear seek for one-shots. v1 dropped
|
||||
# these, silencing looping background music in every chunk but the first.
|
||||
if aend <= range_start or astart >= range_end:
|
||||
continue
|
||||
start_time = max(0, timing.timestamp - AUDIO_OFFSET_SECONDS)
|
||||
src_offset = 0.0
|
||||
if astart < range_start:
|
||||
into = range_start - astart
|
||||
if adef.loop and adef.duration:
|
||||
src_offset = into % adef.duration
|
||||
else:
|
||||
src_offset = into
|
||||
astart = range_start
|
||||
events.append(
|
||||
AudioEvent(
|
||||
audio_id=audio_id,
|
||||
start_time=start_time,
|
||||
audio_def=audio[audio_id],
|
||||
start_time=astart,
|
||||
audio_def=adef,
|
||||
src_offset=src_offset,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@@ -58,6 +58,10 @@ def validate_project(
|
||||
elif marker.startswith("narration:"):
|
||||
referenced_video_ids.add(marker[10:].lower())
|
||||
|
||||
# (Key-reuse across cutout/layer is legal: presentation is resolved per-occurrence
|
||||
# from the shorthand prefix now — see transformer.resolve_video_presentation — so
|
||||
# one handle can appear as vst: (above) and vsb: (below) without colliding.)
|
||||
|
||||
# Check for malformed markers first (these are likely typos)
|
||||
if malformed_markers:
|
||||
for line_num, marker_text in malformed_markers:
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Plan-level validation for chunking v2 (docs/chunking_v2.md).
|
||||
|
||||
Verifies the transformer now INCLUDES clips that span a chunk boundary and seeks
|
||||
into them (skip_override / src_offset), instead of the v1 behaviour that dropped
|
||||
them. This is a pure plan-level check — the ffmpeg concat-seam (frame alignment via
|
||||
-c copy) still needs a real render on the rig to confirm.
|
||||
|
||||
Run: ./venv/bin/python tests/test_chunking_v2.py
|
||||
"""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from gnommo.transformer import (
|
||||
MarkerTiming,
|
||||
_extract_audio_events,
|
||||
_extract_video_events,
|
||||
AUDIO_OFFSET_SECONDS,
|
||||
)
|
||||
from gnommo.models import AudioDefinition, VideoSource, CutoutDefinition, SlideDefinition
|
||||
|
||||
_fails = []
|
||||
|
||||
|
||||
def check(name, cond, detail=""):
|
||||
print(f" {'PASS' if cond else 'FAIL'} {name}" + (f" — {detail}" if detail and not cond else ""))
|
||||
if not cond:
|
||||
_fails.append(name)
|
||||
|
||||
|
||||
# ── audio ────────────────────────────────────────────────────────────────────
|
||||
def test_audio():
|
||||
print("audio events:")
|
||||
audio = {
|
||||
"music": AudioDefinition(file="music.mp3", loop=True, duration=90.0),
|
||||
"sfx": AudioDefinition(file="sfx.wav", loop=False, duration=500.0),
|
||||
"blip": AudioDefinition(file="blip.wav", loop=False, duration=50.0),
|
||||
}
|
||||
# music triggers at t=0, sfx at t=10, blip at t=10
|
||||
markers = [
|
||||
MarkerTiming(marker_id="Amusic", timestamp=0.0, context="", confidence=1.0),
|
||||
MarkerTiming(marker_id="Asfx", timestamp=10.0, context="", confidence=1.0),
|
||||
MarkerTiming(marker_id="Ablip", timestamp=10.0, context="", confidence=1.0),
|
||||
]
|
||||
|
||||
# Chunk window [300, 600): all three started earlier.
|
||||
evs = {e.audio_id: e for e in _extract_audio_events(markers, audio, time_range=(300.0, 600.0))}
|
||||
|
||||
# Looping music: included, resumes at loop phase. astart = max(0, 0-1)=0; into=300;
|
||||
# phase = 300 % 90 = 30.
|
||||
check("looping music spanning boundary is INCLUDED (v1 dropped it)", "music" in evs)
|
||||
if "music" in evs:
|
||||
m = evs["music"]
|
||||
check("music clamped to window start", abs(m.start_time - 300.0) < 1e-6, f"start={m.start_time}")
|
||||
check("music seeks to loop phase 30.0", abs(m.src_offset - 30.0) < 1e-6, f"src_offset={m.src_offset}")
|
||||
|
||||
# One-shot still playing at the window: included, linear seek.
|
||||
# astart = max(0,10-1)=9; aend=9+500=509 > 300 → spans. into=300-9=291.
|
||||
check("one-shot still playing is INCLUDED", "sfx" in evs)
|
||||
if "sfx" in evs:
|
||||
s = evs["sfx"]
|
||||
check("sfx linear seek 291.0", abs(s.src_offset - 291.0) < 1e-6, f"src_offset={s.src_offset}")
|
||||
|
||||
# One-shot that ended before the window: excluded (aend=9+50=59 < 300).
|
||||
check("one-shot ended before window is EXCLUDED", "blip" not in evs)
|
||||
|
||||
# Full render (no range): everything from the start, no seek.
|
||||
full = {e.audio_id: e for e in _extract_audio_events(markers, audio, time_range=None)}
|
||||
check("full render includes music with no seek", "music" in full and full["music"].src_offset == 0.0)
|
||||
|
||||
|
||||
# ── video ────────────────────────────────────────────────────────────────────
|
||||
def test_video():
|
||||
print("video events:")
|
||||
cutouts = {"fullscreen": CutoutDefinition(x=0, y=0, height=1080, width=1920)}
|
||||
videos = {
|
||||
"bg": VideoSource(
|
||||
source_file="bg.mp4", cutout="fullscreen", layer="below",
|
||||
duration=90.0, skip=0.0, end_on="next_video",
|
||||
)
|
||||
}
|
||||
slides = {f"S{i}": SlideDefinition(image=f"S{i}.png", type="slide") for i in range(1, 11)}
|
||||
markers = [MarkerTiming(marker_id=f"S{i}", timestamp=(i - 1) * 60.0, context="", confidence=1.0)
|
||||
for i in range(1, 11)]
|
||||
# Background overlay starts at slide-7 time (360) and, as the only video with
|
||||
# end_on next_video, runs to total_duration (600).
|
||||
markers.append(MarkerTiming(marker_id="vfm:bg", timestamp=360.0, context="", confidence=1.0))
|
||||
|
||||
total = 600.0
|
||||
# Chunk window that STARTS AFTER the video began: [420, 600) (slides 8-10).
|
||||
evs, _ = _extract_video_events(markers, videos, cutouts, slides, total, time_range=(420.0, 600.0))
|
||||
bg = next((e for e in evs if e.video_id == "bg"), None)
|
||||
|
||||
check("spanning background video is INCLUDED in the later chunk (v1 dropped it)", bg is not None)
|
||||
if bg is not None:
|
||||
check("bg clamped to window start", abs(bg.start_time - 420.0) < 1e-6, f"start={bg.start_time}")
|
||||
# into = 420-360 = 60; playable = 90-0 = 90; 60 < 90 → linear seek 60.
|
||||
check("bg linear seek 60.0 (first play-through)", abs((bg.skip_override or 0) - 60.0) < 1e-6,
|
||||
f"skip_override={bg.skip_override}")
|
||||
|
||||
# Window starting deep enough that the 90s clip has looped once: [480, 600).
|
||||
# into = 480-360 = 120; 120 >= 90 → phase = 120 % 90 = 30.
|
||||
evs2, _ = _extract_video_events(markers, videos, cutouts, slides, total, time_range=(480.0, 600.0))
|
||||
bg2 = next((e for e in evs2 if e.video_id == "bg"), None)
|
||||
check("looped background resumes at phase 30.0", bg2 is not None and abs((bg2.skip_override or 0) - 30.0) < 1e-6,
|
||||
f"skip_override={getattr(bg2, 'skip_override', None)}")
|
||||
|
||||
# Full render: no seek.
|
||||
evs3, _ = _extract_video_events(markers, videos, cutouts, slides, total, time_range=None)
|
||||
bg3 = next((e for e in evs3 if e.video_id == "bg"), None)
|
||||
check("full render includes bg with no seek", bg3 is not None and bg3.skip_override is None)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
test_audio()
|
||||
test_video()
|
||||
print()
|
||||
if _fails:
|
||||
print(f"FAILED: {len(_fails)} check(s): {', '.join(_fails)}")
|
||||
sys.exit(1)
|
||||
print("All chunking-v2 plan-level checks passed.")
|
||||
Reference in New Issue
Block a user