Compare commits

..
3 Commits
Author SHA1 Message Date
gitprov 0c8f662bee Adding fixes to the render system 2026-07-27 19:16:02 +02:00
gitprov cdda2e9024 Adding fix to recover narration.json 2026-07-27 10:48:44 +02:00
gitprovandClaude Opus 4.8 a8aab55bd2 Chunking v2: keep clips that span a chunk boundary (overlap + seek)
v1 dropped any overlay video or audio clip whose start fell outside a chunk's
window, so a multi-slide background video — or looping background music — silently
vanished from every chunk after the one it started in, diverging from a full render.

v2 includes any clip that OVERLAPS the window and seeks into clips that began
earlier so they resume mid-clip at the seam instead of restarting:
- _extract_video_events: overlap test + VideoEvent.skip_override (loop-aware phase);
  the input -ss also seeks the clip's embedded audio (tvaud).
- _extract_audio_events: overlap test + AudioEvent.src_offset (loop phase / linear);
  renderer honors it in the loop-with-pauses, standard-loop, and one-shot paths.
- _chunk_boundary_span_warnings downgraded from "will be dropped" to informational.

Full (non-chunked) render is unchanged: time_range=None leaves skip_override/
src_offset at defaults. Slides already used overlap; outros are last-chunk-only.

Plan-level checks in tests/test_chunking_v2.py (all green). Remaining gate: render-
seam frame/phase validation on the rig; crossfade-loop audio seek deferred. See
docs/chunking_v2.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 12:35:09 +02:00
10 changed files with 671 additions and 131 deletions
+165
View File
@@ -0,0 +1,165 @@
# Atomic Events — Design Spec
Status: **Stage A + Stage B implemented (2026-07-27).** Motivated by a future **Glitch
Studio GUI** that edits each video occurrence as a self-contained object.
Implemented:
- Per-occurrence presentation resolves via `transformer.resolve_video_presentation`
(precedence: inline/GUI override > shorthand prefix > videos.json > default; video
`end_on` default = `next_video`, `[narration:]` runs to end).
- events.json is materialized/atomic: `derive_events` writes `handle/cutout/layer/end_on/
take`; `events_to_marker_timings` round-trips them as overrides.
- Inline grammar `[prefix:handle, key=value, …]` (`parser.parse_marker`), threaded through
alignment into `MarkerTiming.overrides`. Supported inline keys: **cutout, layer, end_on,
take** (the fully-wired per-event fields). Unknown keys are ignored.
- The key-reuse collision validator hard-error was removed (reuse is legal now).
Deferred (follow-ups): inline override of the *global* params (skip/zoom/volume/
use_audio_channels/pause_narration) — the renderer reads these from `video_source` in ~13
places, so wiring them per-event is a separate change; stripping the moved fields from
videos.json (kept as fallback defaults for now); a validator warning for unknown/unwired
inline keys.
## Problem
Presentation/timing properties (`cutout`, `layer`, `end_on`, `take`, `pause_narration`)
live on the **videos.json handle**, but they are really properties of *where a clip is
used*, not of the file. The shorthand prefix (`vst:` = square/above, `vsb:` =
square/below) is per-marker, but `_project_markers_to_videos` collapses it onto the
single handle record (last-wins). So one handle used two ways collides:
- `[vst:glitch_ccd_binning]` (above) and `[vsb:glitch_ccd_binning]` (below) → videos.json
can only store `layer: below`, so the first occurrence renders under the slide (hidden).
- video5 has 5 such collisions today (glitch_ccd_binning, pexels/12471039…,
mainvideopart1, shotnoiseacc, slide_periodogram).
A stopgap validator hard-error (`validate_project`, gnommo/validator.py) currently blocks
render on these. This spec removes the *cause* so that guard is no longer needed.
The naive fixes are both rejected: copying the file/handle (duplication on disk), and a
"hybrid override + materialize" layer (too much indirection). Instead: **the per-occurrence
properties move onto the event.**
## Field homes
**videos.json — asset + global defaults (one value per handle):**
`source_file`, `output_file`/`processed_file`, `filter`, `has_audio`, `is_shared`,
`src_mtime`, `duration` (probed; asset-only, never per-event), and the globals
`zoom`, `skip`, `volume`, `use_audio_channels`.
**events.json — per-occurrence (one value per event):**
`handle` (the video id, **prefix-free**), `cutout`, `layer`, `end_on`, `take`,
`pause_narration`.
**Resolution order for a rendered clip:** event field (if set) → videos.json value (for the
globals) → config default. The per-occurrence fields have no videos.json fallback — they
are always materialized onto the event at build time.
Notes:
- `end_on` **defaults to `next_video`** for videos when unspecified (was implicitly
`next_slide`). Existing videos.json `end_on` values are migrated onto events explicitly,
so current projects keep their behavior; only *new* unspecified markers get the new default.
- `take` is the event-level cut length, only meaningful when `end_on=take`; otherwise the
end is implicit from `end_on` and `take` stays null.
- `skip` stays a global (asset trim-in) while `take` is per-event — a deliberate asymmetry:
"where this asset generally starts" vs. "how long this occurrence plays."
- `zoom`/`volume`/`use_audio_channels` stay global but are inline-overridable per event
(below), so they can diverge without a videos.json copy.
## Authoring: shorthand + inline overloads
The manuscript stays the compact authoring surface. The shorthand letters encode
`cutout`+`layer` (and `pause_narration` via the `…p:` variants). Anything the letters
don't encode — chiefly `end_on`, and any per-event override of a global — is given as
inline **`key=value`** pairs (simplified from the earlier `{"json":"form"}`):
```
[vsb:glitch_ccd_binning2] # square/below, end_on defaults to next_video
[vsb:glitch_ccd_binning2, end_on=next_video] # + explicit end_on
[vsb:glitch_ccd_binning2, take=5, volume=0.5] # + per-event overrides of globals
[video:glitch_ccd_binning2, cutout=square, layer=below] # generic; equivalent to [vsb:…]
```
Rules:
- The first token inside `[]` is `prefix:handle` (handle may contain `/`, e.g. `pexels/123`).
- Remaining comma-separated tokens are `key=value`. Values are type-inferred: numeric →
float, `true`/`false` → bool, else string. Allowed keys: `cutout`, `layer`, `end_on`,
`take`, `skip`, `zoom`, `volume`, `use_audio_channels`, `pause_narration`,
`always_visible`.
- An inline key overrides whatever the shorthand implied (e.g. `[vst:x, layer=below]` →
above from the prefix, then below from the override). Last-writer-wins, prefix first.
- `[video:handle, …]` is the fully-explicit form the GUI round-trips: no prefix magic, every
presentation field named.
Why `key=value` over JSON: no braces/quotes to escape inside `[]`, one obvious separator,
and it reads cleanly in a script. The GUI still stores the resolved values as real JSON
fields on the event — the manuscript form is just sugar that populates them.
## Build-time materialization
At build (`build_render_plan` / scaffold construction), each video marker resolves to an
atomic event dict:
```json
{
"type": "video",
"handle": "glitch_ccd_binning",
"cutout": "square",
"layer": "above",
"end_on": "next_video",
"take": null,
"pause_narration": 0.0,
"narration_time": 0.0, "adjustment": 0.0, "final_time": 0.0,
"mapping": "exact", "confidence": 1.0, "context": "…"
}
```
`id` (currently `"vst:glitch_ccd_binning"`) is replaced by `handle` + explicit fields. The
render pass reads presentation straight off the event and no longer consults the prefix or
the videos.json presentation fields. `merge_events` must preserve manual event edits (the
GUI's writes) across rebuilds, the same way it preserves `adjustment` today.
## Code touchpoints
- **models.py** — `VideoSource` sheds `cutout`/`layer`/`end_on`/`take`/`pause_narration`
(or they become defaults-only); `VideoEvent` already carries `cutout`/`layer`/`end_on` —
extend to `take`/`pause_narration` sourced from the event, not the handle.
- **parser.py `parse_manuscript`** — extend the marker grammar to accept
`prefix:handle, key=value, …`; update the malformed-marker detector (which today flags
spaces/commas inside `[]`).
- **transformer.py `_extract_video_events`** — resolve `cutout/layer/end_on/take/
pause_narration` from (prefix inline overrides), not from `video_source`.
- **scaffold.py** — event schema: `handle` + presentation fields; `merge_events` preserves
GUI edits; migration for existing events.json.
- **cli.py** — retire `_project_markers_to_videos` and `_writeback_video_metadata` (they
project/writeback per-handle presentation) in favor of seeding event fields.
- **validator.py** — **remove** the key-reuse collision hard-error (reuse is legal now).
- **renderer.py** — read presentation from the event (mostly already does via `VideoEvent`).
## Migration
Existing projects (video0video6, …) have presentation on the handle and prefixed `id`s in
events.json. A one-shot migration, run on build:
1. For each video event, split the prefixed `id` into `handle` + implied `cutout`/`layer`.
2. Fill `end_on`/`take`/`pause_narration` from the handle's current videos.json values
(preserving today's behavior — including handles that explicitly set `next_slide`).
3. Strip the moved fields from videos.json handles (leave the globals).
4. Idempotent: a second run is a no-op once events carry `handle`.
## Staging
- **Stage A** — schema split + per-event resolution from the shorthand prefix, migration,
remove the collision guard. Shorthand-only authoring keeps working; the 5 video5
collisions resolve. (This is the part that fixes the bug.)
- **Stage B** — the inline `key=value` overload grammar + malformed-marker updates.
Keep the validator collision hard-error in place **until Stage A lands** — removing it
earlier would let the hidden-overlay bug back in on video5.
## Open questions
- `always_visible`, `use_audio_channels`: confirmed as inline-overridable globals — do any
need to become fully per-event?
- Does the GUI want events fully flattened (every field present) or sparse (only overrides,
inherit the rest)? Affects whether the build writes defaults explicitly.
+25 -12
View File
@@ -1,6 +1,9 @@
# Chunked Rendering v2 — Design Spec # Chunked Rendering v2 — Design Spec
Status: **planned** (v1 shipped; v1 boundary limitation is detected + warned, not yet fixed) Status: **implemented on branch `chunking-v2`, pending render-seam validation on the rig.**
Plan-level logic is covered by `tests/test_chunking_v2.py` (all green). What remains
is confirming the ffmpeg concat seam is frame/phase-accurate on a real render — see
"Concat-seam correctness" below.
## Why chunking exists ## Why chunking exists
@@ -113,16 +116,26 @@ via `skip_override`. Validation plan:
## Work items ## Work items
- [ ] `_extract_video_events`: overlap test + `skip_override` (loop-aware). - [x] `_extract_video_events`: overlap test + `skip_override` (loop-aware).
- [ ] `_extract_audio_events`: add `end_time`/`skip_override`, overlap + seek. - [x] `_extract_audio_events`: overlap + `src_offset` seek (loop phase / linear).
- [ ] `AudioEvent`: `skip_override` field; renderer audio path honors it. - [x] `AudioEvent`: `src_offset` field; renderer audio paths (loop-with-pauses,
- [ ] `OutroEvent`: same treatment. standard loop, one-shot) honor it.
- [ ] Frame/audio seam-diff test (chunked vs full) in the test suite. - [x] `VideoEvent.skip_override`: renderer video input `-ss` honors it; the clip's
- [ ] Flip `_chunk_boundary_span_warnings` from "will be dropped" to a debug-only embedded audio (`tvaud`) is seeked automatically by the same input seek.
assertion once v2 is the default. - [x] `_chunk_boundary_span_warnings`: downgraded from the "will be dropped" v1
warning to an informational note (logged; terminal only under `--verbose`).
- [x] Plan-level tests: `tests/test_chunking_v2.py`.
- [ ] **Render-seam validation on the rig** (chunked-vs-full frame/audio diff) — the
remaining gate before making v2 the trusted default.
- [ ] Crossfade-loop audio (`_build_crossfade_loop_filter`) does not yet apply
`src_offset` — a crossfaded looping bed restarts phase at the seam. Standard
(non-crossfade) loops and one-shots are handled. Low priority.
- [ ] `OutroEvent`: not needed — outros are extracted for the last chunk only
(`config.outro if is_last_chunk`), so they never split across a seam.
## Seams already in place (v2 preparations, shipped) ## Not affected (verified)
- `VideoEvent.skip_override` (models.py) — inert; renderer honors it when set. - Slides: `_extract_slide_events` already used overlap+clamp — no change.
- `cli._chunk_boundary_span_warnings` — detection + warning. - Full-screen `plan.background`: a separate always-included input.
- v1-limitation comments at both filter sites in `transformer.py`. - Full (non-chunked) render: `time_range=None` path leaves `skip_override`/
`src_offset` at their defaults, so output is byte-identical to before.
+37 -47
View File
@@ -707,20 +707,17 @@ def cmd_import(project_path: Path, force: bool, verbose: bool) -> int:
if len(keynote_files) > 1: if len(keynote_files) > 1:
print(f" Warning: Multiple .key files found, using {keynote_file.name}") print(f" Warning: Multiple .key files found, using {keynote_file.name}")
_import_presenter_notes(project_path, keynote_file, verbose, config) _import_presenter_notes(project_path, keynote_file, verbose, config)
elif verbose:
# Generate slides.json for each slide directory (after Keynote export)
slides_base = project_path / "media" / "slides"
slides_dirs = (
[d for d in slides_base.glob("*/") if d.is_dir()]
if slides_base.exists()
else []
)
for slides_dir in slides_dirs:
_generate_slides_json(slides_dir, verbose)
else:
if verbose:
print(" No .key file found, skipping presenter notes import") print(" No .key file found, skipping presenter notes import")
# Generate slides.json for this project's slide export dir only. Slides always
# export to media/slides/<project>/ (consistent across projects); sibling folders
# like spec/, prompt/, render/ are auxiliary AI-workflow files, not slide images,
# so scanning them just produced spurious "No image files" warnings.
project_slides_dir = project_path / "media" / "slides" / project_path.name.lower()
if project_slides_dir.is_dir():
_generate_slides_json(project_slides_dir, verbose)
# Import shared assets (pexels, etc.) from shared_assets directory # Import shared assets (pexels, etc.) from shared_assets directory
# Look for shared_assets relative to project or in parent directories # Look for shared_assets relative to project or in parent directories
shared_assets_dir = _find_shared_assets(project_path) shared_assets_dir = _find_shared_assets(project_path)
@@ -1317,8 +1314,9 @@ def _generate_slides_json(directory: Path, verbose: bool) -> None:
# each new videos.json entry (and backfilled onto existing ones) so nothing is # each new videos.json entry (and backfilled onto existing ones) so nothing is
# an invisible optional. Auto-managed fields (output_file, duration, has_audio, # an invisible optional. Auto-managed fields (output_file, duration, has_audio,
# is_shared, attribution) are intentionally omitted — they're set by the tool. # is_shared, attribution) are intentionally omitted — they're set by the tool.
# end_on defaults to null = "use the marker-type default" (video → next_slide, # end_on defaults to null = "use the marker-type default" (video → next_video,
# narration → play to end); set it to "next_slide", "end", or "take" explicitly. # narration → play to end); set it to "next_slide", "next_video", "loop", "end", or
# "take" explicitly (resolved per-event by transformer.resolve_video_presentation).
_VIDEO_DEFAULTS = { _VIDEO_DEFAULTS = {
"cutout": "square", # named zone from project.json cutouts "cutout": "square", # named zone from project.json cutouts
"layer": "above", # above | mid | below (relative to slides/narrator) "layer": "above", # above | mid | below (relative to slides/narrator)
@@ -1330,7 +1328,7 @@ _VIDEO_DEFAULTS = {
"use_audio_channels": "both", # both | left | right "use_audio_channels": "both", # both | left | right
"always_visible": False, # always on screen (like the talking head) "always_visible": False, # always on screen (like the talking head)
"pause_narration": 0.0, # seconds to freeze narration for a cutscene "pause_narration": 0.0, # seconds to freeze narration for a cutscene
"end_on": None, # null | next_slide | end | take "end_on": None, # null | next_slide | next_video | loop | end | take
} }
@@ -4137,16 +4135,14 @@ def _writeback_video_metadata(plan, project_path, config) -> None:
def _chunk_boundary_span_warnings(plan, groups) -> list[str]: def _chunk_boundary_span_warnings(plan, groups) -> list[str]:
"""Detect events that span a chunk boundary. """Report clips that span a chunk boundary.
KNOWN LIMITATION (chunking v1): the render's time-range filter keeps only events Chunking v2 (docs/chunking_v2.md) INCLUDES these in the later chunk and seeks
whose *start* falls inside a chunk's window (transformer.py `_extract_video_events` into them (VideoEvent.skip_override / AudioEvent.src_offset) so they resume
/ `_extract_audio_events`: `if start_time < range_start: continue`). An overlay mid-clip instead of being dropped. The seam then relies on `-c copy` joining
video or audio clip that began in an earlier chunk and is still playing across the frame-aligned chunks, which is the one thing worth eyeballing — so this stays as
boundary is therefore DROPPED from every later chunk it overlaps — so the chunked an informational list (logged; shown on the terminal only with --verbose), not
output silently diverges from a full render. This detector makes that non-silent. the hard "will be dropped" warning of v1.
The real fix (overlap-inclusion + per-event seek) is chunking v2 — see
docs/chunking_v2.md.
""" """
slide_start = {e.slide_id: e.start_time for e in plan.slide_events} slide_start = {e.slide_id: e.start_time for e in plan.slide_events}
boundaries = [] # (slide_id, output_time) at each chunk seam after the first boundaries = [] # (slide_id, output_time) at each chunk seam after the first
@@ -4166,8 +4162,8 @@ def _chunk_boundary_span_warnings(plan, groups) -> list[str]:
extra = f" (and {len(crossed) - 1} more)" if len(crossed) > 1 else "" extra = f" (and {len(crossed) - 1} more)" if len(crossed) > 1 else ""
warnings.append( warnings.append(
f"{kind} '{label}' plays {_format_time(start)}{_format_time(end)} and " f"{kind} '{label}' plays {_format_time(start)}{_format_time(end)} and "
f"crosses the chunk boundary at {sid} ({_format_time(bt)}){extra}; it will " f"crosses the chunk boundary at {sid} ({_format_time(bt)}){extra}; v2 "
f"be DROPPED from the chunk(s) after the boundary." f"seeks into it so it continues across the seam."
) )
for e in plan.video_events: for e in plan.video_events:
@@ -4211,29 +4207,23 @@ def _chunked_render(
f"\n Auto-chunking: {len(slide_ids)} slides → {len(groups)} chunks of ≤{chunk_size}" f"\n Auto-chunking: {len(slide_ids)} slides → {len(groups)} chunks of ≤{chunk_size}"
) )
# Warn about events that span a chunk boundary — chunking v1 drops these from # Report clips that span a chunk boundary. v2 seeks into them so they continue
# the later chunk, so the output would silently differ from a full render. # across the seam (no longer dropped); this is informational — logged always,
# and echoed to the terminal only under --verbose — with a nudge to eyeball the
# seam since concat uses -c copy.
if plan is not None: if plan is not None:
_span_warnings = _chunk_boundary_span_warnings(plan, groups) _span_notes = _chunk_boundary_span_warnings(plan, groups)
if _span_warnings: if _span_notes:
banner = " " + "!" * 64 _render_log(f"chunking v2: {len(_span_notes)} clip(s) span a boundary (seam-seeked):")
print(f"\n{banner}", file=sys.stderr) for w in _span_notes:
print(" CHUNK-BOUNDARY WARNING — chunked output will differ from a full render:",
file=sys.stderr)
for w in _span_warnings:
print(f" - {w}", file=sys.stderr)
print(" These overlay video/audio clips span a chunk seam. Chunking v1 keeps",
file=sys.stderr)
print(" only clips that START inside a chunk, so the ones above vanish after",
file=sys.stderr)
print(" the boundary. Options: render this project WITHOUT chunking (lower",
file=sys.stderr)
print(" cpu_limit instead), or move the chunk size so no seam splits them.",
file=sys.stderr)
print(f"{banner}\n", file=sys.stderr)
_render_log("CHUNK-BOUNDARY WARNING (v1 drops cross-boundary clips):")
for w in _span_warnings:
_render_log(f" - {w}") _render_log(f" - {w}")
if verbose:
print(f"\n {len(_span_notes)} clip(s) span a chunk boundary — v2 seeks across the seam:",
file=sys.stderr)
for w in _span_notes:
print(f" - {w}", file=sys.stderr)
print(" If a seam looks off, verify frame alignment (concat uses -c copy).",
file=sys.stderr)
chunks_dir = out_dir / "chunks" chunks_dir = out_dir / "chunks"
chunks_dir.mkdir(parents=True, exist_ok=True) chunks_dir.mkdir(parents=True, exist_ok=True)
+5
View File
@@ -416,6 +416,11 @@ class AudioEvent:
audio_id: str audio_id: str
start_time: float # When to start playing (marker time - offset) start_time: float # When to start playing (marker time - offset)
audio_def: AudioDefinition audio_def: AudioDefinition
# Chunking v2 (docs/chunking_v2.md): when a clip began before this chunk's
# window it must resume mid-track, not restart at the seam. src_offset is the
# position (seconds) into the source stream to begin at — the loop phase for
# looping music, or a linear seek for one-shots. 0.0 = play from the start (v1).
src_offset: float = 0.0
@dataclass @dataclass
+80 -5
View File
@@ -54,6 +54,55 @@ def _resolve_case_insensitive(path: Path) -> Path:
return resolved return resolved
# Inline marker-override keys honored at build time. These are the per-occurrence
# presentation fields materialized onto events.json and resolved per-event
# (transformer.resolve_video_presentation). Global params (skip/zoom/volume/…) are not
# yet overridable inline; unknown keys are ignored.
_MARKER_OVERRIDE_KEYS = frozenset({"cutout", "layer", "end_on", "take"})
def _coerce_marker_value(key: str, raw: str):
"""Coerce an inline override value: `take` → float; everything else → string."""
v = raw.strip().strip('"').strip("'")
if key == "take":
try:
return float(v)
except ValueError:
return None
return v
def parse_marker(raw: str) -> "tuple[str, Optional[dict]]":
"""Split a bracket's contents into (marker_id, overrides).
Supports the inline-overload grammar `[prefix:handle, key=value, key=value]`:
the text before the first comma is the marker id (prefix:handle); the rest are
comma-separated key=value overrides. Only _MARKER_OVERRIDE_KEYS are kept (others
ignored). A plain marker (no comma) returns (marker_id, None).
Examples:
"vsb:clip" -> ("vsb:clip", None)
"vsb:clip, end_on=next_video" -> ("vsb:clip", {"end_on": "next_video"})
"video:clip, cutout=square, layer=below"
-> ("video:clip", {"cutout": "square", "layer": "below"})
"""
if "," not in raw:
return raw.strip(), None
head, _, tail = raw.partition(",")
marker_id = head.strip()
overrides: dict = {}
for tok in tail.split(","):
if "=" not in tok:
continue
key, _, val = tok.partition("=")
key = key.strip().lower()
if key in _MARKER_OVERRIDE_KEYS:
coerced = _coerce_marker_value(key, val)
if coerced is not None:
overrides[key] = coerced
return marker_id, (overrides or None)
def parse_manuscript( def parse_manuscript(
project_path: Path, project_path: Path,
) -> tuple[str, list[str], list[tuple[int, str]], list[Citation]]: ) -> tuple[str, list[str], list[tuple[int, str]], list[Citation]]:
@@ -86,9 +135,12 @@ def parse_manuscript(
text = re.sub(r"\[pause\]", "", text) text = re.sub(r"\[pause\]", "", text)
text = re.sub(r"\[stop\]", "", text) text = re.sub(r"\[stop\]", "", text)
# Extract all valid markers like [S1], [video:demo], [vf2m:pexels/clip-name], etc. # Extract all valid markers like [S1], [video:demo], [vf2m:pexels/clip-name], and
# Include / and - to capture pexels/library video IDs; . to catch file extensions in markers. # inline-override forms like [vsb:clip, end_on=next_video]. Include / and - for
markers = re.findall(r"\[([A-Za-z0-9_:./\-]+)\]", text) # pexels/library video IDs; . for file extensions; an optional ",…" tail carries
# per-event overrides (parsed out by parse_marker, so `markers` holds bare ids).
raw_markers = re.findall(r"\[([A-Za-z0-9_:./\-]+(?:,[^\]\n]*)?)\]", text)
markers = [parse_marker(m)[0] for m in raw_markers]
# Find malformed markers (missing brackets, extra spaces, etc.) # Find malformed markers (missing brackets, extra spaces, etc.)
malformed: list[tuple[int, str]] = [] malformed: list[tuple[int, str]] = []
@@ -625,12 +677,35 @@ def parse_narration(
default_filters = config.default_filters if config else {} default_filters = config.default_filters if config else {}
narration = {} narration = {}
_narr_video_exts = {".mov", ".mp4", ".webm", ".avi", ".mkv", ".m4v"}
for segment_id, segment_data in data.items(): for segment_id, segment_data in data.items():
if "source_file" not in segment_data: if not segment_data.get("source_file"):
# source_file can drift out of an entry (case/sync churn between
# machines). Recover the way import/prune/trim do: find the raw
# recording whose stem matches the segment id, case-insensitively.
recovered = None
for sub in ("raw_mov", "processed"):
sub_dir = narration_dir / sub
if not sub_dir.is_dir():
continue
for f in sorted(sub_dir.iterdir()):
if (
f.is_file()
and f.suffix.lower() in _narr_video_exts
and f.stem.lower() == segment_id.lower()
):
recovered = f"{sub}/{f.name}"
break
if recovered:
break
if recovered is None:
raise ParseError( raise ParseError(
f"Narration segment '{segment_id}' missing required field 'source_file'", f"Narration segment '{segment_id}' missing required field 'source_file' "
f"and no matching recording was found in raw_mov/ or processed/. "
f"Add a 'source_file' or run 'import' to repair narration.json.",
narration_path, narration_path,
) )
segment_data = {**segment_data, "source_file": recovered}
# Resolve filter - can be a list or a string reference to default_filters # Resolve filter - can be a list or a string reference to default_filters
filter_value = segment_data.get("filter", []) filter_value = segment_data.get("filter", [])
+12 -5
View File
@@ -1473,7 +1473,9 @@ def build_filter_complex(
for p in plan.narration_pauses for p in plan.narration_pauses
if p.output_time > event.start_time if p.output_time > event.start_time
] ]
src_pos = 0.0 # Chunking v2: start partway into the looped stream when the
# clip began in an earlier chunk (docs/chunking_v2.md).
src_pos = getattr(event, "src_offset", 0.0)
seg_start = event.start_time seg_start = event.start_time
seg_count = 0 seg_count = 0
@@ -1530,21 +1532,26 @@ def build_filter_complex(
) )
filters.extend(crossfade_filters) filters.extend(crossfade_filters)
else: else:
# Standard loop without crossfade # Standard loop without crossfade. Chunking v2: seek to
# the loop phase when the clip began in an earlier chunk.
_off = getattr(event, "src_offset", 0.0)
filters.append( filters.append(
f"[{audio_idx}:a]aloop=loop=-1:size=2e+09," f"[{audio_idx}:a]aloop=loop=-1:size=2e+09,"
f"atrim=0:{remaining:.3f}," f"atrim={_off:.3f}:{_off + remaining:.3f},"
f"asetpts=PTS-STARTPTS," f"asetpts=PTS-STARTPTS,"
f"adelay={delay_ms}|{delay_ms}," f"adelay={delay_ms}|{delay_ms},"
f"volume={volume:.2f}[{label}]" f"volume={volume:.2f}[{label}]"
) )
audio_labels_to_mix.append(f"[{label}]") audio_labels_to_mix.append(f"[{label}]")
else: else:
# One-shot audio: delay to trigger time # One-shot audio: delay to trigger time. Chunking v2: seek in if
# the clip began in an earlier chunk (docs/chunking_v2.md).
label = f"aud{i}" label = f"aud{i}"
delay_ms = int(event.start_time * 1000) delay_ms = int(event.start_time * 1000)
_off = getattr(event, "src_offset", 0.0)
_seek = f"atrim={_off:.3f},asetpts=PTS-STARTPTS," if _off > 0 else ""
filters.append( filters.append(
f"[{audio_idx}:a]adelay={delay_ms}|{delay_ms},volume={volume:.2f}[{label}]" f"[{audio_idx}:a]{_seek}adelay={delay_ms}|{delay_ms},volume={volume:.2f}[{label}]"
) )
audio_labels_to_mix.append(f"[{label}]") audio_labels_to_mix.append(f"[{label}]")
+42 -2
View File
@@ -28,7 +28,11 @@ from pathlib import Path
from typing import Optional from typing import Optional
from .models import CAMERA_PRESETS from .models import CAMERA_PRESETS
from .transformer import MarkerTiming from .transformer import MarkerTiming, resolve_video_presentation
# Per-occurrence presentation fields materialized onto video events (atomic events.json,
# GUI-ready). Round-tripped as overrides so a stored value drives render over the default.
_PRESENTATION_KEYS = ("cutout", "layer", "end_on", "take")
EVENTS_FILE = "events.json" EVENTS_FILE = "events.json"
SCAFFOLD_FILE = "scaffold.json" SCAFFOLD_FILE = "scaffold.json"
@@ -97,6 +101,16 @@ _PAUSE_MARKER_PREFIXES = (
) )
def _lookup_video(marker_id: str, videos: dict):
"""Case-insensitive videos.json lookup for a video marker (prefix stripped)."""
if not videos:
return None
handle = marker_id.split(":", 1)[1].lower() if ":" in marker_id else marker_id.lower()
return videos.get(handle) or next(
(v for k, v in videos.items() if k.lower() == handle), None
)
def _pause_duration(marker_id: str, videos: dict) -> float: def _pause_duration(marker_id: str, videos: dict) -> float:
"""Seconds a pause-variant video marker freezes the narration for, else 0.""" """Seconds a pause-variant video marker freezes the narration for, else 0."""
if not marker_id.startswith(_PAUSE_MARKER_PREFIXES): if not marker_id.startswith(_PAUSE_MARKER_PREFIXES):
@@ -138,8 +152,9 @@ def derive_events(
if (placed and t.confidence >= _EXACT_THRESHOLD and not after_prev) if (placed and t.confidence >= _EXACT_THRESHOLD and not after_prev)
else MAPPING_INTERPOLATED else MAPPING_INTERPOLATED
) )
etype = marker_type(t.marker_id, slides, videos, audio)
e = { e = {
"type": marker_type(t.marker_id, slides, videos, audio), "type": etype,
"id": t.marker_id, "id": t.marker_id,
"narration_time": round(t.timestamp, 3) if placed else None, "narration_time": round(t.timestamp, 3) if placed else None,
"adjustment": 0.0, "adjustment": 0.0,
@@ -148,6 +163,27 @@ def derive_events(
"confidence": round(t.confidence, 3), "confidence": round(t.confidence, 3),
"context": (t.context or "")[:80], "context": (t.context or "")[:80],
} }
# Materialize per-occurrence presentation onto video events so events.json is
# atomic (each occurrence self-contained, GUI-editable) rather than depending on
# the handle's videos.json entry. Resolved from the shorthand prefix + any prior
# override the marker carried.
if etype == "video":
vs = _lookup_video(t.marker_id, videos)
if vs is not None:
pres = resolve_video_presentation(
t.marker_id,
vs,
t.overrides,
default_end_on=(
None if t.marker_id.startswith("narration:") else "next_video"
),
)
e["handle"] = pres["handle"]
e["cutout"] = pres["cutout"]
e["layer"] = pres["layer"]
e["end_on"] = pres["end_on"]
if pres["take"] is not None:
e["take"] = pres["take"]
pd = _pause_duration(t.marker_id, videos) pd = _pause_duration(t.marker_id, videos)
if pd: if pd:
e["pause_duration"] = pd e["pause_duration"] = pd
@@ -289,12 +325,16 @@ def events_to_marker_timings(events: list[dict]) -> list[MarkerTiming]:
for e in events: for e in events:
n = e.get("narration_time") n = e.get("narration_time")
eff = (n + e.get("adjustment", 0.0)) if n is not None else -1.0 eff = (n + e.get("adjustment", 0.0)) if n is not None else -1.0
# Carry any stored presentation as overrides so render honors the atomic event
# (e.g. a GUI edit) over the shorthand/videos.json default. Absent on old events.
overrides = {k: e[k] for k in _PRESENTATION_KEYS if k in e} or None
timings.append( timings.append(
MarkerTiming( MarkerTiming(
marker_id=e["id"], marker_id=e["id"],
timestamp=eff, timestamp=eff,
context=e.get("context", ""), context=e.get("context", ""),
confidence=float(e.get("confidence", 1.0)), confidence=float(e.get("confidence", 1.0)),
overrides=overrides,
) )
) )
return timings return timings
+175 -57
View File
@@ -22,7 +22,7 @@ from .models import (
VideoEvent, VideoEvent,
VideoSource, VideoSource,
) )
from .parser import get_video_duration, resolve_missing_videos from .parser import get_video_duration, resolve_missing_videos, parse_marker
from .transcriber import TranscribedWord from .transcriber import TranscribedWord
# Audio trigger offset: play sound this many seconds before the marker # Audio trigger offset: play sound this many seconds before the marker
@@ -54,6 +54,59 @@ _SHORTHAND_PREFIXES: dict[str, tuple] = {
"vsmp:": ("square", "mid"), "vsmp:": ("square", "mid"),
} }
# Cutout zone the narration talking-head defaults to when a segment/source has none.
# Matches the convention import uses (cli._import_narration_segments writes this).
_NARRATION_CUTOUT = "talkinghead"
def resolve_video_presentation(
marker_id: str,
video_source,
overrides: Optional[dict] = None,
default_end_on: Optional[str] = "next_video",
) -> dict:
"""Resolve a video marker's per-occurrence presentation to an atomic dict.
Single source of truth for how a marker becomes an event, shared by the render
transformer (_extract_video_events) and the scaffold that materializes events.json.
Presentation is per-occurrence: the shorthand prefix (vst: square/above) decides
cutout+layer for THIS marker, so one handle can appear as vst: and vsb: without a
videos.json collision.
Precedence per field: explicit event override > shorthand prefix > videos.json
default > built-in default. `default_end_on` is "next_video" for video triggers and
None for [narration:] (which runs to the end).
Returns {handle, cutout, layer, end_on, take, pause_narration}.
"""
overrides = overrides or {}
prefix = next((p for p in _SHORTHAND_PREFIXES if marker_id.startswith(p)), None)
if prefix is not None:
handle = marker_id[len(prefix):].lower()
impl_cutout, impl_layer = _SHORTHAND_PREFIXES[prefix]
else:
# Legacy [video:X] / [narration:X] — strip the generic prefix if present.
handle = (marker_id.split(":", 1)[1] if ":" in marker_id else marker_id).lower()
impl_cutout = impl_layer = None
cutout = overrides.get("cutout") or impl_cutout or video_source.cutout
layer = overrides.get("layer") or impl_layer or video_source.layer
end_on = overrides.get("end_on") or video_source.end_on or default_end_on
take = overrides["take"] if "take" in overrides else video_source.take
pause_narration = overrides.get(
"pause_narration", video_source.pause_narration or 0.0
)
return {
"handle": handle,
"cutout": cutout,
"layer": layer,
"end_on": end_on,
"take": take,
"pause_narration": float(pause_narration or 0.0),
}
@dataclass @dataclass
class MarkerTiming: class MarkerTiming:
@@ -63,6 +116,9 @@ class MarkerTiming:
timestamp: float # -1 if not found timestamp: float # -1 if not found
context: str # the text following the marker context: str # the text following the marker
confidence: float # 0-1, how confident the match is confidence: float # 0-1, how confident the match is
# Per-occurrence presentation overrides carried from events.json (GUI edits) so
# render honors them over the shorthand/videos.json defaults. None on fresh align.
overrides: Optional[dict] = None
def _normalize_text(text: str) -> str: def _normalize_text(text: str) -> str:
@@ -226,7 +282,10 @@ def _extract_marker_contexts(
raw_contexts = [] raw_contexts = []
for i in range(1, len(parts), 2): for i in range(1, len(parts), 2):
marker_id = parts[i] # Split the bracket into a bare marker id and any inline overrides
# ([vsb:clip, end_on=next_video]); the id drives alignment, the overrides ride
# along to the event.
marker_id, overrides = parse_marker(parts[i])
if not _is_known_marker(marker_id, slides, videos, audio): if not _is_known_marker(marker_id, slides, videos, audio):
continue continue
@@ -240,7 +299,7 @@ def _extract_marker_contexts(
j += 1 j += 1
if j >= len(parts): if j >= len(parts):
break break
if _is_known_marker(parts[j], slides, videos, audio): if _is_known_marker(parse_marker(parts[j])[0], slides, videos, audio):
break break
j += 1 j += 1
@@ -248,22 +307,22 @@ def _extract_marker_contexts(
following_text = " ".join(following_text.split()) following_text = " ".join(following_text.split())
following_text = _strip_unknown_markers(following_text, slides, videos, audio) following_text = _strip_unknown_markers(following_text, slides, videos, audio)
following_text = " ".join(following_text.split()) following_text = " ".join(following_text.split())
raw_contexts.append((marker_id, following_text)) raw_contexts.append((marker_id, following_text, overrides))
contexts = [] contexts = []
for i, (marker_id, following_text) in enumerate(raw_contexts): for i, (marker_id, following_text, overrides) in enumerate(raw_contexts):
if following_text: if following_text:
words = following_text.split()[:10] words = following_text.split()[:10]
contexts.append((marker_id, " ".join(words), False, "before")) contexts.append((marker_id, " ".join(words), False, "before", overrides))
else: else:
borrowed = False borrowed = False
for j in range(i + 1, len(raw_contexts)): for j in range(i + 1, len(raw_contexts)):
next_marker_id, next_text = raw_contexts[j] next_marker_id, next_text, _ = raw_contexts[j]
if next_text: if next_text:
if next_marker_id in (slides or {}): if next_marker_id in (slides or {}):
break break
words = next_text.split()[:10] words = next_text.split()[:10]
contexts.append((marker_id, " ".join(words), True, "before")) contexts.append((marker_id, " ".join(words), True, "before", overrides))
borrowed = True borrowed = True
break break
if not borrowed: if not borrowed:
@@ -278,9 +337,9 @@ def _extract_marker_contexts(
if preceding_text: if preceding_text:
words = preceding_text.split() words = preceding_text.split()
tail = " ".join(words[-6:]) tail = " ".join(words[-6:])
contexts.append((marker_id, tail, False, "after")) contexts.append((marker_id, tail, False, "after", overrides))
else: else:
contexts.append((marker_id, "", False, "before")) contexts.append((marker_id, "", False, "before", overrides))
return contexts return contexts
@@ -484,7 +543,7 @@ def align_markers_to_transcription(
last_idx = 0 last_idx = 0
last_end_time = 0.0 last_end_time = 0.0
for marker_id, anchor_text, is_borrowed, anchor_type in contexts: for marker_id, anchor_text, is_borrowed, anchor_type, overrides in contexts:
if not anchor_text.strip(): if not anchor_text.strip():
marker_time = last_end_time + 1.0 marker_time = last_end_time + 1.0
timings.append( timings.append(
@@ -493,6 +552,7 @@ def align_markers_to_transcription(
timestamp=marker_time, timestamp=marker_time,
context="(after previous)", context="(after previous)",
confidence=1.0, confidence=1.0,
overrides=overrides,
) )
) )
last_end_time = marker_time last_end_time = marker_time
@@ -517,6 +577,7 @@ def align_markers_to_transcription(
timestamp=marker_time, timestamp=marker_time,
context=f"(end of: {anchor_text[:40]})", context=f"(end of: {anchor_text[:40]})",
confidence=confidence, confidence=confidence,
overrides=overrides,
) )
) )
last_idx = match_end_idx last_idx = match_end_idx
@@ -529,6 +590,7 @@ def align_markers_to_transcription(
timestamp=adjusted_time, timestamp=adjusted_time,
context=anchor_text[:50], context=anchor_text[:50],
confidence=confidence, confidence=confidence,
overrides=overrides,
) )
) )
if not is_borrowed: if not is_borrowed:
@@ -544,6 +606,7 @@ def align_markers_to_transcription(
timestamp=-1.0, timestamp=-1.0,
context=anchor_text[:50], context=anchor_text[:50],
confidence=0.0, confidence=0.0,
overrides=overrides,
) )
) )
@@ -557,7 +620,7 @@ def align_markers_to_transcription(
if timing.timestamp >= 0: if timing.timestamp >= 0:
continue continue
marker_id, anchor_text, is_borrowed, anchor_type = contexts[i] marker_id, anchor_text, is_borrowed, anchor_type, overrides = contexts[i]
if not anchor_text.strip(): if not anchor_text.strip():
continue continue
@@ -612,6 +675,7 @@ def align_markers_to_transcription(
timestamp=marker_time, timestamp=marker_time,
context=f"(repaired: {anchor_text[:40]})", context=f"(repaired: {anchor_text[:40]})",
confidence=confidence, confidence=confidence,
overrides=overrides,
) )
# Deduplicate slide markers. The manuscript pattern [SN]\n\n[SN] text... is # Deduplicate slide markers. The manuscript pattern [SN]\n\n[SN] text... is
@@ -733,8 +797,9 @@ def build_render_plan(
# sum of the segment durations and skip is already baked into each segment. # sum of the segment durations and skip is already baked into each segment.
if narration_schedule: if narration_schedule:
narration_video_id = "narration" narration_video_id = "narration"
# cutout left unset — resolved to the talkinghead zone below (single source).
narration_video = narration_source or VideoSource( narration_video = narration_source or VideoSource(
source_file="", cutout=config.default_slide_type, always_visible=True source_file="", always_visible=True
) )
narration_skip = 0.0 narration_skip = 0.0
full_duration = sum(seg.duration for seg in narration_schedule) full_duration = sum(seg.duration for seg in narration_schedule)
@@ -770,7 +835,20 @@ def build_render_plan(
if timing.timestamp >= 0: if timing.timestamp >= 0:
marker_times[timing.marker_id] = timing.timestamp marker_times[timing.marker_id] = timing.timestamp
cutout = config.cutouts[narration_video.cutout] # Narration talking-head cutout. Narration segments carry no cutout of their own,
# so default to the dedicated "talkinghead" zone (the convention import uses when it
# creates narration entries — see cli._import_narration_segments), falling back to
# the project's default slide type only if that zone isn't defined. Guard the lookup
# so a misconfigured cutout gives a clear error instead of a bare KeyError.
narration_cutout_name = narration_video.cutout or (
_NARRATION_CUTOUT if _NARRATION_CUTOUT in config.cutouts else config.default_slide_type
)
if narration_cutout_name not in config.cutouts:
raise ValueError(
f"Narration cutout '{narration_cutout_name}' not found in project cutouts "
f"{list(config.cutouts)}"
)
cutout = config.cutouts[narration_cutout_name]
# Adjust duration for skip (content starts at skip, so effective duration is less) # Adjust duration for skip (content starts at skip, so effective duration is less)
effective_duration = full_duration - narration_skip effective_duration = full_duration - narration_skip
narration_videos: list[tuple[str, VideoSource, CutoutDefinition]] = [ narration_videos: list[tuple[str, VideoSource, CutoutDefinition]] = [
@@ -1194,14 +1272,10 @@ def _extract_video_events(
marker_timings, slides, total_duration marker_timings, slides, total_duration
) )
# Pause-variant prefixes — the only thing the render pass still needs from # Collect video markers. Carry the full marker_id (so presentation resolves from
# shorthand markers at event-build time (pause_narration is per-event, not stored in videos.json). # its shorthand prefix PER occurrence) and any per-event overrides from events.json.
_PAUSE_PREFIXES = {"vftp:", "vfbp:", "vfmp:", "vf2tp:", "vf2bp:", "vf2mp:", "vstp:", "vsbp:", "vsmp:"} # video_markers: (timestamp, marker_id, handle, trigger_type, overrides)
video_markers: list[tuple[float, str, str, str, Optional[dict]]] = []
# Collect video markers: (time, video_id, event_type, pause_narration)
# video_markers: (timestamp, video_id, marker_type, pause_narration)
# cutout and layer are read from videos.json (projected there by _project_markers_to_videos)
video_markers: list[tuple[float, str, str, bool]] = []
for timing in marker_timings: for timing in marker_timings:
if timing.timestamp < 0: if timing.timestamp < 0:
@@ -1229,8 +1303,9 @@ def _extract_video_events(
f"run render once to project values, or set cutout manually." f"run render once to project values, or set cutout manually."
) )
continue continue
pause_narration = shorthand_match in _PAUSE_PREFIXES video_markers.append(
video_markers.append((timing.timestamp, video_id, "video", pause_narration)) (timing.timestamp, mid, video_id, "video", timing.overrides)
)
continue continue
# --- legacy [video:xxx] --- # --- legacy [video:xxx] ---
@@ -1247,7 +1322,9 @@ def _extract_video_events(
f"[video:{video_id}] has no valid cutout in videos.json — skipped." f"[video:{video_id}] has no valid cutout in videos.json — skipped."
) )
continue continue
video_markers.append((timing.timestamp, video_id, "video", False)) video_markers.append(
(timing.timestamp, mid, video_id, "video", timing.overrides)
)
continue continue
# --- [narration:xxx] --- # --- [narration:xxx] ---
@@ -1264,29 +1341,41 @@ def _extract_video_events(
f"[narration:{video_id}] has no valid cutout in videos.json — skipped." f"[narration:{video_id}] has no valid cutout in videos.json — skipped."
) )
continue continue
video_markers.append((timing.timestamp, video_id, "narration", False)) video_markers.append(
(timing.timestamp, mid, video_id, "narration", timing.overrides)
)
# Sorted start times of all video markers — used by end_on="next_video" to cap # Sorted start times of all video markers — used by end_on="next_video" to cap
# a clip when the next video begins, so videos never overlap. # a clip when the next video begins, so videos never overlap.
video_start_times = sorted(t for t, _, _, _ in video_markers) video_start_times = sorted(t for t, *_ in video_markers)
events: list[VideoEvent] = [] events: list[VideoEvent] = []
for start_time, video_id, marker_type, pause_narration in video_markers: for start_time, marker_id, video_id, trigger_type, overrides in video_markers:
video_source = videos[video_id] video_source = videos[video_id]
# Read cutout and layer directly from videos.json (projected by ETL) # Resolve presentation per-occurrence: shorthand prefix (and any events.json
cutout_name = video_source.cutout # override) wins over the videos.json default, so one handle can render above
# in one place and below in another. [narration:] runs to the end by default.
pres = resolve_video_presentation(
marker_id,
video_source,
overrides,
default_end_on=(None if trigger_type == "narration" else "next_video"),
)
cutout_name = pres["cutout"]
cutout = cutouts[cutout_name] cutout = cutouts[cutout_name]
layer = video_source.layer layer = pres["layer"]
end_on = pres["end_on"]
take = pres["take"]
pause_narration = pres["pause_narration"]
end_on = video_source.end_on if end_on == "take" and take is not None:
if end_on == "take" and video_source.take is not None: end_time = start_time + take
end_time = start_time + video_source.take
elif end_on == "end": elif end_on == "end":
# Play the clip once through its natural length, then stop — no looping. # Play the clip once through its natural length, then stop — no looping.
# Natural length = explicit take, else the file's own duration past skip. # Natural length = explicit take, else the file's own duration past skip.
if video_source.take is not None: if take is not None:
natural = video_source.take natural = take
elif video_source.duration is not None: elif video_source.duration is not None:
natural = max(0.0, video_source.duration - (video_source.skip or 0.0)) natural = max(0.0, video_source.duration - (video_source.skip or 0.0))
else: else:
@@ -1304,9 +1393,9 @@ def _extract_video_events(
end_time = vt end_time = vt
break break
# A pause-narration video must stay for at least the pause it holds. # A pause-narration video must stay for at least the pause it holds.
if video_source.pause_narration: if pause_narration:
end_time = max(end_time, start_time + video_source.pause_narration) end_time = max(end_time, start_time + pause_narration)
elif end_on in ("next_slide", "slide") or (end_on is None and marker_type == "video"): elif end_on in ("next_slide", "slide"):
# End at next slide marker ("slide" is a recognised alias for "next_slide") # End at next slide marker ("slide" is a recognised alias for "next_slide")
end_time = total_duration end_time = total_duration
for slide_time in slide_times: for slide_time in slide_times:
@@ -1315,21 +1404,31 @@ def _extract_video_events(
break break
# pause_narration videos must stay visible for the full pause duration — # pause_narration videos must stay visible for the full pause duration —
# the narration is held for that long, so the overlay should match. # the narration is held for that long, so the overlay should match.
if video_source.pause_narration: if pause_narration:
end_time = max(end_time, start_time + video_source.pause_narration) end_time = max(end_time, start_time + pause_narration)
else: else:
# end_on is None and marker_type == "narration": runs to end # end_on None ([narration:] with no explicit end) — runs to end.
end_time = total_duration end_time = total_duration
# Filter by time range. # Filter by time range — CHUNKING v2 (docs/chunking_v2.md).
# CHUNKING v1 LIMITATION: this keeps only clips whose START is inside the # Include any clip that OVERLAPS the window (not just those starting inside
# window, so a clip that began in an earlier chunk and is still playing # it), so a clip spanning a chunk boundary survives into the later chunk.
# across the boundary is DROPPED here — the chunked output then differs from if end_time <= range_start or start_time >= range_end:
# a full render. cli._chunk_boundary_span_warnings surfaces this. The v2 fix
# is overlap-inclusion + a per-event seek (skip_override); see
# docs/chunking_v2.md.
if start_time < range_start or start_time >= range_end:
continue continue
# A clip that began before this window is already mid-playback at the seam;
# seek into it so it resumes at the right frame instead of restarting.
skip_override = None
if start_time < range_start:
into = range_start - start_time # elapsed since the clip started
base = video_source.skip or 0.0
playable = (video_source.duration - base) if video_source.duration else None
if playable and playable > 0 and into >= playable:
# the clip has looped by the window start → resume at the loop phase
skip_override = base + (into % playable)
else:
# still within the first play-through (or unknown length) → linear seek
skip_override = base + into
start_time = range_start # -> 0 after time_offset subtraction
end_time = min(end_time, range_end) end_time = min(end_time, range_end)
events.append( events.append(
@@ -1341,6 +1440,7 @@ def _extract_video_events(
cutout=cutout, cutout=cutout,
cutout_name=cutout_name, cutout_name=cutout_name,
layer=layer, layer=layer,
skip_override=skip_override,
) )
) )
@@ -1367,17 +1467,35 @@ def _extract_audio_events(
elif marker_id.startswith("audio:"): elif marker_id.startswith("audio:"):
audio_id = marker_id[6:] audio_id = marker_id[6:]
if audio_id is not None and audio_id in audio: if audio_id is not None and audio_id in audio:
# CHUNKING v1 LIMITATION (same as video, worse for looping background adef = audio[audio_id]
# music): a clip started before the window is dropped from later chunks. astart = max(0, timing.timestamp - AUDIO_OFFSET_SECONDS)
# v2 = overlap-inclusion + audio seek; see docs/chunking_v2.md. # Effective end of this clip on the output timeline.
if timing.timestamp < range_start or timing.timestamp >= range_end: if adef.loop:
aend = range_end # a loop fills to the window/render end
elif adef.duration is not None:
aend = astart + adef.duration
else:
aend = float("inf") # unknown one-shot length — assume it may span
# CHUNKING v2 (docs/chunking_v2.md): include if it OVERLAPS the window,
# and seek into clips that began earlier so they resume mid-track — the
# loop phase for looping music, a linear seek for one-shots. v1 dropped
# these, silencing looping background music in every chunk but the first.
if aend <= range_start or astart >= range_end:
continue continue
start_time = max(0, timing.timestamp - AUDIO_OFFSET_SECONDS) src_offset = 0.0
if astart < range_start:
into = range_start - astart
if adef.loop and adef.duration:
src_offset = into % adef.duration
else:
src_offset = into
astart = range_start
events.append( events.append(
AudioEvent( AudioEvent(
audio_id=audio_id, audio_id=audio_id,
start_time=start_time, start_time=astart,
audio_def=audio[audio_id], audio_def=adef,
src_offset=src_offset,
) )
) )
+4
View File
@@ -58,6 +58,10 @@ def validate_project(
elif marker.startswith("narration:"): elif marker.startswith("narration:"):
referenced_video_ids.add(marker[10:].lower()) referenced_video_ids.add(marker[10:].lower())
# (Key-reuse across cutout/layer is legal: presentation is resolved per-occurrence
# from the shorthand prefix now — see transformer.resolve_video_presentation — so
# one handle can appear as vst: (above) and vsb: (below) without colliding.)
# Check for malformed markers first (these are likely typos) # Check for malformed markers first (these are likely typos)
if malformed_markers: if malformed_markers:
for line_num, marker_text in malformed_markers: for line_num, marker_text in malformed_markers:
+123
View File
@@ -0,0 +1,123 @@
#!/usr/bin/env python3
"""Plan-level validation for chunking v2 (docs/chunking_v2.md).
Verifies the transformer now INCLUDES clips that span a chunk boundary and seeks
into them (skip_override / src_offset), instead of the v1 behaviour that dropped
them. This is a pure plan-level check the ffmpeg concat-seam (frame alignment via
-c copy) still needs a real render on the rig to confirm.
Run: ./venv/bin/python tests/test_chunking_v2.py
"""
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from gnommo.transformer import (
MarkerTiming,
_extract_audio_events,
_extract_video_events,
AUDIO_OFFSET_SECONDS,
)
from gnommo.models import AudioDefinition, VideoSource, CutoutDefinition, SlideDefinition
_fails = []
def check(name, cond, detail=""):
print(f" {'PASS' if cond else 'FAIL'} {name}" + (f"{detail}" if detail and not cond else ""))
if not cond:
_fails.append(name)
# ── audio ────────────────────────────────────────────────────────────────────
def test_audio():
print("audio events:")
audio = {
"music": AudioDefinition(file="music.mp3", loop=True, duration=90.0),
"sfx": AudioDefinition(file="sfx.wav", loop=False, duration=500.0),
"blip": AudioDefinition(file="blip.wav", loop=False, duration=50.0),
}
# music triggers at t=0, sfx at t=10, blip at t=10
markers = [
MarkerTiming(marker_id="Amusic", timestamp=0.0, context="", confidence=1.0),
MarkerTiming(marker_id="Asfx", timestamp=10.0, context="", confidence=1.0),
MarkerTiming(marker_id="Ablip", timestamp=10.0, context="", confidence=1.0),
]
# Chunk window [300, 600): all three started earlier.
evs = {e.audio_id: e for e in _extract_audio_events(markers, audio, time_range=(300.0, 600.0))}
# Looping music: included, resumes at loop phase. astart = max(0, 0-1)=0; into=300;
# phase = 300 % 90 = 30.
check("looping music spanning boundary is INCLUDED (v1 dropped it)", "music" in evs)
if "music" in evs:
m = evs["music"]
check("music clamped to window start", abs(m.start_time - 300.0) < 1e-6, f"start={m.start_time}")
check("music seeks to loop phase 30.0", abs(m.src_offset - 30.0) < 1e-6, f"src_offset={m.src_offset}")
# One-shot still playing at the window: included, linear seek.
# astart = max(0,10-1)=9; aend=9+500=509 > 300 → spans. into=300-9=291.
check("one-shot still playing is INCLUDED", "sfx" in evs)
if "sfx" in evs:
s = evs["sfx"]
check("sfx linear seek 291.0", abs(s.src_offset - 291.0) < 1e-6, f"src_offset={s.src_offset}")
# One-shot that ended before the window: excluded (aend=9+50=59 < 300).
check("one-shot ended before window is EXCLUDED", "blip" not in evs)
# Full render (no range): everything from the start, no seek.
full = {e.audio_id: e for e in _extract_audio_events(markers, audio, time_range=None)}
check("full render includes music with no seek", "music" in full and full["music"].src_offset == 0.0)
# ── video ────────────────────────────────────────────────────────────────────
def test_video():
print("video events:")
cutouts = {"fullscreen": CutoutDefinition(x=0, y=0, height=1080, width=1920)}
videos = {
"bg": VideoSource(
source_file="bg.mp4", cutout="fullscreen", layer="below",
duration=90.0, skip=0.0, end_on="next_video",
)
}
slides = {f"S{i}": SlideDefinition(image=f"S{i}.png", type="slide") for i in range(1, 11)}
markers = [MarkerTiming(marker_id=f"S{i}", timestamp=(i - 1) * 60.0, context="", confidence=1.0)
for i in range(1, 11)]
# Background overlay starts at slide-7 time (360) and, as the only video with
# end_on next_video, runs to total_duration (600).
markers.append(MarkerTiming(marker_id="vfm:bg", timestamp=360.0, context="", confidence=1.0))
total = 600.0
# Chunk window that STARTS AFTER the video began: [420, 600) (slides 8-10).
evs, _ = _extract_video_events(markers, videos, cutouts, slides, total, time_range=(420.0, 600.0))
bg = next((e for e in evs if e.video_id == "bg"), None)
check("spanning background video is INCLUDED in the later chunk (v1 dropped it)", bg is not None)
if bg is not None:
check("bg clamped to window start", abs(bg.start_time - 420.0) < 1e-6, f"start={bg.start_time}")
# into = 420-360 = 60; playable = 90-0 = 90; 60 < 90 → linear seek 60.
check("bg linear seek 60.0 (first play-through)", abs((bg.skip_override or 0) - 60.0) < 1e-6,
f"skip_override={bg.skip_override}")
# Window starting deep enough that the 90s clip has looped once: [480, 600).
# into = 480-360 = 120; 120 >= 90 → phase = 120 % 90 = 30.
evs2, _ = _extract_video_events(markers, videos, cutouts, slides, total, time_range=(480.0, 600.0))
bg2 = next((e for e in evs2 if e.video_id == "bg"), None)
check("looped background resumes at phase 30.0", bg2 is not None and abs((bg2.skip_override or 0) - 30.0) < 1e-6,
f"skip_override={getattr(bg2, 'skip_override', None)}")
# Full render: no seek.
evs3, _ = _extract_video_events(markers, videos, cutouts, slides, total, time_range=None)
bg3 = next((e for e in evs3 if e.video_id == "bg"), None)
check("full render includes bg with no seek", bg3 is not None and bg3.skip_override is None)
if __name__ == "__main__":
test_audio()
test_video()
print()
if _fails:
print(f"FAILED: {len(_fails)} check(s): {', '.join(_fails)}")
sys.exit(1)
print("All chunking-v2 plan-level checks passed.")