gnommo/docs/chunking_v2.md

Chunked Rendering v2 — Design Spec

Status: implemented on branch chunking-v2, pending render-seam validation on the rig. Plan-level logic is covered by tests/test_chunking_v2.py (all green). What remains is confirming the ffmpeg concat seam is frame/phase-accurate on a real render — see "Concat-seam correctness" below.

Why chunking exists

The render is a single ffmpeg -filter_complex pass that opens every -i input up front. On memory-constrained machines (an 8 GB VM, or a WSL2 rig whose VM RAM is a slice of the host) the aggregate decoder + filter buffers OOM-kill the process — or, on WSL2, the whole VM. render_chunk_slides (in ~/.gnommo.conf [performance], or --chunk-slides N) splits the timeline into groups of N slides, renders each as a partial render, and concatenates the chunks with -c copy.

Each chunk is a partial render built via build_render_plan(slide_range=(start,end)), which passes time_range=(time_offset, render_end_time) to the event extractors. See partial-rendering-spec.md for the partial-render mechanics chunking reuses.

The v1 defect this fixes

_extract_video_events / _extract_audio_events keep an event only if its start falls inside the window:

if start_time < range_start or start_time >= range_end:
    continue

So a clip that began in an earlier chunk and is still playing across the boundary is dropped from every later chunk it overlaps. The full (non-chunked) render is correct; the chunked render silently diverges.

Who is affected:

• Overlay videos that span slides: end_on: next_video, loop, long take/end (multi-slide backgrounds / persistent picture-in-picture).
• Looping background audio/music: started once early, meant to underlie the whole video — dropped from every chunk after the first. Highest blast radius.
• Not affected: end_on: next_slide clips (they end exactly at a slide marker = a chunk seam), per-slide content, and the full-screen plan.background (a separate always-included input).

v1 status: cli._chunk_boundary_span_warnings(plan, groups) detects spanning video/outro/audio events and prints a loud warning before rendering, so the divergence is never silent. It does not yet correct the output.

v2 algorithm

1. Overlap inclusion (not start-inside)

Replace the start-inside test with an overlap test in both extractors:

# keep the event if it overlaps [range_start, range_end)
if end_time <= range_start or start_time >= range_end:
    continue

2. Per-event seek for clips that began earlier

A clip included by overlap whose start_time < range_start is already mid-playback at the chunk boundary. It must resume at the correct frame, not restart. Compute how far into the clip the window begins and carry it as a per-event seek:

into = range_start - start_time            # seconds of the clip already elapsed
base = video_source.skip or 0.0
# non-looping clip:
skip_override = base + into
# looping clip (end_on: loop / next_video that wraps):
period = (video_source.duration or 0) - base      # one loop's playable length
skip_override = base + (into % period) if period > 0 else base

Then clamp the event to the window and let the offset pass zero it:

start_time = max(start_time, range_start)   # -> 0 after time_offset subtraction
end_time   = min(end_time, range_end)

The seam for the seek is VideoEvent.skip_override (already added, inert until v2): the renderer prefers it over video_source.skip when set. -ss {skip} is applied as an input option, so ffmpeg decodes to that point — frame-accurate for the codecs in use.

3. Audio equivalent

AudioEvent has no skip_override / end_time yet. v2 adds both (or derives end from audio_def.duration / loop → total_duration) and applies the same overlap + seek. For looping music the seek is the loop-phase modulo above; ignore_pauses and overlap (crossfade) interactions must be re-checked at the seam.

4. Outro events

OutroEvent also spans (start_time/end_time). Extend the same treatment; outros normally live in the final chunk so this is lower priority but should be covered for completeness.

Concat-seam correctness (the risk to validate)

Chunks are joined with -c copy, so the two sides of a seam must be frame-aligned: chunk k ends showing the clip at position P, chunk k+1 must resume at exactly P via skip_override. Validation plan:

1. Pick a project with a known multi-slide overlay video and looping music (or synthesize one).
2. Render it full (reference) and chunked (small chunk_slides, so a seam falls mid clip).
3. Compare: identical duration; frame diff at ±3 frames around each seam below a threshold; audio cross-correlation shows no gap/jump; the overlay is present in every chunk it overlaps (the v1 bug is gone).
4. Assert _chunk_boundary_span_warnings returns empty for the fixed path.

Work items

• _extract_video_events: overlap test + skip_override (loop-aware).
• _extract_audio_events: overlap + src_offset seek (loop phase / linear).
• AudioEvent: src_offset field; renderer audio paths (loop-with-pauses, standard loop, one-shot) honor it.
• VideoEvent.skip_override: renderer video input -ss honors it; the clip's embedded audio (tvaud) is seeked automatically by the same input seek.
• _chunk_boundary_span_warnings: downgraded from the "will be dropped" v1 warning to an informational note (logged; terminal only under --verbose).
• Plan-level tests: tests/test_chunking_v2.py.
• Render-seam validation on the rig (chunked-vs-full frame/audio diff) — the remaining gate before making v2 the trusted default.
• Crossfade-loop audio (_build_crossfade_loop_filter) does not yet apply src_offset — a crossfaded looping bed restarts phase at the seam. Standard (non-crossfade) loops and one-shots are handled. Low priority.
• OutroEvent: not needed — outros are extracted for the last chunk only (config.outro if is_last_chunk), so they never split across a seam.

Not affected (verified)

• Slides: _extract_slide_events already used overlap+clamp — no change.
• Full-screen plan.background: a separate always-included input.
• Full (non-chunked) render: time_range=None path leaves skip_override/ src_offset at their defaults, so output is byte-identical to before.