gnommo/docs/chunking_v2.md

Chunked Rendering v2 — Design Spec

Status: planned (v1 shipped; v1 boundary limitation is detected + warned, not yet fixed)

Why chunking exists

The render is a single ffmpeg -filter_complex pass that opens every -i input up front. On memory-constrained machines (an 8 GB VM, or a WSL2 rig whose VM RAM is a slice of the host) the aggregate decoder + filter buffers OOM-kill the process — or, on WSL2, the whole VM. render_chunk_slides (in ~/.gnommo.conf [performance], or --chunk-slides N) splits the timeline into groups of N slides, renders each as a partial render, and concatenates the chunks with -c copy.

Each chunk is a partial render built via build_render_plan(slide_range=(start,end)), which passes time_range=(time_offset, render_end_time) to the event extractors. See partial-rendering-spec.md for the partial-render mechanics chunking reuses.

The v1 defect this fixes

_extract_video_events / _extract_audio_events keep an event only if its start falls inside the window:

if start_time < range_start or start_time >= range_end:
    continue

So a clip that began in an earlier chunk and is still playing across the boundary is dropped from every later chunk it overlaps. The full (non-chunked) render is correct; the chunked render silently diverges.

Who is affected:

• Overlay videos that span slides: end_on: next_video, loop, long take/end (multi-slide backgrounds / persistent picture-in-picture).
• Looping background audio/music: started once early, meant to underlie the whole video — dropped from every chunk after the first. Highest blast radius.
• Not affected: end_on: next_slide clips (they end exactly at a slide marker = a chunk seam), per-slide content, and the full-screen plan.background (a separate always-included input).

v1 status: cli._chunk_boundary_span_warnings(plan, groups) detects spanning video/outro/audio events and prints a loud warning before rendering, so the divergence is never silent. It does not yet correct the output.

v2 algorithm

1. Overlap inclusion (not start-inside)

Replace the start-inside test with an overlap test in both extractors:

# keep the event if it overlaps [range_start, range_end)
if end_time <= range_start or start_time >= range_end:
    continue

2. Per-event seek for clips that began earlier

A clip included by overlap whose start_time < range_start is already mid-playback at the chunk boundary. It must resume at the correct frame, not restart. Compute how far into the clip the window begins and carry it as a per-event seek:

into = range_start - start_time            # seconds of the clip already elapsed
base = video_source.skip or 0.0
# non-looping clip:
skip_override = base + into
# looping clip (end_on: loop / next_video that wraps):
period = (video_source.duration or 0) - base      # one loop's playable length
skip_override = base + (into % period) if period > 0 else base

Then clamp the event to the window and let the offset pass zero it:

start_time = max(start_time, range_start)   # -> 0 after time_offset subtraction
end_time   = min(end_time, range_end)

The seam for the seek is VideoEvent.skip_override (already added, inert until v2): the renderer prefers it over video_source.skip when set. -ss {skip} is applied as an input option, so ffmpeg decodes to that point — frame-accurate for the codecs in use.

3. Audio equivalent

AudioEvent has no skip_override / end_time yet. v2 adds both (or derives end from audio_def.duration / loop → total_duration) and applies the same overlap + seek. For looping music the seek is the loop-phase modulo above; ignore_pauses and overlap (crossfade) interactions must be re-checked at the seam.

4. Outro events

OutroEvent also spans (start_time/end_time). Extend the same treatment; outros normally live in the final chunk so this is lower priority but should be covered for completeness.

Concat-seam correctness (the risk to validate)

Chunks are joined with -c copy, so the two sides of a seam must be frame-aligned: chunk k ends showing the clip at position P, chunk k+1 must resume at exactly P via skip_override. Validation plan:

1. Pick a project with a known multi-slide overlay video and looping music (or synthesize one).
2. Render it full (reference) and chunked (small chunk_slides, so a seam falls mid clip).
3. Compare: identical duration; frame diff at ±3 frames around each seam below a threshold; audio cross-correlation shows no gap/jump; the overlay is present in every chunk it overlaps (the v1 bug is gone).
4. Assert _chunk_boundary_span_warnings returns empty for the fixed path.

Work items

• _extract_video_events: overlap test + skip_override (loop-aware).
• _extract_audio_events: add end_time/skip_override, overlap + seek.
• AudioEvent: skip_override field; renderer audio path honors it.
• OutroEvent: same treatment.
• Frame/audio seam-diff test (chunked vs full) in the test suite.
• Flip _chunk_boundary_span_warnings from "will be dropped" to a debug-only assertion once v2 is the default.

Seams already in place (v2 preparations, shipped)

• VideoEvent.skip_override (models.py) — inert; renderer honors it when set.
• cli._chunk_boundary_span_warnings — detection + warning.
• v1-limitation comments at both filter sites in transformer.py.