Fix before chunking v2.0

ZZ

docs/chunking_v2.md

@@ -0,0 +1,128 @@
+# Chunked Rendering v2 — Design Spec
+
+Status: **planned** (v1 shipped; v1 boundary limitation is detected + warned, not yet fixed)
+
+## Why chunking exists
+
+The render is a single `ffmpeg -filter_complex` pass that opens every `-i` input up
+front. On memory-constrained machines (an 8 GB VM, or a WSL2 rig whose VM RAM is a
+slice of the host) the aggregate decoder + filter buffers OOM-kill the process — or,
+on WSL2, the whole VM. `render_chunk_slides` (in `~/.gnommo.conf` `[performance]`, or
+`--chunk-slides N`) splits the timeline into groups of N slides, renders each as a
+partial render, and concatenates the chunks with `-c copy`.
+
+Each chunk is a partial render built via `build_render_plan(slide_range=(start,end))`,
+which passes `time_range=(time_offset, render_end_time)` to the event extractors. See
+`partial-rendering-spec.md` for the partial-render mechanics chunking reuses.
+
+## The v1 defect this fixes
+
+`_extract_video_events` / `_extract_audio_events` keep an event **only if its start
+falls inside the window**:
+
+```python
+if start_time < range_start or start_time >= range_end:
+    continue
+```
+
+So a clip that **began in an earlier chunk and is still playing across the boundary**
+is dropped from every later chunk it overlaps. The full (non-chunked) render is
+correct; the chunked render silently diverges.
+
+Who is affected:
+
+- **Overlay videos that span slides:** `end_on: next_video`, `loop`, long `take`/`end`
+  (multi-slide backgrounds / persistent picture-in-picture).
+- **Looping background audio/music:** started once early, meant to underlie the whole
+  video — dropped from every chunk after the first. Highest blast radius.
+- **Not affected:** `end_on: next_slide` clips (they end exactly at a slide marker =
+  a chunk seam), per-slide content, and the full-screen `plan.background` (a separate
+  always-included input).
+
+v1 status: `cli._chunk_boundary_span_warnings(plan, groups)` detects spanning
+video/outro/audio events and prints a loud warning before rendering, so the divergence
+is never silent. It does not yet correct the output.
+
+## v2 algorithm
+
+### 1. Overlap inclusion (not start-inside)
+
+Replace the start-inside test with an overlap test in both extractors:
+
+```python
+# keep the event if it overlaps [range_start, range_end)
+if end_time <= range_start or start_time >= range_end:
+    continue
+```
+
+### 2. Per-event seek for clips that began earlier
+
+A clip included by overlap whose `start_time < range_start` is already mid-playback at
+the chunk boundary. It must resume at the correct frame, not restart. Compute how far
+into the clip the window begins and carry it as a **per-event seek**:
+
+```
+into = range_start - start_time            # seconds of the clip already elapsed
+base = video_source.skip or 0.0
+# non-looping clip:
+skip_override = base + into
+# looping clip (end_on: loop / next_video that wraps):
+period = (video_source.duration or 0) - base      # one loop's playable length
+skip_override = base + (into % period) if period > 0 else base
+```
+
+Then clamp the event to the window and let the offset pass zero it:
+
+```
+start_time = max(start_time, range_start)   # -> 0 after time_offset subtraction
+end_time   = min(end_time, range_end)
+```
+
+The seam for the *seek* is `VideoEvent.skip_override` (already added, inert until v2):
+the renderer prefers it over `video_source.skip` when set. `-ss {skip}` is applied as
+an input option, so ffmpeg decodes to that point — frame-accurate for the codecs in
+use.
+
+### 3. Audio equivalent
+
+`AudioEvent` has no `skip_override` / `end_time` yet. v2 adds both (or derives end from
+`audio_def.duration` / `loop` → `total_duration`) and applies the same overlap + seek.
+For looping music the seek is the loop-phase modulo above; `ignore_pauses` and
+`overlap` (crossfade) interactions must be re-checked at the seam.
+
+### 4. Outro events
+
+`OutroEvent` also spans (start_time/end_time). Extend the same treatment; outros
+normally live in the final chunk so this is lower priority but should be covered for
+completeness.
+
+## Concat-seam correctness (the risk to validate)
+
+Chunks are joined with `-c copy`, so the two sides of a seam must be frame-aligned:
+chunk *k* ends showing the clip at position `P`, chunk *k+1* must resume at exactly `P`
+via `skip_override`. Validation plan:
+
+1. Pick a project with a known multi-slide overlay video **and** looping music
+   (or synthesize one).
+2. Render it full (reference) and chunked (small `chunk_slides`, so a seam falls mid
+   clip).
+3. Compare: identical duration; frame diff at ±3 frames around each seam below a
+   threshold; audio cross-correlation shows no gap/jump; the overlay is present in
+   every chunk it overlaps (the v1 bug is gone).
+4. Assert `_chunk_boundary_span_warnings` returns empty for the fixed path.
+
+## Work items
+
+- [ ] `_extract_video_events`: overlap test + `skip_override` (loop-aware).
+- [ ] `_extract_audio_events`: add `end_time`/`skip_override`, overlap + seek.
+- [ ] `AudioEvent`: `skip_override` field; renderer audio path honors it.
+- [ ] `OutroEvent`: same treatment.
+- [ ] Frame/audio seam-diff test (chunked vs full) in the test suite.
+- [ ] Flip `_chunk_boundary_span_warnings` from "will be dropped" to a debug-only
+      assertion once v2 is the default.
+
+## Seams already in place (v2 preparations, shipped)
+
+- `VideoEvent.skip_override` (models.py) — inert; renderer honors it when set.
+- `cli._chunk_boundary_span_warnings` — detection + warning.
+- v1-limitation comments at both filter sites in `transformer.py`.

gnommo/cli.py

@@ -4136,6 +4136,58 @@ def _writeback_video_metadata(plan, project_path, config) -> None:
         print(f"  Updated videos.json: {written}")
 
 
+def _chunk_boundary_span_warnings(plan, groups) -> list[str]:
+    """Detect events that span a chunk boundary.
+
+    KNOWN LIMITATION (chunking v1): the render's time-range filter keeps only events
+    whose *start* falls inside a chunk's window (transformer.py `_extract_video_events`
+    / `_extract_audio_events`: `if start_time < range_start: continue`). An overlay
+    video or audio clip that began in an earlier chunk and is still playing across the
+    boundary is therefore DROPPED from every later chunk it overlaps — so the chunked
+    output silently diverges from a full render. This detector makes that non-silent.
+    The real fix (overlap-inclusion + per-event seek) is chunking v2 — see
+    docs/chunking_v2.md.
+    """
+    slide_start = {e.slide_id: e.start_time for e in plan.slide_events}
+    boundaries = []  # (slide_id, output_time) at each chunk seam after the first
+    for g in groups[1:]:
+        t = slide_start.get(g[0])
+        if t is not None:
+            boundaries.append((g[0], t))
+    if not boundaries:
+        return []
+
+    warnings: list[str] = []
+
+    def _check(start, end, label, kind):
+        crossed = [(sid, bt) for sid, bt in boundaries if start < bt < end]
+        if crossed:
+            sid, bt = crossed[0]
+            extra = f" (and {len(crossed) - 1} more)" if len(crossed) > 1 else ""
+            warnings.append(
+                f"{kind} '{label}' plays {_format_time(start)}–{_format_time(end)} and "
+                f"crosses the chunk boundary at {sid} ({_format_time(bt)}){extra}; it will "
+                f"be DROPPED from the chunk(s) after the boundary."
+            )
+
+    for e in plan.video_events:
+        _check(e.start_time, e.end_time,
+               getattr(e.video_source, "source_file", e.video_id), "video")
+    for e in plan.outro_events:
+        _check(e.start_time, e.end_time,
+               getattr(e.video_source, "source_file", e.video_id), "outro video")
+    for e in plan.audio_events:
+        ad = e.audio_def
+        if getattr(ad, "loop", False):
+            end = plan.total_duration
+        elif getattr(ad, "duration", None) is not None:
+            end = e.start_time + ad.duration
+        else:
+            continue  # unknown length — can't judge span
+        _check(e.start_time, end, ad.file, "audio")
+    return warnings
+
+
 def _chunked_render(
     project_path: Path,
     verbose: bool,
@@ -4146,6 +4198,7 @@ def _chunked_render(
     slide_ids: list[str],
     out_dir: Path,
     final_output: Path,
+    plan=None,
 ) -> int:
     """Render in slide-based chunks then concatenate — avoids filter graph OOM."""
     import math
@@ -4158,6 +4211,30 @@ def _chunked_render(
         f"\n  Auto-chunking: {len(slide_ids)} slides → {len(groups)} chunks of ≤{chunk_size}"
     )
 
+    # Warn about events that span a chunk boundary — chunking v1 drops these from
+    # the later chunk, so the output would silently differ from a full render.
+    if plan is not None:
+        _span_warnings = _chunk_boundary_span_warnings(plan, groups)
+        if _span_warnings:
+            banner = "  " + "!" * 64
+            print(f"\n{banner}", file=sys.stderr)
+            print("  CHUNK-BOUNDARY WARNING — chunked output will differ from a full render:",
+                  file=sys.stderr)
+            for w in _span_warnings:
+                print(f"    - {w}", file=sys.stderr)
+            print("  These overlay video/audio clips span a chunk seam. Chunking v1 keeps",
+                  file=sys.stderr)
+            print("  only clips that START inside a chunk, so the ones above vanish after",
+                  file=sys.stderr)
+            print("  the boundary. Options: render this project WITHOUT chunking (lower",
+                  file=sys.stderr)
+            print("  cpu_limit instead), or move the chunk size so no seam splits them.",
+                  file=sys.stderr)
+            print(f"{banner}\n", file=sys.stderr)
+            _render_log("CHUNK-BOUNDARY WARNING (v1 drops cross-boundary clips):")
+            for w in _span_warnings:
+                _render_log(f"  - {w}")
+
     chunks_dir = out_dir / "chunks"
     chunks_dir.mkdir(parents=True, exist_ok=True)
 
@@ -5112,6 +5189,7 @@ def _cmd_render_impl(
             _slide_ids,
             out_dir,
             output_path,
+            plan=plan,
         )
 
     plan.output_path = output_path
@@ -5197,7 +5275,7 @@ def _cmd_render_impl(
         _render_log("FFmpeg command:\n" + generate_ffmpeg_command_string(plan, output_path))
     except Exception as _e:
         _render_log(f"(could not serialize ffmpeg command: {_e})")
-    render(plan, output_path, verbose=verbose)
+    render(plan, output_path, verbose=verbose, log=_render_log)
     print(f"  Output: {output_path}")
 
     if _render_gateable:

gnommo/models.py

@@ -429,6 +429,11 @@ class VideoEvent:
     cutout: "CutoutDefinition"
     cutout_name: str = ""  # resolved cutout name (e.g. "fullscreen"), for display
     layer: str = "above"  # "above" = on top of slides; "below" = behind slides
+    # Chunking v2 seam (see docs/chunking_v2.md): when a clip began before this
+    # chunk's window, the render must seek into it so it resumes mid-clip instead of
+    # restarting at the boundary. None = play from video_source.skip (the v1/default).
+    # Set only by the v2 overlap-inclusion path; the renderer prefers it when present.
+    skip_override: Optional[float] = None
 
 
 @dataclass

gnommo/preprocessor.py

@@ -428,7 +428,12 @@ def _oom_postmortem(returncode: int, log_text: str) -> None:
     sys.stdout.flush()
 
 
-def run_ffmpeg_with_progress(cmd, duration, description="Processing", loglevel=None):
+def run_ffmpeg_with_progress(cmd, duration, description="Processing", loglevel=None,
+                             progress_hook=None):
+    """progress_hook(seconds): optional callback invoked ~every 5s with the current
+    output timestamp. The render passes one that logs the active slide/asset + RAM,
+    so the last line flushed before a hard kill (even a whole-VM WSL crash) names
+    where in the timeline memory blew up."""
     from collections import deque
 
     cmd = cmd.copy()
@@ -492,6 +497,7 @@ def run_ffmpeg_with_progress(cmd, duration, description="Processing", loglevel=N
     last_percent = 0
     seen_any_progress = False
     last_log_line = ""
+    last_hook = 0.0
     logs = deque(maxlen=_LOG_TAIL)
 
     def draw(percent, suffix=""):
@@ -553,6 +559,12 @@ def run_ffmpeg_with_progress(cmd, duration, description="Processing", loglevel=N
                         last_update = time.time()
                         seen_any_progress = True
                         draw(last_percent, "")
+                        if progress_hook is not None and (time.time() - last_hook) >= 5.0:
+                            last_hook = time.time()
+                            try:
+                                progress_hook(t_s)
+                            except Exception:
+                                pass
                     except ValueError:
                         pass
 

gnommo/renderer.py

@@ -159,7 +159,64 @@ def _build_crossfade_loop_filter(
     return filters
 
 
-def render(plan: RenderPlan, output_path: Path, verbose: bool = False) -> None:
+def _fmt_t(s: float) -> str:
+    s = max(0, int(s))
+    return f"{s // 60:02d}:{s % 60:02d}"
+
+
+def _active_slide_at(plan: RenderPlan, t: float):
+    """slide_id enabled at output time t (last slide whose window has opened)."""
+    cur = None
+    for e in plan.slide_events:
+        if e.start_time <= t < e.end_time:
+            return e.slide_id
+        if e.start_time <= t:
+            cur = e.slide_id
+    return cur
+
+
+def _active_videos_at(plan: RenderPlan, t: float) -> list:
+    """source_file of every video/outro event playing at output time t."""
+    names = []
+    for e in list(plan.video_events) + list(plan.outro_events):
+        if e.start_time <= t < e.end_time:
+            src = getattr(getattr(e, "video_source", None), "source_file", None)
+            names.append(src or "?")
+    return names
+
+
+def _log_render_timeline(plan: RenderPlan, log) -> None:
+    """Static output-time -> asset map (log only). Lets any render position — incl.
+    the last one before a hard kill — be mapped to a slide/asset after the fact."""
+    log("[timeline] video/outro assets by output time (RAM-relevant inputs):")
+    rows = [
+        (e.start_time, e.end_time, getattr(getattr(e, "video_source", None), "source_file", "?"))
+        for e in list(plan.video_events) + list(plan.outro_events)
+    ]
+    for start, end, src in sorted(rows):
+        log(f"  {_fmt_t(start)}-{_fmt_t(end)}  {src}")
+
+
+def _make_render_progress_hook(plan: RenderPlan, log):
+    from .preprocessor import _mem_snapshot
+
+    def hook(t: float) -> None:
+        mem = _mem_snapshot()
+        memstr = ""
+        if mem and mem[1]:
+            avail, total = mem
+            memstr = f"{100 * (total - avail) / total:.0f}% mem, {avail / 1e9:.1f}GB free | "
+        vids = _active_videos_at(plan, t)
+        vidstr = ", ".join(vids) if vids else "(none)"
+        log(
+            f"[render {_fmt_t(t)}/{_fmt_t(plan.total_duration)}] {memstr}"
+            f"slide={_active_slide_at(plan, t)} | videos: {vidstr}"
+        )
+
+    return hook
+
+
+def render(plan: RenderPlan, output_path: Path, verbose: bool = False, log=None) -> None:
     """
     Render the final video using FFmpeg.
 
@@ -167,6 +224,11 @@ def render(plan: RenderPlan, output_path: Path, verbose: bool = False) -> None:
     1. Scales background video (if present) or creates solid color
     2. Overlays talking head at configured position
     3. Overlays slides at their configured positions with time-based enable
+
+    `log`: optional callback (line -> None) writing to the render log only. When
+    given, a static timeline map is logged and the render position + active
+    slide/asset + RAM are logged ~every 5s, so a hard kill's last flushed line
+    pinpoints which asset the memory blew up on.
     """
     # Ensure output directory exists
     output_path.parent.mkdir(parents=True, exist_ok=True)
@@ -179,9 +241,17 @@ def render(plan: RenderPlan, output_path: Path, verbose: bool = False) -> None:
         print(" ".join(cmd))
         print()
 
+    hook = None
+    if log is not None:
+        try:
+            _log_render_timeline(plan, log)
+        except Exception:
+            pass
+        hook = _make_render_progress_hook(plan, log)
+
     # Run with progress bar and ETA
     result = run_ffmpeg_with_progress(
-        cmd, duration=plan.total_duration, description="Rendering"
+        cmd, duration=plan.total_duration, description="Rendering", progress_hook=hook
     )
 
     if result.returncode != 0:
@@ -519,7 +589,10 @@ def build_ffmpeg_command(plan: RenderPlan, output_path: Path) -> list[str]:
         video_path = _resolve_video_path(
             videos_dir, event.video_source, shared_assets_dir, project_path
         )
-        skip = event.video_source.skip or 0.0
+        # Chunking v2 (docs/chunking_v2.md): a clip that began before this chunk
+        # resumes mid-clip via skip_override. None today (v1) → the source's own skip.
+        skip = event.skip_override if getattr(event, "skip_override", None) is not None \
+            else (event.video_source.skip or 0.0)
 
         # How long this clip needs to play in the output
         clip_duration = event.end_time - event.start_time

gnommo/transformer.py

@@ -1321,7 +1321,13 @@ def _extract_video_events(
             # end_on is None and marker_type == "narration": runs to end
             end_time = total_duration
 
-        # Filter by time range
+        # Filter by time range.
+        # CHUNKING v1 LIMITATION: this keeps only clips whose START is inside the
+        # window, so a clip that began in an earlier chunk and is still playing
+        # across the boundary is DROPPED here — the chunked output then differs from
+        # a full render. cli._chunk_boundary_span_warnings surfaces this. The v2 fix
+        # is overlap-inclusion + a per-event seek (skip_override); see
+        # docs/chunking_v2.md.
         if start_time < range_start or start_time >= range_end:
             continue
         end_time = min(end_time, range_end)
@@ -1361,6 +1367,9 @@ def _extract_audio_events(
         elif marker_id.startswith("audio:"):
             audio_id = marker_id[6:]
         if audio_id is not None and audio_id in audio:
+            # CHUNKING v1 LIMITATION (same as video, worse for looping background
+            # music): a clip started before the window is dropped from later chunks.
+            # v2 = overlap-inclusion + audio seek; see docs/chunking_v2.md.
             if timing.timestamp < range_start or timing.timestamp >= range_end:
                 continue
             start_time = max(0, timing.timestamp - AUDIO_OFFSET_SECONDS)