gitprov/gnommo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Adding updates to titlesZZ

Browse Source
This commit is contained in:
gitprov
2026-07-14 14:04:44 +02:00
parent f9ff847f6b
commit 308a9f8bcd
17 changed files with 4608 additions and 186 deletions
Download Patch File Download Diff File Expand all files Collapse all files
### Problems.md
+35
View File
@@ -0,0 +1,35 @@
### Problems
# Video 1 :
Music ok
Talkinghead too high, move down.
Glitch logo lasts 1 seconds, but render shows it should last 6 seconds and pause narration for 6 seconds.
The narration correctly resumes after 6 seconds, leaving. The problem appears to be that S2 is indicated to start at 00:32.12 "Welcome to this video series about digital philosophy", which is incorrect. It should start at 00.37 or 00.38. So the positioning of the slide does not respect the pause narration.
Sent to claude
02.07 slide of xray background with no text
02.30 : Move to "not just the sky around Antares" in a frame?
Discrete font on the one, two, three slide.
Black frame on wavepacket
Gedankenexperiment missing slide
Done
"Assume the universe is fundamentally discrete" weird pause. Fixed
Need a slide before "Something else that has".
13.42: The Glitch slide. Talking head is above the cloud background. Cloud background should be mid level.
Change to glitch video
DOne
# Video 2 :
The same problem as v1, the vf2m appears below the talking head. But the order is supposed to be
GlitchTrailer/audio.json
+8
View File
@@ -0,0 +1,8 @@
{
"musicforyourdreams": {
"file": "musicforyourdreams.m4a",
"is_shared": true,
"volume": 1.0,
"duration": 448.832
}
}
GlitchTrailer/manuscript.txt
+3 -3
View File
@@ -10,8 +10,9 @@ Like Minecraft - just with absurdly tiny blocks?
[S3]
This question has been around forever.
But it's always been filed under "too weird to bother."
That's about to change.
But it seems too weird to be worth trying.
Nobody's really tried to take it seriously.
Until now
[S4]
Explore the tech-tree.
@@ -21,7 +22,6 @@ We're committed to scientific rigour.
Falsifiability. Truth-seeking.
And not being a complete bore.
[S5]
Don’t enroll now.
Enroll later.
GlitchTrailer/media/slides/GlitchTrailer/slides.json
+6 -6
View File
@@ -1,26 +1,26 @@
{
"S1": {
"image": "GlitchTrailer.001.png",
"image": "glitchtrailer.001.png",
"type": "fullscreen"
},
"S2": {
"image": "GlitchTrailer.002.png",
"image": "glitchtrailer.002.png",
"type": "fullscreen"
},
"S3": {
"image": "GlitchTrailer.003.png",
"image": "glitchtrailer.003.png",
"type": "fullscreen"
},
"S4": {
"image": "GlitchTrailer.004.png",
"image": "glitchtrailer.004.png",
"type": "fullscreen"
},
"S5": {
"image": "GlitchTrailer.005.png",
"image": "glitchtrailer.005.png",
"type": "fullscreen"
},
"S6": {
"image": "GlitchTrailer.006.png",
"image": "glitchtrailer.006.png",
"type": "fullscreen"
}
}
GlitchTrailer/project.json
+3 -3
View File
@@ -68,9 +68,9 @@
},
"cutouts": {
"talkinghead": {
"x": "-23%",
"y": "10%",
"height": "90%"
"x": "-10%",
"y": "40%",
"height": "80%"
},
"square": {
"x": "46.5%",
GlitchTrailer/videos.json
+1941
View File
File diff suppressed because it is too large Load Diff
GlitchTrailer/youtube_meta.txt
+2
View File
@@ -0,0 +1,2 @@
== Description ==
Welcome to Glitch University.
INT5000/manuscript.txt
+103
View File
@@ -0,0 +1,103 @@
[S1]
Today we’re not proving quantum mechanics is discrete.
We’re running a test.
One test, named INT5000.
[S2]
We already completed some simple tests. We called them the naïve test suite.
It required us to define what is the Action, how does stuff move around.
And what is momentum.
But might be neat, but all it showed is that the CA glider has vaguely particle-like properties.
But a fundamentally discrete universe is 50% of the option space. The other 50% is a continuous universe. Ruling out 50% of the option space like a decent bang for the buck, so we committed to giving a discrete universe every chance it can have to be true. Even if it sounds unlikely.
We need to make it pass more tests. Lets introduce the advanced test suite.
[S3]
If quantum mechanics is emergent from something discrete, there are many constraints it must satisfy.
It needs Lorentz invariance, otherwise it contradicts Einstein.
Most of these tests are beyond my abilities, even with AI assistance.
But there is one I’m excited about, and it’s this one.
Can a Cellular Automaton reproduce the spinorial rotation structure - SU(2)?
[S4]
In quantum mechanics, certain objects - spinors - behave strangely.
Rotate them 360 degrees, and they don’t return to their original state.
They pick up a minus sign.
Only after 720 degrees do they truly return.
This behavior is encoded mathematically in a group called SU(2).
If we can’t reproduce this, we can’t reproduce spin.
If we can’t do that, QM is out of reach. And that means we’re back to the default scenario.
[S5]
And the default scenario is - that the Naga is an neat and weird Cellular Automaton, of no significance to physics at all.
Just like this channel, with its middle aged amateur creator dressed up in a yellow jumpsuit
Trying to do stuff I don’t have the necessary training to do.
This is the most likely outcome. I owning it, not afraid of it. It doesn’t hurt anyone, so lets keep going.
[S6]
And as you know, physicists generally don’t understand code-speak, and developers struggle with physics-speak. So we’ll keep translating showing both sides.
In Code speak - SU(2) is a interface exposed by QM that we must support.
If we can’t support it, the migration fails.
In physics speak - we isolate the SU(2) representation structure. Then we evaluate only whether the generators exposed by the proposed structure allows to satisfy the Lie algebra.
[S7]
Basically we must derive SU(2) from the Naga in an elegant way. If we make any assumptions doing so, those same assumptions carry over to all other integrations tests.
That’s the contract.
[S8]
Importantly, this test does not require
Not Lorentz invariance.
Not isotropy of space.
Not full quantum field theory.
Just this single interface Naga - > SU(2)
[S9]
It might be unnecessary to say this, but you never know, some people are sloppy when meting out criticism. So passing this test does NOT mean
1. that QM is discrete
2. That Bells theorem is not relevant
3. Or that the theory not falsifiable
It just mean that given the assumptions and tests, the test either fails or passes.
If it passes, it earns the right to attempt the next one.
That’s it. It’s test-driven development.
[S10]
A Cellular Automaton is discrete.
SU(2) is continuous.
To derive something continuous from something discrete requires that the continuous is an approximation to the discrete.
And this is very
[S11]
So Spinors liven 3D space. Which means it should be possible to rotate them like this cup. I rotate it 360° brings you back.
But spinors are weird, they don’t work like that.
It’s kind of like this sock. Now if I rotate this sock 360 degrees, it returns back to where it was. But spinors is kind of like a sock that gets inside out each time it rotates. So when I rotate it 360 degrees it inverts. This inversion is captured by this extra -1. Then if I rotate it again it inverts again. So to return completely to both the same position and the same inside-out, we need two rotations.
That “extra twist” is not decorative is deep in QM. You’re made from objets which rotate in this way, not the normal way.
[S12]
What this means for our CA, is that if it naturally accumulates some kind of twist deficit after 2π that only resolves after 4π,
we’re in business.
Mathematically - we need a two-valued memory of rotation that flips sign after 2π.
[S13]
So lets have a look at the CA. Let us first identify when it goes back to normal. We look at the head here, and keep iterating until the head is back in the original position. This takes N steps. So N steps is a full rotation. Now what happens if we apply just half of these steps, where do we en up - we end up here.
Periodicity alone is not enough. We now examine how orientation accumulates during those steps.
[S14]
And if we look carefully, we executed half of the X, half of the Y and half the Z. Which means that this midway point corresponds to half a rotation in all three planes. See here xy-plane, half rotated. Xz plane, half rotated. Yz plane half rotated. Meaning that T = N gives us a full rotation in all these three planes.
A full rotation in one plane is 2pi, so a full rotation in all three plane should be 6pi. That is surprising. But look
[S15]
A full rotation in the xy plane is also half a rotation in the xz plane, because the x-components are shared.
And a full rotation in they yz plane is also a half rotation in the xz plane, because the z components are shared.
Meaning that a full rotation in two planes gives us the full rotation in the third plane for free. 2/3 out of 6 pi is 4 pi.
[S16]
Did that not convince you? Let’s do it differently.
Suppose you define a rotation as the theta value of the HEAD relative to the center in a chosen plane.
You will now have three rotation generators.
You can’t add them together because they are never orthogonal to both other planes at the same time.
Each theta value is orthogonal to half the theta value in each of the two remaining planes.
Meaning you have theta = theta_xy + 1/2 theta_xz + 1/2 theta_yz.
A slightly more general form of this is : 2/3(theta_xz + theta_xy + theta_yz)
So if after a full rotation in a plane theta_xy = theta_xz = theta_yz = 2\pi, then the full rotation becomes
2/3 (2 pi + 2pi + 2pi ) = 4 pi
[S17]
INT5000/media/audio/audio.json
+7
View File
@@ -0,0 +1,7 @@
{
"musicforyourdreams": {
"file": "musicforyourdreams.m4a",
"is_shared": true,
"volume": 1.0
}
}
INT5000/media/slides/int5000/slides.json
+70
View File
@@ -0,0 +1,70 @@
{
"S1": {
"image": "int5000.001.png",
"type": "fullscreen"
},
"S2": {
"image": "int5000.002.png",
"type": "fullscreen"
},
"S3": {
"image": "int5000.003.png",
"type": "fullscreen"
},
"S4": {
"image": "int5000.004.png",
"type": "fullscreen"
},
"S5": {
"image": "int5000.005.png",
"type": "fullscreen"
},
"S6": {
"image": "int5000.006.png",
"type": "fullscreen"
},
"S7": {
"image": "int5000.007.png",
"type": "fullscreen"
},
"S8": {
"image": "int5000.008.png",
"type": "fullscreen"
},
"S9": {
"image": "int5000.009.png",
"type": "fullscreen"
},
"S10": {
"image": "int5000.010.png",
"type": "fullscreen"
},
"S11": {
"image": "int5000.011.png",
"type": "fullscreen"
},
"S12": {
"image": "int5000.012.png",
"type": "fullscreen"
},
"S13": {
"image": "int5000.013.png",
"type": "fullscreen"
},
"S14": {
"image": "int5000.014.png",
"type": "fullscreen"
},
"S15": {
"image": "int5000.015.png",
"type": "fullscreen"
},
"S16": {
"image": "int5000.016.png",
"type": "fullscreen"
},
"S17": {
"image": "int5000.017.png",
"type": "fullscreen"
}
}
INT5000/media/videos/videos.json
+1941
View File
File diff suppressed because it is too large Load Diff
INT5000/youtube_meta.txt
View File
example/project.json
+1 -1
View File
@@ -77,7 +77,7 @@
"talkinghead": {
"x": "-10%",
"y": "40%",
"height": "60%"
"height": "80%"
},
"square": {
"x": "45%",
gnommo/cli.py
+279 -4
View File
@@ -38,6 +38,7 @@ Examples:
gnommo -p video1 validate Validate only
gnommo -p video1 import Generate slides.json from images
gnommo -p video1 pre Preprocess videos (chroma key, etc.)
gnommo -p video1 clear Delete preprocessed outputs so preprocess re-runs them
gnommo -p video1 stitch --res tiny -f Fast stitch with new begin/end values
gnommo -p video1 trim Auto-detect silence and set skip/take in narration.json
gnommo -p video1 trim --force Redo trim even for segments that already have skip/take
@@ -50,6 +51,7 @@ Examples:
gnommo -p video1 transcode --processed --alpha-quality 0.5 More aggressive alpha compression
gnommo -p video1 transcode --processed --dry-run Preview what would be compressed
gnommo -p video1 transcode --force Re-transcode even if output already exists
gnommo -p video0 new Create a new project with standard folder structure
gnommo -p video1 all Full pipeline: import → preprocess → trim → stitch → render → push → handoff → up
gnommo -p video1 render --dry-run Show FFmpeg command without running
gnommo -p video1 description Generate YouTube description file
@@ -107,6 +109,8 @@ Examples:
"handoff",
"transcode",
"pexels",
"clear",
"new",
],
help="Action to perform (default: render)",
)
@@ -259,6 +263,10 @@ Examples:
return cmd_import(project_path, args.force, args.verbose)
elif action == "validate":
return cmd_validate(project_path, args.verbose)
elif action == "new":
return cmd_new(project_path, args.verbose)
elif action == "clear":
return cmd_clear(project_path, args.verbose)
elif action in ("preprocess", "pre"):
return cmd_preprocess(
project_path,
@@ -1909,6 +1917,235 @@ def _resolve_narration_combined(
return None
def cmd_new(project_path: Path, verbose: bool) -> int:
"""Create a new gnommo project with standard folder structure and a project.json template."""
project_name = project_path.name
project_id = project_name
if project_path.exists() and list(project_path.iterdir()):
print(f"Initialising project: {project_path} (folder exists, filling in missing structure)")
else:
print(f"Creating new project: {project_path}")
# ------------------------------------------------------------------ #
# Directories #
# ------------------------------------------------------------------ #
dirs = [
project_path,
project_path / "media" / "videos",
project_path / "media" / "audio",
project_path / "media" / "narration" / "raw_mov",
project_path / "media" / "slides",
project_path / "out",
]
for d in dirs:
d.mkdir(parents=True, exist_ok=True)
if verbose:
print(f" mkdir {d.relative_to(project_path.parent)}")
# ------------------------------------------------------------------ #
# Copy talkinghead filter from the nearest sibling project #
# ------------------------------------------------------------------ #
talkinghead_filter = None
for sibling in sorted(project_path.parent.iterdir()):
if sibling == project_path or not sibling.is_dir():
continue
sib_json = sibling / "project.json"
if sib_json.exists():
try:
sib_cfg = json.loads(sib_json.read_text(encoding="utf-8"))
talkinghead_filter = (sib_cfg.get("default_filters") or {}).get("talkinghead")
if talkinghead_filter:
print(f" Copied talkinghead filter from: {sibling.name}/project.json")
break
except (json.JSONDecodeError, OSError):
pass
if not talkinghead_filter:
# Sensible placeholder — user should tweak gnommokey values for their camera
talkinghead_filter = [
{
"type": "audio_normalize",
"compress": False,
"normalize": True,
"target_lufs": -14,
"target_lra": 11,
"target_tp": -1.5,
},
{
"type": "gnommokey",
"screen_color": [81, 137, 65],
"screen_gain": 175,
"screen_balance": 58,
"despill_bias": [217, 240, 255],
"despill_strength": 5.0,
"edge_erode": 1.0,
"clip_black": 0,
"clip_white": 100,
},
{
"type": "color_grade",
"saturation": 0.95,
"contrast": 1.06,
"rm": -0.05,
"gm": 0.02,
"bm": -0.04,
"curves_master": "0/0.02 0.5/0.5 1/0.97",
},
{
"type": "mask",
"left": 0.05,
"right": 0.1,
"top": 0.1,
"bottom": 0.0,
},
]
print(" Using default talkinghead filter (adjust gnommokey values for your camera)")
# ------------------------------------------------------------------ #
# project.json #
# ------------------------------------------------------------------ #
project_json_path = project_path / "project.json"
if not project_json_path.exists():
project_data = {
"id": project_id,
"name": "",
"description": "",
"platform_targets": ["youtube"],
"status": "scripted",
"resolution": [1920, 1080],
"fps": 30,
"manuscript": "manuscript.txt",
"videos": "media/videos/videos.json",
"narration": "media/narration/narration.json",
"slides": f"media/slides/{project_id}/slides.json",
"audio": "media/audio/audio.json",
"output_video": f"{project_id}.mp4",
"default_filters": {"talkinghead": talkinghead_filter},
"cutouts": {
"talkinghead": {"x": "-10%", "y": "40%", "height": "80%"},
"square": {"x": "46.5%", "y": "4.5%", "width": "50%", "height": "90%"},
"fullscreen": {"x": "0%", "y": "0%", "height": "100%"},
"fullscreen2": {"x": "10%", "y": "7%", "height": "80%"},
},
}
project_json_path.write_text(
json.dumps(project_data, indent=2, ensure_ascii=False), encoding="utf-8"
)
print(" Created: project.json")
else:
print(" Skipped: project.json (already exists)")
# ------------------------------------------------------------------ #
# Stub JSON files #
# ------------------------------------------------------------------ #
stubs: dict[str, object] = {
"media/videos/videos.json": {},
"media/audio/audio.json": {},
"media/narration/narration.json": {},
}
for rel, content in stubs.items():
p = project_path / rel
if not p.exists():
p.write_text(json.dumps(content, indent=2), encoding="utf-8")
print(f" Created: {rel}")
# ------------------------------------------------------------------ #
# Manuscript template #
# ------------------------------------------------------------------ #
manuscript_path = project_path / "manuscript.txt"
if not manuscript_path.exists():
manuscript_path.write_text(
"[S1]\nYour narration for slide 1 goes here.\n\n[S2]\n\n",
encoding="utf-8",
)
print(" Created: manuscript.txt")
# ------------------------------------------------------------------ #
# Instructions #
# ------------------------------------------------------------------ #
print(f"""
Done. Here is what to do next:
1. Place your Keynote presentation (.key) in the project folder:
{project_path}/
2. Record your talking head segments using a teleprompter.
Name each recording after the slide range it covers:
S1-10.mov covers slides 1 – 10
S11-32.mov covers slides 11 – 32
S33-end.mov covers slides 33 to the end
Place the recordings in:
{project_path}/media/narration/raw_mov/
3. Edit manuscript.txt so the spoken words appear under the right [SN] marker.
Add [vfb:Logo6sec] or other video markers where needed.
4. Run the pipeline step by step:
gnommo -p {project_name} import # extract slides from Keynote
gnommo -p {project_name} pre # chroma key + audio normalise
gnommo -p {project_name} trim # auto-detect skip/take per segment
gnommo -p {project_name} stitch # join narration into one file
gnommo -p {project_name} render # produce the final video
Or run everything in one go:
gnommo -p {project_name} all
Output: {project_path}/out/{project_id}.mp4
""")
return 0
def cmd_clear(project_path: Path, verbose: bool) -> int:
"""Delete preprocessed outputs so that 'preprocess' re-runs them from scratch.
Removes *_processed.mov files from the processed/ directory (or the
process cache on the external disk if one is configured). narration.json
skip/take values and raw source files are NOT touched.
"""
from .parser import parse_project_config
print(f"Clearing preprocessed outputs: {project_path.name}")
config = parse_project_config(project_path)
narration_dir = project_path / "media" / "narration"
cache_root = _resolve_process_cache(project_path, config)
if cache_root:
processed_dir = cache_root / "media" / "narration" / "processed"
print(f" Cache: {processed_dir}")
else:
processed_dir = narration_dir / "processed"
print(f" Local: {processed_dir}")
if not processed_dir.exists():
print(" Nothing to clear — processed/ directory does not exist.")
return 0
candidates = sorted(
f for f in processed_dir.iterdir()
if f.is_file() and "_processed" in f.stem
)
if not candidates:
print(" Nothing to clear — no *_processed.* files found.")
return 0
total_bytes = 0
for f in candidates:
size = f.stat().st_size
total_bytes += size
print(f" Deleting {f.name} ({size / 1e9:.2f} GB)")
f.unlink()
print(f"\n Cleared {len(candidates)} file(s), freed {total_bytes / 1e9:.2f} GB.")
print(" Run 'gnommo preprocess' (without --force) to reprocess selectively.")
return 0
def cmd_preprocess(
project_path: Path,
verbose: bool,
@@ -1955,6 +2192,12 @@ def cmd_preprocess(
processed_dir = (cache_narration_dir or narration_dir) / "processed"
processed_dir.mkdir(parents=True, exist_ok=True)
# Remove any .tmp files left by a previously interrupted preprocess run.
# These are partial outputs that would block the segment from being reprocessed.
for stale_tmp in processed_dir.glob("*.tmp"):
print(f" Removing incomplete output from previous run: {stale_tmp.name}")
stale_tmp.unlink()
# Resolve intermediate directory
gnommo_scratch = None
if config.gnommo_scratch:
@@ -2419,11 +2662,13 @@ def cmd_trim(
new_skip = max(0.0, round(words[0].start - 0.5, 3))
if end_slide is None:
# S{N}-end: only trim start
new_take = round(total_dur - new_skip, 3)
# S{N}-end: trim start and 2s after last spoken word
last_word_end = words[-1].end
new_take = round(min(last_word_end + 2.0 - new_skip, total_dur - new_skip), 3)
new_take = max(0.0, new_take)
print(
f" first={words[0].start:.2f}s"
f" → skip={new_skip:.3f}s take={new_take:.3f}s (no end trim)"
f" first={words[0].start:.2f}s last={last_word_end:.2f}s"
f" → skip={new_skip:.3f}s take={new_take:.3f}s"
)
else:
last_slide_text = slide_texts.get(end_slide, "")
@@ -3257,6 +3502,27 @@ def _project_markers_to_videos(
if not projection:
return
# Build a case-insensitive index of shared_assets pause_narration values.
# When a video is marked is_shared but its local entry is missing pause_narration,
# we pull the value from the shared canonical entry so it's never lost when
# the ETL writes back cutout/layer under a lowercase key.
_shared_pause: dict[str, float] = {}
for _shared_candidate in [
project_path / "shared_assets" / "videos.json",
project_path.parent / "shared_assets" / "videos.json",
]:
if _shared_candidate and _shared_candidate.exists():
try:
with open(_shared_candidate, "r", encoding="utf-8") as _f:
_shared_raw = json.load(_f)
for _k, _v in _shared_raw.items():
pn = _v.get("pause_narration")
if pn:
_shared_pause[_k.lower()] = float(pn)
except (json.JSONDecodeError, OSError):
pass
break
def _apply_projection(json_path: Path) -> list[str]:
"""Apply projection to one videos.json file; return list of updated IDs."""
if not json_path.exists():
@@ -3284,6 +3550,15 @@ def _project_markers_to_videos(
entry[field] = value
changed = True
video_changed = True
# For is_shared entries: inherit pause_narration from shared_assets if
# not already set locally (handles case where explicit pause_narration
# lives on a different-case key in the shared library).
if entry.get("is_shared") and not entry.get("pause_narration"):
shared_pn = _shared_pause.get(video_id.lower())
if shared_pn:
entry["pause_narration"] = shared_pn
changed = True
video_changed = True
if video_changed:
updated.append(video_id)
if changed:
gnommo/preprocessor.py
+149 -155
View File
@@ -87,6 +87,42 @@ def _video_has_alpha(video_path: Path) -> bool:
return "yuva" in pix_fmt or "rgba" in pix_fmt or "bgra" in pix_fmt
# Codecs where every frame is independently coded (all-intra).
# Stream copy with -ss is frame-accurate for these — no re-encode needed.
_INTRA_ONLY_CODECS = frozenset(
{
"prores",
"prores_ks",
"mjpeg",
"dnxhd",
"dvvideo",
"huffyuv",
"ffv1",
"rawvideo",
"png",
"tiff",
}
)
def _get_video_codec(video_path: Path) -> str:
"""Return the codec name of the first video stream (lowercase)."""
cmd = [
"ffprobe",
"-v",
"error",
"-select_streams",
"v:0",
"-show_entries",
"stream=codec_name",
"-of",
"default=noprint_wrappers=1:nokey=1",
str(video_path),
]
result = subprocess.run(cmd, capture_output=True, text=True)
return result.stdout.strip().lower() if result.returncode == 0 else ""
def format_time(seconds: float) -> str:
"""Format seconds as human-readable time string."""
if seconds < 60:
@@ -725,8 +761,11 @@ def preprocess_video(
final_output.parent.mkdir(parents=True, exist_ok=True)
# Copy the final intermediate to the output location
shutil.copy2(current_input, final_output)
# Write to a .tmp file first so that an interrupted copy never leaves a
# partial file at the final path (which would be silently skipped next run).
tmp_output = final_output.with_suffix(".tmp")
shutil.copy2(current_input, tmp_output)
tmp_output.replace(final_output) # atomic on same filesystem
if verbose:
print(f" Final output: {final_output}")
@@ -2235,6 +2274,37 @@ def needs_preprocessing(videos_dir: Path, video_source: VideoSource) -> bool:
return True
def _build_loudnorm_filter(loudnorm_config: Optional[dict]) -> str:
_cfg = loudnorm_config or {}
_lufs = float(_cfg.get("target_lufs", -14))
_lra = float(_cfg.get("target_lra", 11))
_tp = float(_cfg.get("target_tp", -1.5))
return f"loudnorm=I={_lufs:.1f}:LRA={_lra:.1f}:TP={_tp:.1f}"
def _build_reencode_args(source_path: Path) -> tuple[list[str], list[str]]:
"""
Return (video_args, audio_args) for re-encoding an inter-frame source to a
normalized intra-frame format. Does not include -avoid_negative_ts or the
output path — callers add those.
"""
has_alpha = _video_has_alpha(source_path)
if has_alpha:
video_args = [
"-vf", "fps=30,format=yuva444p10le",
"-c:v", "prores_ks", "-profile:v", "4", "-pix_fmt", "yuva444p10le",
]
audio_args = ["-c:a", "pcm_s16le"]
else:
video_args = [
"-vf", "fps=30",
"-c:v", "libx264", "-preset", "fast", "-crf", "18",
"-movflags", "+faststart",
]
audio_args = ["-c:a", "aac", "-b:a", "192k"]
return video_args, audio_args
def stitch_narration_segments(
videos_dir: Path,
segment_ids: list[str],
@@ -2264,150 +2334,124 @@ def stitch_narration_segments(
"""
print(f" Concatenating {len(segment_ids)} narration segment(s)...")
# Create temp directory for trimmed segments
temp_dir = output_path.parent / "concat_temp"
temp_dir.mkdir(parents=True, exist_ok=True)
needs_loudnorm = any(
videos[seg_id].defer_loudnorm for seg_id in segment_ids if seg_id in videos
)
loudnorm_filter = _build_loudnorm_filter(loudnorm_config) if needs_loudnorm else None
trimmed_segments: list[Path] = []
for i, video_id in enumerate(segment_ids):
# ------------------------------------------------------------------ #
# Gather per-segment metadata #
# ------------------------------------------------------------------ #
segments: list[tuple[Path, float, Optional[float], float, str, bool]] = []
for video_id in segment_ids:
if video_id not in videos:
raise PreprocessError(
f"Narration segment '{video_id}' not found in videos.json",
filter_type=None,
)
video_source = videos[video_id]
source_path = get_preprocessed_path(videos_dir, video_source)
if not source_path.exists():
raise PreprocessError(
f"Narration segment not found: {source_path}",
filter_type=None,
)
# Get segment duration
full_duration = get_video_duration(source_path)
skip = video_source.skip or 0.0
take = video_source.take
# Apply default end trim if no explicit take/end was set
if take is None and default_end_trim > 0:
take = max(0.0, full_duration - skip - default_end_trim)
# Calculate effective duration
if take is not None:
effective_duration = min(take, full_duration - skip)
else:
effective_duration = full_duration - skip
effective_duration = min(take, full_duration - skip) if take is not None else full_duration - skip
codec = _get_video_codec(source_path)
is_intra = codec in _INTRA_ONLY_CODECS
if verbose:
print(f" Segment {i+1}: {video_id}")
print(f" Source: {source_path.name}")
print(
f" Skip: {skip}s, Take: {take or 'all'}s, Duration: {effective_duration:.1f}s"
)
mode = "stream copy (fast)" if is_intra else "re-encode"
print(f" {video_id}: {source_path.name} [{codec or '?'}] skip={skip}s take={take or 'all'}s → {effective_duration:.1f}s ({mode})")
segments.append((source_path, skip, take, effective_duration, codec, is_intra))
# Always re-encode every segment to normalize fps and timestamps.
# Mixing un-normalized source files (e.g. 60fps camera) with
# trimmed-and-re-encoded 30fps segments causes cumulative A/V drift
# in the final concat.
# Trim/normalize the segment
trimmed_path = temp_dir / f"segment_{i:03d}.mov"
# Check if source has alpha channel (for ProRes 4444, etc.)
has_alpha = _video_has_alpha(source_path)
# Re-encode to normalize framerate and fix timestamps
# Different segments may have different framerates which breaks concatenation
# ------------------------------------------------------------------ #
# Single-segment fast path — no temp file, no concat round-trip #
# ------------------------------------------------------------------ #
if len(segments) == 1:
source_path, skip, take, effective_duration, codec, is_intra = segments[0]
print(f" Trimming → {output_path.name}" + (" + loudnorm" if needs_loudnorm else "") + "...")
cmd = ["ffmpeg", "-y"]
if skip > 0:
cmd.extend(["-ss", str(skip)])
cmd.extend(["-i", str(source_path)])
if take is not None:
cmd.extend(["-t", str(take)])
if has_alpha:
# Preserve alpha with ProRes 4444
cmd.extend(
[
"-vf",
"fps=30,format=yuva444p10le",
"-c:v",
"prores_ks",
"-profile:v",
"4",
"-pix_fmt",
"yuva444p10le",
"-c:a",
"pcm_s16le",
"-avoid_negative_ts",
"make_zero",
str(trimmed_path),
]
)
if is_intra:
cmd.extend(["-c:v", "copy"])
audio_args = ["-c:a", "copy"]
else:
# No alpha - use fast h264 encoding
cmd.extend(
[
"-vf",
"fps=30",
"-c:v",
"libx264",
"-preset",
"fast",
"-crf",
"18",
"-c:a",
"aac",
"-b:a",
"192k",
"-avoid_negative_ts",
"make_zero",
"-movflags",
"+faststart",
str(trimmed_path),
]
)
video_args, audio_args = _build_reencode_args(source_path)
cmd.extend(video_args)
if needs_loudnorm:
cmd.extend(["-af", loudnorm_filter, "-c:a", "aac", "-b:a", "192k"])
else:
cmd.extend(audio_args)
cmd.extend(["-avoid_negative_ts", "make_zero", str(output_path)])
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
raise PreprocessError(
f"Failed to trim segment {video_id}",
f"Failed to trim segment {segment_ids[0]}",
filter_type="concat",
command=" ".join(cmd),
stderr=result.stderr,
)
total_duration = get_video_duration(output_path)
print(f" Stitched duration: {format_time(total_duration)}")
return output_path
# ------------------------------------------------------------------ #
# Multi-segment path — trim each to temp, then concat + loudnorm #
# ------------------------------------------------------------------ #
temp_dir = output_path.parent / "concat_temp"
temp_dir.mkdir(parents=True, exist_ok=True)
trimmed_segments: list[Path] = []
for i, (source_path, skip, take, effective_duration, codec, is_intra) in enumerate(segments):
trimmed_path = temp_dir / f"segment_{i:03d}.mov"
cmd = ["ffmpeg", "-y"]
if skip > 0:
cmd.extend(["-ss", str(skip)])
cmd.extend(["-i", str(source_path)])
if take is not None:
cmd.extend(["-t", str(take)])
if is_intra:
cmd.extend(["-c:v", "copy", "-c:a", "copy"])
else:
video_args, audio_args = _build_reencode_args(source_path)
cmd.extend(video_args)
cmd.extend(audio_args)
cmd.extend(["-avoid_negative_ts", "make_zero", str(trimmed_path)])
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
raise PreprocessError(
f"Failed to trim segment {segment_ids[i]}",
filter_type="concat",
command=" ".join(cmd),
stderr=result.stderr,
)
trimmed_segments.append(trimmed_path)
# Build concat file list
concat_list = temp_dir / "concat_list.txt"
with open(concat_list, "w", encoding="utf-8") as f:
for segment in trimmed_segments:
f.write(f"file '{segment.resolve()}'\n")
# Concatenate all segments
print(f" Stitching {len(trimmed_segments)} segments -> {output_path.name}")
for seg in trimmed_segments:
f.write(f"file '{seg.resolve()}'\n")
print(f" Stitching {len(trimmed_segments)} segments → {output_path.name}" + (" + loudnorm" if needs_loudnorm else "") + "...")
cmd = [
"ffmpeg",
"-y",
"-f",
"concat",
"-safe",
"0",
"-i",
str(concat_list),
"-c:v",
"copy",
"-c:a",
"copy",
"-movflags",
"+faststart",
str(output_path),
"ffmpeg", "-y",
"-f", "concat", "-safe", "0", "-i", str(concat_list),
"-c:v", "copy",
]
if needs_loudnorm:
cmd.extend(["-af", loudnorm_filter, "-c:a", "aac", "-b:a", "192k"])
else:
cmd.extend(["-c:a", "copy"])
cmd.extend(["-movflags", "+faststart", str(output_path)])
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
@@ -2418,58 +2462,9 @@ def stitch_narration_segments(
stderr=result.stderr,
)
# Apply loudnorm if any segment had defer_loudnorm=True
needs_loudnorm = any(
videos[seg_id].defer_loudnorm for seg_id in segment_ids if seg_id in videos
)
if needs_loudnorm:
print(" Applying loudness normalization to stitched output...")
normalized_path = (
output_path.parent / f"{output_path.stem}_normalized{output_path.suffix}"
)
# Build loudnorm filter string from project config (or fall back to defaults)
_cfg = loudnorm_config or {}
_lufs = float(_cfg.get("target_lufs", -14))
_lra = float(_cfg.get("target_lra", 11))
_tp = float(_cfg.get("target_tp", -1.5))
loudnorm_filter = f"loudnorm=I={_lufs:.1f}:LRA={_lra:.1f}:TP={_tp:.1f}"
loudnorm_cmd = [
"ffmpeg",
"-y",
"-i",
str(output_path),
"-c:v",
"copy",
"-af",
loudnorm_filter,
"-c:a",
"aac",
"-b:a",
"192k",
"-movflags",
"+faststart",
str(normalized_path),
]
result = subprocess.run(loudnorm_cmd, capture_output=True, text=True)
if result.returncode != 0:
raise PreprocessError(
"Loudness normalization failed",
filter_type="loudnorm",
command=" ".join(loudnorm_cmd),
stderr=result.stderr,
)
# Replace original with normalized version
output_path.unlink()
normalized_path.rename(output_path)
print(" Loudness normalization complete.")
# Clean up temp files
for segment in trimmed_segments:
if segment.parent == temp_dir and segment.exists():
if segment.exists():
segment.unlink()
concat_list.unlink()
try:
@@ -2479,5 +2474,4 @@ def stitch_narration_segments(
total_duration = get_video_duration(output_path)
print(f" Stitched duration: {format_time(total_duration)}")
return output_path
gnommo/transformer.py
+57 -11
View File
@@ -974,6 +974,49 @@ def _resolve_video_path(
return source_path, False
def _interpolate_slide_times(
marker_timings: list[MarkerTiming],
slides: dict,
total_duration: float,
) -> list[float]:
"""
Return sorted slide timestamps with unaligned slides (timestamp < 0)
interpolated evenly between their aligned neighbours. Used by both
_extract_slide_events and _extract_video_events so video end-times
never skip over a slide that Whisper failed to align.
"""
all_markers = [
(t.timestamp, t.marker_id)
for t in marker_timings
if t.marker_id in slides
]
if not all_markers:
return []
n = len(all_markers)
resolved = list(all_markers)
i = 0
while i < n:
if resolved[i][0] < 0:
run_start = i
while i < n and resolved[i][0] < 0:
i += 1
run_end = i
prev_time = resolved[run_start - 1][0] if run_start > 0 else 0.0
next_time = resolved[run_end][0] if run_end < n else total_duration
count = run_end - run_start
for j, idx in enumerate(range(run_start, run_end)):
frac = (j + 1) / (count + 1)
resolved[idx] = (
prev_time + (next_time - prev_time) * frac,
resolved[idx][1],
)
else:
i += 1
return sorted(t for t, _ in resolved)
def _extract_slide_events(
marker_timings: list[MarkerTiming],
slides: dict[str, SlideDefinition],
@@ -991,7 +1034,8 @@ def _extract_slide_events(
"""
range_start, range_end = time_range if time_range else (0.0, float("inf"))
# Get ALL slide markers in manuscript order (aligned and unaligned)
# Get ALL slide markers in manuscript order (aligned and unaligned),
# with unaligned ones interpolated via the shared helper.
all_slide_markers: list[tuple[float, str]] = []
for timing in marker_timings:
if timing.marker_id in slides:
@@ -1000,9 +1044,8 @@ def _extract_slide_events(
if not all_slide_markers:
return []
# Interpolate timestamps for unaligned slides (timestamp < 0).
# For each run of consecutive unaligned slides, spread them evenly between
# the nearest aligned slides before and after in manuscript order.
# Re-derive interpolated times (same logic as _interpolate_slide_times but
# we need the (time, id) pairs here for event building).
n = len(all_slide_markers)
resolved: list[tuple[float, str]] = list(all_slide_markers)
@@ -1075,13 +1118,12 @@ def _extract_video_events(
warnings: list[str] = []
range_start, range_end = time_range if time_range else (0.0, float("inf"))
# Collect slide times for video: end time calculation
slide_times: list[float] = sorted(
[
t.timestamp
for t in marker_timings
if t.marker_id in slides and t.timestamp >= 0
]
# Collect slide times for video end-time calculation.
# Use the interpolated times (same as _extract_slide_events) so that a slide
# Whisper failed to align doesn't get skipped, causing the preceding video to
# bleed through into the following slide.
slide_times: list[float] = _interpolate_slide_times(
marker_timings, slides, total_duration
)
# Pause-variant prefixes — the only thing the render pass still needs from
@@ -1177,6 +1219,10 @@ def _extract_video_events(
if slide_time > start_time:
end_time = slide_time
break
# pause_narration videos must stay visible for the full pause duration —
# the narration is held for that long, so the overlay should match.
if video_source.pause_narration:
end_time = max(end_time, start_time + video_source.pause_narration)
else:
# end_on is None and marker_type == "narration": runs to end
end_time = total_duration
short_is_universe_pixelated/project.json
+3 -3
View File
@@ -21,9 +21,9 @@
"defaultSlideType": "fullscreen",
"cutouts": {
"talkinghead": {
"x": "-23%",
"y": "10%",
"height": "90%"
"x": "-10%",
"y": "40%",
"height": "80%"
},
"fullscreen": {
"x": "0%",