Compare commits
62
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0c8f662bee | ||
|
|
cdda2e9024 | ||
|
|
a8aab55bd2 | ||
|
|
4e1bfe03e2 | ||
|
|
9824f6d081 | ||
|
|
46e243b4dc | ||
|
|
f79a0cb082 | ||
|
|
e7dc402d9e | ||
|
|
409145f214 | ||
|
|
a84e02b494 | ||
|
|
b60be96978 | ||
|
|
9f30698801 | ||
|
|
a91acca695 | ||
|
|
7312fa2366 | ||
|
|
35c5e52fb0 | ||
|
|
5a439380ae | ||
|
|
c3e892cf89 | ||
|
|
0d48a38516 | ||
|
|
8247e5aa65 | ||
|
|
78e3aef0f2 | ||
|
|
09620c4e61 | ||
|
|
5f974a710e | ||
|
|
4913195b9d | ||
|
|
ee4d0b8b0e | ||
|
|
2ef88f3657 | ||
|
|
de329a8679 | ||
|
|
b1136a9f7d | ||
|
|
a3919f595a | ||
|
|
715a36cf6e | ||
|
|
cc566208c5 | ||
|
|
99ddc2e425 | ||
|
|
71f2c51b1b | ||
|
|
17e35920b6 | ||
|
|
4a05c1c78d | ||
|
|
c65b246401 | ||
|
|
e5bb437768 | ||
|
|
fa8512ff3d | ||
|
|
24b04fd55e | ||
|
|
316441ccb8 | ||
|
|
38e6c7940c | ||
|
|
83715e85e8 | ||
|
|
56e1cd985e | ||
|
|
eb9196e50d | ||
|
|
ce9c35c9b2 | ||
|
|
456070903c | ||
|
|
e8fc8411ea | ||
|
|
c4dedfa720 | ||
|
|
02a6131d15 | ||
|
|
bb1b17d531 | ||
|
|
c30c0f1c5e | ||
|
|
c6efbaf2ed | ||
|
|
7f9f8e3395 | ||
|
|
60478e13a3 | ||
|
|
9abaa237f7 | ||
|
|
d5dc3c5e33 | ||
|
|
1fcd511f77 | ||
|
|
308a9f8bcd | ||
|
|
f9ff847f6b | ||
|
|
b9b5a8e77d | ||
|
|
980bb84dac | ||
|
|
20aba06be1 | ||
|
|
12b052eb1d |
@@ -0,0 +1,35 @@
|
|||||||
|
### Problems
|
||||||
|
|
||||||
|
# Video 1 :
|
||||||
|
Music ok
|
||||||
|
Talkinghead too high, move down.
|
||||||
|
|
||||||
|
Glitch logo lasts 1 seconds, but render shows it should last 6 seconds and pause narration for 6 seconds.
|
||||||
|
The narration correctly resumes after 6 seconds, leaving. The problem appears to be that S2 is indicated to start at 00:32.12 "Welcome to this video series about digital philosophy", which is incorrect. It should start at 00.37 or 00.38. So the positioning of the slide does not respect the pause narration.
|
||||||
|
|
||||||
|
Sent to claude
|
||||||
|
|
||||||
|
|
||||||
|
02.07 slide of xray background with no text
|
||||||
|
|
||||||
|
02.30 : Move to "not just the sky around Antares" in a frame?
|
||||||
|
|
||||||
|
Discrete font on the one, two, three slide.
|
||||||
|
|
||||||
|
Black frame on wavepacket
|
||||||
|
|
||||||
|
Gedankenexperiment missing slide
|
||||||
|
Done
|
||||||
|
|
||||||
|
"Assume the universe is fundamentally discrete" weird pause. Fixed
|
||||||
|
|
||||||
|
Need a slide before "Something else that has".
|
||||||
|
|
||||||
|
13.42: The Glitch slide. Talking head is above the cloud background. Cloud background should be mid level.
|
||||||
|
Change to glitch video
|
||||||
|
|
||||||
|
DOne
|
||||||
|
|
||||||
|
# Video 2 :
|
||||||
|
|
||||||
|
The same problem as v1, the vf2m appears below the talking head. But the order is supposed to be
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
{
|
||||||
|
"musicforyourdreams": {
|
||||||
|
"file": "musicforyourdreams.m4a",
|
||||||
|
"is_shared": true,
|
||||||
|
"volume": 1.0,
|
||||||
|
"duration": 448.832
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -10,8 +10,9 @@ Like Minecraft - just with absurdly tiny blocks?
|
|||||||
|
|
||||||
[S3]
|
[S3]
|
||||||
This question has been around forever.
|
This question has been around forever.
|
||||||
But it's always been filed under "too weird to bother."
|
But it seems too weird to be worth trying.
|
||||||
That's about to change.
|
Nobody's really tried to take it seriously.
|
||||||
|
Until now
|
||||||
|
|
||||||
[S4]
|
[S4]
|
||||||
Explore the tech-tree.
|
Explore the tech-tree.
|
||||||
@@ -21,7 +22,6 @@ We're committed to scientific rigour.
|
|||||||
Falsifiability. Truth-seeking.
|
Falsifiability. Truth-seeking.
|
||||||
And not being a complete bore.
|
And not being a complete bore.
|
||||||
|
|
||||||
|
|
||||||
[S5]
|
[S5]
|
||||||
Don’t enroll now.
|
Don’t enroll now.
|
||||||
Enroll later.
|
Enroll later.
|
||||||
|
|||||||
@@ -1,26 +1,26 @@
|
|||||||
{
|
{
|
||||||
"S1": {
|
"S1": {
|
||||||
"image": "GlitchTrailer.001.png",
|
"image": "glitchtrailer.001.png",
|
||||||
"type": "fullscreen"
|
"type": "fullscreen"
|
||||||
},
|
},
|
||||||
"S2": {
|
"S2": {
|
||||||
"image": "GlitchTrailer.002.png",
|
"image": "glitchtrailer.002.png",
|
||||||
"type": "fullscreen"
|
"type": "fullscreen"
|
||||||
},
|
},
|
||||||
"S3": {
|
"S3": {
|
||||||
"image": "GlitchTrailer.003.png",
|
"image": "glitchtrailer.003.png",
|
||||||
"type": "fullscreen"
|
"type": "fullscreen"
|
||||||
},
|
},
|
||||||
"S4": {
|
"S4": {
|
||||||
"image": "GlitchTrailer.004.png",
|
"image": "glitchtrailer.004.png",
|
||||||
"type": "fullscreen"
|
"type": "fullscreen"
|
||||||
},
|
},
|
||||||
"S5": {
|
"S5": {
|
||||||
"image": "GlitchTrailer.005.png",
|
"image": "glitchtrailer.005.png",
|
||||||
"type": "fullscreen"
|
"type": "fullscreen"
|
||||||
},
|
},
|
||||||
"S6": {
|
"S6": {
|
||||||
"image": "GlitchTrailer.006.png",
|
"image": "glitchtrailer.006.png",
|
||||||
"type": "fullscreen"
|
"type": "fullscreen"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
+12
-10
@@ -7,13 +7,14 @@
|
|||||||
"platform_targets": ["youtube"],
|
"platform_targets": ["youtube"],
|
||||||
"status": "scripted",
|
"status": "scripted",
|
||||||
"youtube_url": null,
|
"youtube_url": null,
|
||||||
"resolution": [1960, 1080],
|
"resolution": [1920, 1080],
|
||||||
"fps": 30,
|
"fps": 30,
|
||||||
"duration_seconds": null,
|
"duration_seconds": null,
|
||||||
"default_filters": {
|
"default_filters": {
|
||||||
"audioonly": [
|
"audioonly": [
|
||||||
{
|
{
|
||||||
"type": "audio_normalize",
|
"type": "audio_normalize",
|
||||||
|
"enable":false,
|
||||||
"compress": false,
|
"compress": false,
|
||||||
"normalize": true,
|
"normalize": true,
|
||||||
"target_lufs": -14,
|
"target_lufs": -14,
|
||||||
@@ -24,6 +25,7 @@
|
|||||||
"talkinghead": [
|
"talkinghead": [
|
||||||
{
|
{
|
||||||
"type": "audio_normalize",
|
"type": "audio_normalize",
|
||||||
|
"enable":false,
|
||||||
"normalize": true,
|
"normalize": true,
|
||||||
"target_lufs": -14,
|
"target_lufs": -14,
|
||||||
"target_lra": 11,
|
"target_lra": 11,
|
||||||
@@ -46,9 +48,9 @@
|
|||||||
"screen_gain": 175,
|
"screen_gain": 175,
|
||||||
"screen_balance": 58,
|
"screen_balance": 58,
|
||||||
"despill_bias": [
|
"despill_bias": [
|
||||||
217,
|
235,
|
||||||
240,
|
222,
|
||||||
255
|
210
|
||||||
],
|
],
|
||||||
"despill_strength": 5.0,
|
"despill_strength": 5.0,
|
||||||
"edge_erode": 1.0,
|
"edge_erode": 1.0,
|
||||||
@@ -66,15 +68,15 @@
|
|||||||
},
|
},
|
||||||
"cutouts": {
|
"cutouts": {
|
||||||
"talkinghead": {
|
"talkinghead": {
|
||||||
"x": "-23%",
|
"x": "-10%",
|
||||||
"y": "10%",
|
"y": "40%",
|
||||||
"height": "90%"
|
"height": "80%"
|
||||||
},
|
},
|
||||||
"square": {
|
"square": {
|
||||||
"x": "46.5%",
|
"x": "47.91875%",
|
||||||
"y": "4.5%",
|
"y": "5.55%",
|
||||||
"width": "50%",
|
"width": "50%",
|
||||||
"height": "90%"
|
"height": "88.888888%"
|
||||||
},
|
},
|
||||||
"fullscreen": {
|
"fullscreen": {
|
||||||
"x": "0%",
|
"x": "0%",
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,2 @@
|
|||||||
|
== Description ==
|
||||||
|
Welcome to Glitch University.
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
[S1]
|
||||||
|
Today we’re not proving quantum mechanics is discrete.
|
||||||
|
We’re running a test.
|
||||||
|
One test, named INT5000.
|
||||||
|
|
||||||
|
[S2]
|
||||||
|
We already completed some simple tests. We called them the naïve test suite.
|
||||||
|
It required us to define what is the Action, how does stuff move around.
|
||||||
|
And what is momentum.
|
||||||
|
But might be neat, but all it showed is that the CA glider has vaguely particle-like properties.
|
||||||
|
But a fundamentally discrete universe is 50% of the option space. The other 50% is a continuous universe. Ruling out 50% of the option space like a decent bang for the buck, so we committed to giving a discrete universe every chance it can have to be true. Even if it sounds unlikely.
|
||||||
|
We need to make it pass more tests. Lets introduce the advanced test suite.
|
||||||
|
|
||||||
|
[S3]
|
||||||
|
If quantum mechanics is emergent from something discrete, there are many constraints it must satisfy.
|
||||||
|
It needs Lorentz invariance, otherwise it contradicts Einstein.
|
||||||
|
Most of these tests are beyond my abilities, even with AI assistance.
|
||||||
|
But there is one I’m excited about, and it’s this one.
|
||||||
|
Can a Cellular Automaton reproduce the spinorial rotation structure - SU(2)?
|
||||||
|
|
||||||
|
[S4]
|
||||||
|
In quantum mechanics, certain objects - spinors - behave strangely.
|
||||||
|
Rotate them 360 degrees, and they don’t return to their original state.
|
||||||
|
They pick up a minus sign.
|
||||||
|
Only after 720 degrees do they truly return.
|
||||||
|
This behavior is encoded mathematically in a group called SU(2).
|
||||||
|
If we can’t reproduce this, we can’t reproduce spin.
|
||||||
|
If we can’t do that, QM is out of reach. And that means we’re back to the default scenario.
|
||||||
|
|
||||||
|
[S5]
|
||||||
|
And the default scenario is - that the Naga is an neat and weird Cellular Automaton, of no significance to physics at all.
|
||||||
|
Just like this channel, with its middle aged amateur creator dressed up in a yellow jumpsuit
|
||||||
|
Trying to do stuff I don’t have the necessary training to do.
|
||||||
|
This is the most likely outcome. I owning it, not afraid of it. It doesn’t hurt anyone, so lets keep going.
|
||||||
|
|
||||||
|
[S6]
|
||||||
|
And as you know, physicists generally don’t understand code-speak, and developers struggle with physics-speak. So we’ll keep translating showing both sides.
|
||||||
|
In Code speak - SU(2) is a interface exposed by QM that we must support.
|
||||||
|
If we can’t support it, the migration fails.
|
||||||
|
In physics speak - we isolate the SU(2) representation structure. Then we evaluate only whether the generators exposed by the proposed structure allows to satisfy the Lie algebra.
|
||||||
|
|
||||||
|
[S7]
|
||||||
|
Basically we must derive SU(2) from the Naga in an elegant way. If we make any assumptions doing so, those same assumptions carry over to all other integrations tests.
|
||||||
|
That’s the contract.
|
||||||
|
|
||||||
|
[S8]
|
||||||
|
Importantly, this test does not require
|
||||||
|
Not Lorentz invariance.
|
||||||
|
Not isotropy of space.
|
||||||
|
Not full quantum field theory.
|
||||||
|
Just this single interface Naga - > SU(2)
|
||||||
|
|
||||||
|
[S9]
|
||||||
|
It might be unnecessary to say this, but you never know, some people are sloppy when meting out criticism. So passing this test does NOT mean
|
||||||
|
1. that QM is discrete
|
||||||
|
2. That Bells theorem is not relevant
|
||||||
|
3. Or that the theory not falsifiable
|
||||||
|
It just mean that given the assumptions and tests, the test either fails or passes.
|
||||||
|
If it passes, it earns the right to attempt the next one.
|
||||||
|
That’s it. It’s test-driven development.
|
||||||
|
|
||||||
|
[S10]
|
||||||
|
A Cellular Automaton is discrete.
|
||||||
|
SU(2) is continuous.
|
||||||
|
To derive something continuous from something discrete requires that the continuous is an approximation to the discrete.
|
||||||
|
And this is very
|
||||||
|
|
||||||
|
[S11]
|
||||||
|
So Spinors liven 3D space. Which means it should be possible to rotate them like this cup. I rotate it 360° brings you back.
|
||||||
|
But spinors are weird, they don’t work like that.
|
||||||
|
It’s kind of like this sock. Now if I rotate this sock 360 degrees, it returns back to where it was. But spinors is kind of like a sock that gets inside out each time it rotates. So when I rotate it 360 degrees it inverts. This inversion is captured by this extra -1. Then if I rotate it again it inverts again. So to return completely to both the same position and the same inside-out, we need two rotations.
|
||||||
|
That “extra twist” is not decorative is deep in QM. You’re made from objets which rotate in this way, not the normal way.
|
||||||
|
|
||||||
|
[S12]
|
||||||
|
What this means for our CA, is that if it naturally accumulates some kind of twist deficit after 2π that only resolves after 4π,
|
||||||
|
we’re in business.
|
||||||
|
Mathematically - we need a two-valued memory of rotation that flips sign after 2π.
|
||||||
|
|
||||||
|
[S13]
|
||||||
|
So lets have a look at the CA. Let us first identify when it goes back to normal. We look at the head here, and keep iterating until the head is back in the original position. This takes N steps. So N steps is a full rotation. Now what happens if we apply just half of these steps, where do we en up - we end up here.
|
||||||
|
Periodicity alone is not enough. We now examine how orientation accumulates during those steps.
|
||||||
|
|
||||||
|
[S14]
|
||||||
|
And if we look carefully, we executed half of the X, half of the Y and half the Z. Which means that this midway point corresponds to half a rotation in all three planes. See here xy-plane, half rotated. Xz plane, half rotated. Yz plane half rotated. Meaning that T = N gives us a full rotation in all these three planes.
|
||||||
|
A full rotation in one plane is 2pi, so a full rotation in all three plane should be 6pi. That is surprising. But look
|
||||||
|
|
||||||
|
[S15]
|
||||||
|
A full rotation in the xy plane is also half a rotation in the xz plane, because the x-components are shared.
|
||||||
|
And a full rotation in they yz plane is also a half rotation in the xz plane, because the z components are shared.
|
||||||
|
Meaning that a full rotation in two planes gives us the full rotation in the third plane for free. 2/3 out of 6 pi is 4 pi.
|
||||||
|
|
||||||
|
[S16]
|
||||||
|
Did that not convince you? Let’s do it differently.
|
||||||
|
Suppose you define a rotation as the theta value of the HEAD relative to the center in a chosen plane.
|
||||||
|
You will now have three rotation generators.
|
||||||
|
You can’t add them together because they are never orthogonal to both other planes at the same time.
|
||||||
|
Each theta value is orthogonal to half the theta value in each of the two remaining planes.
|
||||||
|
Meaning you have theta = theta_xy + 1/2 theta_xz + 1/2 theta_yz.
|
||||||
|
A slightly more general form of this is : 2/3(theta_xz + theta_xy + theta_yz)
|
||||||
|
So if after a full rotation in a plane theta_xy = theta_xz = theta_yz = 2\pi, then the full rotation becomes
|
||||||
|
2/3 (2 pi + 2pi + 2pi ) = 4 pi
|
||||||
|
|
||||||
|
[S17]
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
{
|
||||||
|
"musicforyourdreams": {
|
||||||
|
"file": "musicforyourdreams.m4a",
|
||||||
|
"is_shared": true,
|
||||||
|
"volume": 1.0
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
{
|
||||||
|
"S1": {
|
||||||
|
"image": "int5000.001.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S2": {
|
||||||
|
"image": "int5000.002.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S3": {
|
||||||
|
"image": "int5000.003.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S4": {
|
||||||
|
"image": "int5000.004.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S5": {
|
||||||
|
"image": "int5000.005.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S6": {
|
||||||
|
"image": "int5000.006.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S7": {
|
||||||
|
"image": "int5000.007.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S8": {
|
||||||
|
"image": "int5000.008.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S9": {
|
||||||
|
"image": "int5000.009.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S10": {
|
||||||
|
"image": "int5000.010.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S11": {
|
||||||
|
"image": "int5000.011.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S12": {
|
||||||
|
"image": "int5000.012.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S13": {
|
||||||
|
"image": "int5000.013.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S14": {
|
||||||
|
"image": "int5000.014.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S15": {
|
||||||
|
"image": "int5000.015.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S16": {
|
||||||
|
"image": "int5000.016.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
},
|
||||||
|
"S17": {
|
||||||
|
"image": "int5000.017.png",
|
||||||
|
"type": "fullscreen"
|
||||||
|
}
|
||||||
|
}
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -111,6 +111,29 @@ gnommo -p myproject import
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
### Stage 3: Grading
|
||||||
|
|
||||||
|
Iterate on keying/grading without running a full preprocess. It seeks a few
|
||||||
|
seconds into a raw clip, runs it through the `talkinghead` filter chain, and
|
||||||
|
writes `grade_preview.mov` (ProRes 4444 with alpha) to the project root.
|
||||||
|
|
||||||
|
# 1. KEY — auto, objective. Generates a candidate + preview, applies on pick.
|
||||||
|
gnommo -p video3 grade --stage key
|
||||||
|
open video3/grade_sweep/key_1.png # sanity-check the matte
|
||||||
|
gnommo -p video3 grade --pick key_1 # writes it to project.json
|
||||||
|
|
||||||
|
# 2. DESPILL — sweep, YOU pick. 7 stills, spill_suppress 0.7–1.5.
|
||||||
|
gnommo -p video3 grade --stage despill
|
||||||
|
open video3/grade_sweep/ # eyeball despill_1..7.png
|
||||||
|
gnommo -p video3 grade --pick despill_5 # apply whichever looks clean
|
||||||
|
|
||||||
|
# 3. GRADE — sweep, YOU pick. 6 stills, paleness 0.0–1.0.
|
||||||
|
gnommo -p video3 grade --stage grade
|
||||||
|
open video3/grade_sweep/ # eyeball grade_1..6.png
|
||||||
|
gnommo -p video3 grade --pick grade_3 # apply the paleness you like
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### Stage 3: Preprocess
|
### Stage 3: Preprocess
|
||||||
|
|
||||||
Applies video filters (chroma key, scaling, etc.) to narration segments.
|
Applies video filters (chroma key, scaling, etc.) to narration segments.
|
||||||
@@ -126,6 +149,58 @@ gnommo -p myproject pre
|
|||||||
|
|
||||||
**When to use:** After recording narration that needs background removal, sound normalization or other processing.
|
**When to use:** After recording narration that needs background removal, sound normalization or other processing.
|
||||||
|
|
||||||
|
#### Green screen keying (`gnommokey`)
|
||||||
|
|
||||||
|
The `talkinghead` preset in `project.json` chains a `gnommokey` keyer (a
|
||||||
|
Keylight-style color-difference keyer) with a `color_grade` step. Key fields:
|
||||||
|
|
||||||
|
| Field | What it does |
|
||||||
|
|-------|--------------|
|
||||||
|
| `screen_color` | RGB of your green screen (sampled from the footage). |
|
||||||
|
| `screen_gain` | Key extraction strength. Higher = more aggressive matte. |
|
||||||
|
| `screen_balance` | Mixes luminance into the key (helps darker/lighter greens). |
|
||||||
|
| `clip_black` / `clip_white` | Compress the matte range (crush transparent / opaque). |
|
||||||
|
| `despill_bias` | RGB the keyed *edges* shift toward. A light neutral/skin tone reads better than cool blue. |
|
||||||
|
| `despill_strength` | How hard the edge/dominant-green despill pulls toward `despill_bias`. |
|
||||||
|
| `spill_suppress` | **Interior green-limiter — see below.** |
|
||||||
|
| `yellow_protect` | Shields saturated yellows/warm fabrics from `spill_suppress` (0–1). |
|
||||||
|
| `edge_erode` | Shrinks the alpha edge by N passes to kill green fringe (0–5). |
|
||||||
|
|
||||||
|
**`spill_suppress` — the bald-head knob.** The regular despill only acts where
|
||||||
|
green is the *dominant* channel (`g > max(r,b)`). On skin — especially a bald
|
||||||
|
head catching green bounce — red stays dominant, so that despill never touches
|
||||||
|
the interior and a green cast survives. `spill_suppress` caps green across the
|
||||||
|
**whole frame** at a reference extrapolated through the other two channels:
|
||||||
|
|
||||||
|
```
|
||||||
|
0.0 → cap at max(r, b) (only strong spill; effectively off)
|
||||||
|
1.0 → cap at avg(r, b) (removes visible cast)
|
||||||
|
2.0 → cap at min(r, b) (green can never exceed the smallest channel — nuclear)
|
||||||
|
```
|
||||||
|
|
||||||
|
Green is only ever *reduced*, never boosted, so clean pixels are untouched.
|
||||||
|
Start around `0.6`; go past `1.0` for heavy close-up spill. If skin tips
|
||||||
|
magenta/pink, ease back down. Range is `0.0`–`2.0`.
|
||||||
|
|
||||||
|
**`yellow_protect` — keep yellows yellow.** Because `spill_suppress` caps green
|
||||||
|
*everywhere*, it also drains green from things that are legitimately yellow (a
|
||||||
|
yellow jumpsuit, warm props), turning them orange. Blue is the tell: skin keeps
|
||||||
|
some blue, saturated yellow fabric reflects almost none. `yellow_protect`
|
||||||
|
(0.0–1.0) gates the green-limiter back down where `min(r,g) − b` is high
|
||||||
|
(yellow) while leaving skin/scalp spill fully suppressed. Bump it toward `1.0`
|
||||||
|
if warm colours go orange; leave at `0` if you have no strong yellows.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
```bash
|
||||||
|
gnommo -p myproject grade # first raw_mov clip, 3s from 5s in
|
||||||
|
gnommo -p myproject grade --ss 12 --dur 4 # seek 12s in, make a 4s preview
|
||||||
|
gnommo -p myproject grade --file media/narration/raw_mov/clipA.mov
|
||||||
|
```
|
||||||
|
|
||||||
|
Tweak the `gnommokey` / `color_grade` values in `project.json`, re-run `grade`,
|
||||||
|
re-open the `.mov`, repeat until it looks right.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### Stage 4: stitch
|
### Stage 4: stitch
|
||||||
@@ -168,6 +243,53 @@ gnommo -p myproject render --res low # Fast preview at 490x270
|
|||||||
gnommo -p myproject render --res tiny # Ultrafast preview at 320x180
|
gnommo -p myproject render --res tiny # Ultrafast preview at 320x180
|
||||||
```
|
```
|
||||||
|
|
||||||
|
A partial `--slides S1:S10` render writes a range-suffixed file (e.g.
|
||||||
|
`PHIL_COSM_102_S1_S10.mp4`) so sections don't overwrite each other or the full render.
|
||||||
|
|
||||||
|
**Render log:** every render writes `<project>/<project>.log` (e.g. `video2.log`) with
|
||||||
|
the platform/ffmpeg/memory header, the exact ffmpeg command, and a `[mem …]` memory
|
||||||
|
sample every 3s. If a render crashes, check the tail of this log first.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Render rig configuration (memory / performance)
|
||||||
|
|
||||||
|
The compositing ffmpeg graph holds a lot at once (RGBA layer buffers, many inputs, the
|
||||||
|
final mux), so peak RAM is high. Two knobs keep it bounded — both matter on a render rig.
|
||||||
|
|
||||||
|
**1. FFmpeg thread cap — `~/.gnommo.conf`** (on the render machine)
|
||||||
|
|
||||||
|
Fewer filter threads = far less peak memory (each parallel `format=rgba`/swscaler stage
|
||||||
|
holds its own full-frame buffers). The render honours `[performance] cpu_limit`, a
|
||||||
|
fraction of logical CPUs. **Unset defaults to 1 thread (safest).** On a memory-tight box
|
||||||
|
keep it low:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[performance]
|
||||||
|
cpu_limit = 0.25
|
||||||
|
```
|
||||||
|
|
||||||
|
> Note: the render uses `-filter_complex`, capped by `-filter_complex_threads` (not
|
||||||
|
> `-filter_threads`, which only applies to simple `-vf` graphs). This is why an
|
||||||
|
> uncapped render graph could OOM even when the preprocessor was fine.
|
||||||
|
|
||||||
|
**2. WSL2 memory/swap — `C:\Users\<you>\.wslconfig`** (Windows host, for an Ubuntu-on-WSL rig)
|
||||||
|
|
||||||
|
A WSL2 VM only gets a *slice* of host RAM (default ~50%, or 8 GB on older builds). If the
|
||||||
|
render exceeds that slice, **Windows OOM-kills the whole VM** — it surfaces as
|
||||||
|
`Wsl/Service/E_UNEXPECTED` / "Catastrophic failure", not a normal out-of-memory error, and
|
||||||
|
the Windows host still shows plenty of RAM free. Raise the cap and give it swap headroom:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[wsl2]
|
||||||
|
memory=24GB # give the VM more of the host RAM
|
||||||
|
swap=16GB # headroom so it pages instead of dying catastrophically
|
||||||
|
processors=8
|
||||||
|
```
|
||||||
|
|
||||||
|
Then, from PowerShell: `wsl --shutdown`, and restart the session. Confirm the VM's cap in
|
||||||
|
the render log header — its `memory: … total` is the VM slice, not the host RAM.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Shortcut: All Stages
|
## Shortcut: All Stages
|
||||||
@@ -267,6 +389,7 @@ gnommo -p myproject render
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
gnommo -p myproject validate # Check for errors without rendering
|
gnommo -p myproject validate # Check for errors without rendering
|
||||||
|
gnommo -p myproject grade # Preview the keyer/grade on a few seconds of raw footage
|
||||||
gnommo -p myproject description # Generate YouTube description with chapters
|
gnommo -p myproject description # Generate YouTube description with chapters
|
||||||
gnommo -p myproject transcribe --final # Transcribe final.mp4 for subtitles
|
gnommo -p myproject transcribe --final # Transcribe final.mp4 for subtitles
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -1,9 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
|
|
||||||
./gnommo.sh -p video1 all --force --prod
|
|
||||||
./gnommo.sh -p video2 all --force --prod
|
|
||||||
./gnommo.sh -p video3 all --force --prod
|
|
||||||
#./gnommo.sh -p video4 all --force
|
|
||||||
#./gnommo.sh -p video5 all --force
|
|
||||||
#./gnommo.sh -p video6 all --force
|
|
||||||
|
|
||||||
@@ -0,0 +1,165 @@
|
|||||||
|
# Atomic Events — Design Spec
|
||||||
|
|
||||||
|
Status: **Stage A + Stage B implemented (2026-07-27).** Motivated by a future **Glitch
|
||||||
|
Studio GUI** that edits each video occurrence as a self-contained object.
|
||||||
|
|
||||||
|
Implemented:
|
||||||
|
- Per-occurrence presentation resolves via `transformer.resolve_video_presentation`
|
||||||
|
(precedence: inline/GUI override > shorthand prefix > videos.json > default; video
|
||||||
|
`end_on` default = `next_video`, `[narration:]` runs to end).
|
||||||
|
- events.json is materialized/atomic: `derive_events` writes `handle/cutout/layer/end_on/
|
||||||
|
take`; `events_to_marker_timings` round-trips them as overrides.
|
||||||
|
- Inline grammar `[prefix:handle, key=value, …]` (`parser.parse_marker`), threaded through
|
||||||
|
alignment into `MarkerTiming.overrides`. Supported inline keys: **cutout, layer, end_on,
|
||||||
|
take** (the fully-wired per-event fields). Unknown keys are ignored.
|
||||||
|
- The key-reuse collision validator hard-error was removed (reuse is legal now).
|
||||||
|
|
||||||
|
Deferred (follow-ups): inline override of the *global* params (skip/zoom/volume/
|
||||||
|
use_audio_channels/pause_narration) — the renderer reads these from `video_source` in ~13
|
||||||
|
places, so wiring them per-event is a separate change; stripping the moved fields from
|
||||||
|
videos.json (kept as fallback defaults for now); a validator warning for unknown/unwired
|
||||||
|
inline keys.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
Presentation/timing properties (`cutout`, `layer`, `end_on`, `take`, `pause_narration`)
|
||||||
|
live on the **videos.json handle**, but they are really properties of *where a clip is
|
||||||
|
used*, not of the file. The shorthand prefix (`vst:` = square/above, `vsb:` =
|
||||||
|
square/below) is per-marker, but `_project_markers_to_videos` collapses it onto the
|
||||||
|
single handle record (last-wins). So one handle used two ways collides:
|
||||||
|
|
||||||
|
- `[vst:glitch_ccd_binning]` (above) and `[vsb:glitch_ccd_binning]` (below) → videos.json
|
||||||
|
can only store `layer: below`, so the first occurrence renders under the slide (hidden).
|
||||||
|
- video5 has 5 such collisions today (glitch_ccd_binning, pexels/12471039…,
|
||||||
|
mainvideopart1, shotnoiseacc, slide_periodogram).
|
||||||
|
|
||||||
|
A stopgap validator hard-error (`validate_project`, gnommo/validator.py) currently blocks
|
||||||
|
render on these. This spec removes the *cause* so that guard is no longer needed.
|
||||||
|
|
||||||
|
The naive fixes are both rejected: copying the file/handle (duplication on disk), and a
|
||||||
|
"hybrid override + materialize" layer (too much indirection). Instead: **the per-occurrence
|
||||||
|
properties move onto the event.**
|
||||||
|
|
||||||
|
## Field homes
|
||||||
|
|
||||||
|
**videos.json — asset + global defaults (one value per handle):**
|
||||||
|
`source_file`, `output_file`/`processed_file`, `filter`, `has_audio`, `is_shared`,
|
||||||
|
`src_mtime`, `duration` (probed; asset-only, never per-event), and the globals
|
||||||
|
`zoom`, `skip`, `volume`, `use_audio_channels`.
|
||||||
|
|
||||||
|
**events.json — per-occurrence (one value per event):**
|
||||||
|
`handle` (the video id, **prefix-free**), `cutout`, `layer`, `end_on`, `take`,
|
||||||
|
`pause_narration`.
|
||||||
|
|
||||||
|
**Resolution order for a rendered clip:** event field (if set) → videos.json value (for the
|
||||||
|
globals) → config default. The per-occurrence fields have no videos.json fallback — they
|
||||||
|
are always materialized onto the event at build time.
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
- `end_on` **defaults to `next_video`** for videos when unspecified (was implicitly
|
||||||
|
`next_slide`). Existing videos.json `end_on` values are migrated onto events explicitly,
|
||||||
|
so current projects keep their behavior; only *new* unspecified markers get the new default.
|
||||||
|
- `take` is the event-level cut length, only meaningful when `end_on=take`; otherwise the
|
||||||
|
end is implicit from `end_on` and `take` stays null.
|
||||||
|
- `skip` stays a global (asset trim-in) while `take` is per-event — a deliberate asymmetry:
|
||||||
|
"where this asset generally starts" vs. "how long this occurrence plays."
|
||||||
|
- `zoom`/`volume`/`use_audio_channels` stay global but are inline-overridable per event
|
||||||
|
(below), so they can diverge without a videos.json copy.
|
||||||
|
|
||||||
|
## Authoring: shorthand + inline overloads
|
||||||
|
|
||||||
|
The manuscript stays the compact authoring surface. The shorthand letters encode
|
||||||
|
`cutout`+`layer` (and `pause_narration` via the `…p:` variants). Anything the letters
|
||||||
|
don't encode — chiefly `end_on`, and any per-event override of a global — is given as
|
||||||
|
inline **`key=value`** pairs (simplified from the earlier `{"json":"form"}`):
|
||||||
|
|
||||||
|
```
|
||||||
|
[vsb:glitch_ccd_binning2] # square/below, end_on defaults to next_video
|
||||||
|
[vsb:glitch_ccd_binning2, end_on=next_video] # + explicit end_on
|
||||||
|
[vsb:glitch_ccd_binning2, take=5, volume=0.5] # + per-event overrides of globals
|
||||||
|
[video:glitch_ccd_binning2, cutout=square, layer=below] # generic; equivalent to [vsb:…]
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
- The first token inside `[]` is `prefix:handle` (handle may contain `/`, e.g. `pexels/123`).
|
||||||
|
- Remaining comma-separated tokens are `key=value`. Values are type-inferred: numeric →
|
||||||
|
float, `true`/`false` → bool, else string. Allowed keys: `cutout`, `layer`, `end_on`,
|
||||||
|
`take`, `skip`, `zoom`, `volume`, `use_audio_channels`, `pause_narration`,
|
||||||
|
`always_visible`.
|
||||||
|
- An inline key overrides whatever the shorthand implied (e.g. `[vst:x, layer=below]` →
|
||||||
|
above from the prefix, then below from the override). Last-writer-wins, prefix first.
|
||||||
|
- `[video:handle, …]` is the fully-explicit form the GUI round-trips: no prefix magic, every
|
||||||
|
presentation field named.
|
||||||
|
|
||||||
|
Why `key=value` over JSON: no braces/quotes to escape inside `[]`, one obvious separator,
|
||||||
|
and it reads cleanly in a script. The GUI still stores the resolved values as real JSON
|
||||||
|
fields on the event — the manuscript form is just sugar that populates them.
|
||||||
|
|
||||||
|
## Build-time materialization
|
||||||
|
|
||||||
|
At build (`build_render_plan` / scaffold construction), each video marker resolves to an
|
||||||
|
atomic event dict:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"type": "video",
|
||||||
|
"handle": "glitch_ccd_binning",
|
||||||
|
"cutout": "square",
|
||||||
|
"layer": "above",
|
||||||
|
"end_on": "next_video",
|
||||||
|
"take": null,
|
||||||
|
"pause_narration": 0.0,
|
||||||
|
"narration_time": 0.0, "adjustment": 0.0, "final_time": 0.0,
|
||||||
|
"mapping": "exact", "confidence": 1.0, "context": "…"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`id` (currently `"vst:glitch_ccd_binning"`) is replaced by `handle` + explicit fields. The
|
||||||
|
render pass reads presentation straight off the event and no longer consults the prefix or
|
||||||
|
the videos.json presentation fields. `merge_events` must preserve manual event edits (the
|
||||||
|
GUI's writes) across rebuilds, the same way it preserves `adjustment` today.
|
||||||
|
|
||||||
|
## Code touchpoints
|
||||||
|
|
||||||
|
- **models.py** — `VideoSource` sheds `cutout`/`layer`/`end_on`/`take`/`pause_narration`
|
||||||
|
(or they become defaults-only); `VideoEvent` already carries `cutout`/`layer`/`end_on` —
|
||||||
|
extend to `take`/`pause_narration` sourced from the event, not the handle.
|
||||||
|
- **parser.py `parse_manuscript`** — extend the marker grammar to accept
|
||||||
|
`prefix:handle, key=value, …`; update the malformed-marker detector (which today flags
|
||||||
|
spaces/commas inside `[]`).
|
||||||
|
- **transformer.py `_extract_video_events`** — resolve `cutout/layer/end_on/take/
|
||||||
|
pause_narration` from (prefix ∪ inline overrides), not from `video_source`.
|
||||||
|
- **scaffold.py** — event schema: `handle` + presentation fields; `merge_events` preserves
|
||||||
|
GUI edits; migration for existing events.json.
|
||||||
|
- **cli.py** — retire `_project_markers_to_videos` and `_writeback_video_metadata` (they
|
||||||
|
project/writeback per-handle presentation) in favor of seeding event fields.
|
||||||
|
- **validator.py** — **remove** the key-reuse collision hard-error (reuse is legal now).
|
||||||
|
- **renderer.py** — read presentation from the event (mostly already does via `VideoEvent`).
|
||||||
|
|
||||||
|
## Migration
|
||||||
|
|
||||||
|
Existing projects (video0–video6, …) have presentation on the handle and prefixed `id`s in
|
||||||
|
events.json. A one-shot migration, run on build:
|
||||||
|
|
||||||
|
1. For each video event, split the prefixed `id` into `handle` + implied `cutout`/`layer`.
|
||||||
|
2. Fill `end_on`/`take`/`pause_narration` from the handle's current videos.json values
|
||||||
|
(preserving today's behavior — including handles that explicitly set `next_slide`).
|
||||||
|
3. Strip the moved fields from videos.json handles (leave the globals).
|
||||||
|
4. Idempotent: a second run is a no-op once events carry `handle`.
|
||||||
|
|
||||||
|
## Staging
|
||||||
|
|
||||||
|
- **Stage A** — schema split + per-event resolution from the shorthand prefix, migration,
|
||||||
|
remove the collision guard. Shorthand-only authoring keeps working; the 5 video5
|
||||||
|
collisions resolve. (This is the part that fixes the bug.)
|
||||||
|
- **Stage B** — the inline `key=value` overload grammar + malformed-marker updates.
|
||||||
|
|
||||||
|
Keep the validator collision hard-error in place **until Stage A lands** — removing it
|
||||||
|
earlier would let the hidden-overlay bug back in on video5.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- `always_visible`, `use_audio_channels`: confirmed as inline-overridable globals — do any
|
||||||
|
need to become fully per-event?
|
||||||
|
- Does the GUI want events fully flattened (every field present) or sparse (only overrides,
|
||||||
|
inherit the rest)? Affects whether the build writes defaults explicitly.
|
||||||
@@ -0,0 +1,141 @@
|
|||||||
|
# Chunked Rendering v2 — Design Spec
|
||||||
|
|
||||||
|
Status: **implemented on branch `chunking-v2`, pending render-seam validation on the rig.**
|
||||||
|
Plan-level logic is covered by `tests/test_chunking_v2.py` (all green). What remains
|
||||||
|
is confirming the ffmpeg concat seam is frame/phase-accurate on a real render — see
|
||||||
|
"Concat-seam correctness" below.
|
||||||
|
|
||||||
|
## Why chunking exists
|
||||||
|
|
||||||
|
The render is a single `ffmpeg -filter_complex` pass that opens every `-i` input up
|
||||||
|
front. On memory-constrained machines (an 8 GB VM, or a WSL2 rig whose VM RAM is a
|
||||||
|
slice of the host) the aggregate decoder + filter buffers OOM-kill the process — or,
|
||||||
|
on WSL2, the whole VM. `render_chunk_slides` (in `~/.gnommo.conf` `[performance]`, or
|
||||||
|
`--chunk-slides N`) splits the timeline into groups of N slides, renders each as a
|
||||||
|
partial render, and concatenates the chunks with `-c copy`.
|
||||||
|
|
||||||
|
Each chunk is a partial render built via `build_render_plan(slide_range=(start,end))`,
|
||||||
|
which passes `time_range=(time_offset, render_end_time)` to the event extractors. See
|
||||||
|
`partial-rendering-spec.md` for the partial-render mechanics chunking reuses.
|
||||||
|
|
||||||
|
## The v1 defect this fixes
|
||||||
|
|
||||||
|
`_extract_video_events` / `_extract_audio_events` keep an event **only if its start
|
||||||
|
falls inside the window**:
|
||||||
|
|
||||||
|
```python
|
||||||
|
if start_time < range_start or start_time >= range_end:
|
||||||
|
continue
|
||||||
|
```
|
||||||
|
|
||||||
|
So a clip that **began in an earlier chunk and is still playing across the boundary**
|
||||||
|
is dropped from every later chunk it overlaps. The full (non-chunked) render is
|
||||||
|
correct; the chunked render silently diverges.
|
||||||
|
|
||||||
|
Who is affected:
|
||||||
|
|
||||||
|
- **Overlay videos that span slides:** `end_on: next_video`, `loop`, long `take`/`end`
|
||||||
|
(multi-slide backgrounds / persistent picture-in-picture).
|
||||||
|
- **Looping background audio/music:** started once early, meant to underlie the whole
|
||||||
|
video — dropped from every chunk after the first. Highest blast radius.
|
||||||
|
- **Not affected:** `end_on: next_slide` clips (they end exactly at a slide marker =
|
||||||
|
a chunk seam), per-slide content, and the full-screen `plan.background` (a separate
|
||||||
|
always-included input).
|
||||||
|
|
||||||
|
v1 status: `cli._chunk_boundary_span_warnings(plan, groups)` detects spanning
|
||||||
|
video/outro/audio events and prints a loud warning before rendering, so the divergence
|
||||||
|
is never silent. It does not yet correct the output.
|
||||||
|
|
||||||
|
## v2 algorithm
|
||||||
|
|
||||||
|
### 1. Overlap inclusion (not start-inside)
|
||||||
|
|
||||||
|
Replace the start-inside test with an overlap test in both extractors:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# keep the event if it overlaps [range_start, range_end)
|
||||||
|
if end_time <= range_start or start_time >= range_end:
|
||||||
|
continue
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Per-event seek for clips that began earlier
|
||||||
|
|
||||||
|
A clip included by overlap whose `start_time < range_start` is already mid-playback at
|
||||||
|
the chunk boundary. It must resume at the correct frame, not restart. Compute how far
|
||||||
|
into the clip the window begins and carry it as a **per-event seek**:
|
||||||
|
|
||||||
|
```
|
||||||
|
into = range_start - start_time # seconds of the clip already elapsed
|
||||||
|
base = video_source.skip or 0.0
|
||||||
|
# non-looping clip:
|
||||||
|
skip_override = base + into
|
||||||
|
# looping clip (end_on: loop / next_video that wraps):
|
||||||
|
period = (video_source.duration or 0) - base # one loop's playable length
|
||||||
|
skip_override = base + (into % period) if period > 0 else base
|
||||||
|
```
|
||||||
|
|
||||||
|
Then clamp the event to the window and let the offset pass zero it:
|
||||||
|
|
||||||
|
```
|
||||||
|
start_time = max(start_time, range_start) # -> 0 after time_offset subtraction
|
||||||
|
end_time = min(end_time, range_end)
|
||||||
|
```
|
||||||
|
|
||||||
|
The seam for the *seek* is `VideoEvent.skip_override` (already added, inert until v2):
|
||||||
|
the renderer prefers it over `video_source.skip` when set. `-ss {skip}` is applied as
|
||||||
|
an input option, so ffmpeg decodes to that point — frame-accurate for the codecs in
|
||||||
|
use.
|
||||||
|
|
||||||
|
### 3. Audio equivalent
|
||||||
|
|
||||||
|
`AudioEvent` has no `skip_override` / `end_time` yet. v2 adds both (or derives end from
|
||||||
|
`audio_def.duration` / `loop` → `total_duration`) and applies the same overlap + seek.
|
||||||
|
For looping music the seek is the loop-phase modulo above; `ignore_pauses` and
|
||||||
|
`overlap` (crossfade) interactions must be re-checked at the seam.
|
||||||
|
|
||||||
|
### 4. Outro events
|
||||||
|
|
||||||
|
`OutroEvent` also spans (start_time/end_time). Extend the same treatment; outros
|
||||||
|
normally live in the final chunk so this is lower priority but should be covered for
|
||||||
|
completeness.
|
||||||
|
|
||||||
|
## Concat-seam correctness (the risk to validate)
|
||||||
|
|
||||||
|
Chunks are joined with `-c copy`, so the two sides of a seam must be frame-aligned:
|
||||||
|
chunk *k* ends showing the clip at position `P`, chunk *k+1* must resume at exactly `P`
|
||||||
|
via `skip_override`. Validation plan:
|
||||||
|
|
||||||
|
1. Pick a project with a known multi-slide overlay video **and** looping music
|
||||||
|
(or synthesize one).
|
||||||
|
2. Render it full (reference) and chunked (small `chunk_slides`, so a seam falls mid
|
||||||
|
clip).
|
||||||
|
3. Compare: identical duration; frame diff at ±3 frames around each seam below a
|
||||||
|
threshold; audio cross-correlation shows no gap/jump; the overlay is present in
|
||||||
|
every chunk it overlaps (the v1 bug is gone).
|
||||||
|
4. Assert `_chunk_boundary_span_warnings` returns empty for the fixed path.
|
||||||
|
|
||||||
|
## Work items
|
||||||
|
|
||||||
|
- [x] `_extract_video_events`: overlap test + `skip_override` (loop-aware).
|
||||||
|
- [x] `_extract_audio_events`: overlap + `src_offset` seek (loop phase / linear).
|
||||||
|
- [x] `AudioEvent`: `src_offset` field; renderer audio paths (loop-with-pauses,
|
||||||
|
standard loop, one-shot) honor it.
|
||||||
|
- [x] `VideoEvent.skip_override`: renderer video input `-ss` honors it; the clip's
|
||||||
|
embedded audio (`tvaud`) is seeked automatically by the same input seek.
|
||||||
|
- [x] `_chunk_boundary_span_warnings`: downgraded from the "will be dropped" v1
|
||||||
|
warning to an informational note (logged; terminal only under `--verbose`).
|
||||||
|
- [x] Plan-level tests: `tests/test_chunking_v2.py`.
|
||||||
|
- [ ] **Render-seam validation on the rig** (chunked-vs-full frame/audio diff) — the
|
||||||
|
remaining gate before making v2 the trusted default.
|
||||||
|
- [ ] Crossfade-loop audio (`_build_crossfade_loop_filter`) does not yet apply
|
||||||
|
`src_offset` — a crossfaded looping bed restarts phase at the seam. Standard
|
||||||
|
(non-crossfade) loops and one-shots are handled. Low priority.
|
||||||
|
- [ ] `OutroEvent`: not needed — outros are extracted for the last chunk only
|
||||||
|
(`config.outro if is_last_chunk`), so they never split across a seam.
|
||||||
|
|
||||||
|
## Not affected (verified)
|
||||||
|
|
||||||
|
- Slides: `_extract_slide_events` already used overlap+clamp — no change.
|
||||||
|
- Full-screen `plan.background`: a separate always-included input.
|
||||||
|
- Full (non-chunked) render: `time_range=None` path leaves `skip_override`/
|
||||||
|
`src_offset` at their defaults, so output is byte-identical to before.
|
||||||
@@ -1,14 +1,14 @@
|
|||||||
{
|
{
|
||||||
"talking_head_S1": {
|
"talking_head_S1": {
|
||||||
"source_file": "talking_head_S1.mov",
|
"source_file": "talking_head_S1.mov",
|
||||||
"output_file": "talking_head_S1_processed.mov",
|
"output_file": "processed/talking_head_S1_processed.mov",
|
||||||
"cutout": "talkinghead",
|
"cutout": "talkinghead",
|
||||||
"always_visible": true,
|
"always_visible": true,
|
||||||
"filter": "talkinghead"
|
"filter": "talkinghead"
|
||||||
},
|
},
|
||||||
"talking_head_S3": {
|
"talking_head_S3": {
|
||||||
"source_file": "talking_head_S3.mov",
|
"source_file": "talking_head_S3.mov",
|
||||||
"output_file": "talking_head_S3_processed.mov",
|
"output_file": "processed/talking_head_S3_processed.mov",
|
||||||
"cutout": "talkinghead",
|
"cutout": "talkinghead",
|
||||||
"always_visible": true,
|
"always_visible": true,
|
||||||
"filter": "talkinghead"
|
"filter": "talkinghead"
|
||||||
|
|||||||
@@ -0,0 +1,497 @@
|
|||||||
|
[
|
||||||
|
{
|
||||||
|
"word": "This",
|
||||||
|
"start": 10.74,
|
||||||
|
"end": 11.44
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "is",
|
||||||
|
"start": 11.44,
|
||||||
|
"end": 11.64
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 11.64,
|
||||||
|
"end": 11.82
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "first",
|
||||||
|
"start": 11.82,
|
||||||
|
"end": 12.04
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "slide.",
|
||||||
|
"start": 12.04,
|
||||||
|
"end": 12.44
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "It",
|
||||||
|
"start": 12.92,
|
||||||
|
"end": 13.34
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "appears",
|
||||||
|
"start": 13.34,
|
||||||
|
"end": 13.7
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "immediate.",
|
||||||
|
"start": 13.7,
|
||||||
|
"end": 14.18
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "However,",
|
||||||
|
"start": 15.36,
|
||||||
|
"end": 16.06
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "this",
|
||||||
|
"start": 16.38,
|
||||||
|
"end": 16.48
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "is",
|
||||||
|
"start": 16.48,
|
||||||
|
"end": 16.62
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 16.62,
|
||||||
|
"end": 16.8
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "second",
|
||||||
|
"start": 16.8,
|
||||||
|
"end": 17.08
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "slide.",
|
||||||
|
"start": 17.08,
|
||||||
|
"end": 17.42
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "It",
|
||||||
|
"start": 17.78,
|
||||||
|
"end": 18.02
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "should",
|
||||||
|
"start": 18.02,
|
||||||
|
"end": 18.24
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "appear",
|
||||||
|
"start": 18.24,
|
||||||
|
"end": 18.56
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "one",
|
||||||
|
"start": 18.56,
|
||||||
|
"end": 19.02
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "second",
|
||||||
|
"start": 19.02,
|
||||||
|
"end": 19.5
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "prior",
|
||||||
|
"start": 19.5,
|
||||||
|
"end": 19.92
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "to",
|
||||||
|
"start": 19.92,
|
||||||
|
"end": 20.16
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 20.16,
|
||||||
|
"end": 20.26
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "word",
|
||||||
|
"start": 20.26,
|
||||||
|
"end": 20.54
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "when",
|
||||||
|
"start": 20.54,
|
||||||
|
"end": 21.24
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "I",
|
||||||
|
"start": 21.24,
|
||||||
|
"end": 21.32
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "say",
|
||||||
|
"start": 21.32,
|
||||||
|
"end": 21.5
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "whoever",
|
||||||
|
"start": 21.5,
|
||||||
|
"end": 21.86
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "first",
|
||||||
|
"start": 21.86,
|
||||||
|
"end": 22.44
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "time.",
|
||||||
|
"start": 22.44,
|
||||||
|
"end": 22.7
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "This",
|
||||||
|
"start": 24.3,
|
||||||
|
"end": 25.0
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "is",
|
||||||
|
"start": 25.0,
|
||||||
|
"end": 25.14
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "me",
|
||||||
|
"start": 25.14,
|
||||||
|
"end": 25.38
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "taking,",
|
||||||
|
"start": 25.38,
|
||||||
|
"end": 25.78
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "talking",
|
||||||
|
"start": 26.14,
|
||||||
|
"end": 27.18
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "alongside",
|
||||||
|
"start": 27.18,
|
||||||
|
"end": 27.66
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "a",
|
||||||
|
"start": 27.66,
|
||||||
|
"end": 27.92
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "video.",
|
||||||
|
"start": 27.92,
|
||||||
|
"end": 28.16
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "The",
|
||||||
|
"start": 28.68,
|
||||||
|
"end": 28.96
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "video",
|
||||||
|
"start": 28.96,
|
||||||
|
"end": 29.2
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "is",
|
||||||
|
"start": 29.2,
|
||||||
|
"end": 29.4
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "constrained",
|
||||||
|
"start": 29.4,
|
||||||
|
"end": 29.82
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "within",
|
||||||
|
"start": 29.82,
|
||||||
|
"end": 30.18
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 30.18,
|
||||||
|
"end": 30.36
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "red",
|
||||||
|
"start": 30.36,
|
||||||
|
"end": 30.52
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "square.",
|
||||||
|
"start": 30.52,
|
||||||
|
"end": 30.94
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "Notice",
|
||||||
|
"start": 31.3,
|
||||||
|
"end": 31.48
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "how",
|
||||||
|
"start": 31.48,
|
||||||
|
"end": 31.78
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 31.78,
|
||||||
|
"end": 31.96
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "video",
|
||||||
|
"start": 31.96,
|
||||||
|
"end": 32.16
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "stops",
|
||||||
|
"start": 32.16,
|
||||||
|
"end": 32.48
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "immediately",
|
||||||
|
"start": 32.48,
|
||||||
|
"end": 32.98
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "when",
|
||||||
|
"start": 32.98,
|
||||||
|
"end": 33.4
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "we",
|
||||||
|
"start": 33.4,
|
||||||
|
"end": 33.58
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "make",
|
||||||
|
"start": 33.58,
|
||||||
|
"end": 33.76
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 33.76,
|
||||||
|
"end": 34.0
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "transition",
|
||||||
|
"start": 34.0,
|
||||||
|
"end": 34.42
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "to",
|
||||||
|
"start": 34.42,
|
||||||
|
"end": 34.72
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 34.72,
|
||||||
|
"end": 34.84
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "next",
|
||||||
|
"start": 34.84,
|
||||||
|
"end": 35.06
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "slide.",
|
||||||
|
"start": 35.06,
|
||||||
|
"end": 35.48
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "I",
|
||||||
|
"start": 37.2,
|
||||||
|
"end": 37.76
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "will",
|
||||||
|
"start": 37.76,
|
||||||
|
"end": 37.82
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "continue",
|
||||||
|
"start": 37.82,
|
||||||
|
"end": 38.12
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "to",
|
||||||
|
"start": 38.12,
|
||||||
|
"end": 38.34
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "talk",
|
||||||
|
"start": 38.34,
|
||||||
|
"end": 38.58
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "without",
|
||||||
|
"start": 38.58,
|
||||||
|
"end": 38.92
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "pause,",
|
||||||
|
"start": 38.92,
|
||||||
|
"end": 39.26
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "but",
|
||||||
|
"start": 39.5,
|
||||||
|
"end": 39.6
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "in",
|
||||||
|
"start": 39.6,
|
||||||
|
"end": 39.72
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 39.72,
|
||||||
|
"end": 39.8
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "finished",
|
||||||
|
"start": 39.8,
|
||||||
|
"end": 40.0
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "recording",
|
||||||
|
"start": 40.0,
|
||||||
|
"end": 40.48
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "there",
|
||||||
|
"start": 40.48,
|
||||||
|
"end": 41.22
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "will",
|
||||||
|
"start": 41.22,
|
||||||
|
"end": 41.38
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "be",
|
||||||
|
"start": 41.38,
|
||||||
|
"end": 41.58
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "a",
|
||||||
|
"start": 41.58,
|
||||||
|
"end": 41.68
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "pause",
|
||||||
|
"start": 41.68,
|
||||||
|
"end": 41.96
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "before",
|
||||||
|
"start": 41.96,
|
||||||
|
"end": 42.32
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 42.32,
|
||||||
|
"end": 42.52
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "narration",
|
||||||
|
"start": 42.52,
|
||||||
|
"end": 43.06
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "continues.",
|
||||||
|
"start": 43.06,
|
||||||
|
"end": 43.66
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "Now",
|
||||||
|
"start": 44.44,
|
||||||
|
"end": 44.56
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "a",
|
||||||
|
"start": 44.56,
|
||||||
|
"end": 44.7
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "video",
|
||||||
|
"start": 44.7,
|
||||||
|
"end": 44.94
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "will",
|
||||||
|
"start": 44.94,
|
||||||
|
"end": 45.12
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "play",
|
||||||
|
"start": 45.12,
|
||||||
|
"end": 45.4
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "that",
|
||||||
|
"start": 45.4,
|
||||||
|
"end": 45.8
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "pauses",
|
||||||
|
"start": 45.8,
|
||||||
|
"end": 46.52
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 46.52,
|
||||||
|
"end": 46.8
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "narration.",
|
||||||
|
"start": 46.8,
|
||||||
|
"end": 47.22
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "Notice",
|
||||||
|
"start": 48.66,
|
||||||
|
"end": 49.22
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "how",
|
||||||
|
"start": 49.22,
|
||||||
|
"end": 49.44
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "my",
|
||||||
|
"start": 49.44,
|
||||||
|
"end": 49.6
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "voice",
|
||||||
|
"start": 49.6,
|
||||||
|
"end": 49.84
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "continues",
|
||||||
|
"start": 49.84,
|
||||||
|
"end": 50.38
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "after",
|
||||||
|
"start": 50.38,
|
||||||
|
"end": 50.88
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 50.88,
|
||||||
|
"end": 51.04
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "video",
|
||||||
|
"start": 51.04,
|
||||||
|
"end": 51.28
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "finished.",
|
||||||
|
"start": 51.28,
|
||||||
|
"end": 51.8
|
||||||
|
}
|
||||||
|
]
|
||||||
@@ -0,0 +1,497 @@
|
|||||||
|
[
|
||||||
|
{
|
||||||
|
"word": "This",
|
||||||
|
"start": 10.632,
|
||||||
|
"end": 11.312
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "is",
|
||||||
|
"start": 11.312,
|
||||||
|
"end": 11.512
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 11.512,
|
||||||
|
"end": 11.692
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "first",
|
||||||
|
"start": 11.692,
|
||||||
|
"end": 11.912
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "slide.",
|
||||||
|
"start": 11.912,
|
||||||
|
"end": 12.312
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "It",
|
||||||
|
"start": 12.852,
|
||||||
|
"end": 13.192
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "appears",
|
||||||
|
"start": 13.192,
|
||||||
|
"end": 13.552
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "immediate.",
|
||||||
|
"start": 13.552,
|
||||||
|
"end": 14.032
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "However,",
|
||||||
|
"start": 15.452,
|
||||||
|
"end": 15.932
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "this",
|
||||||
|
"start": 16.272,
|
||||||
|
"end": 16.352
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "is",
|
||||||
|
"start": 16.352,
|
||||||
|
"end": 16.492
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 16.492,
|
||||||
|
"end": 16.652
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "second",
|
||||||
|
"start": 16.652,
|
||||||
|
"end": 16.952
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "slide.",
|
||||||
|
"start": 16.952,
|
||||||
|
"end": 17.292
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "It",
|
||||||
|
"start": 17.572,
|
||||||
|
"end": 17.872
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "should",
|
||||||
|
"start": 17.872,
|
||||||
|
"end": 18.112
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "appear",
|
||||||
|
"start": 18.112,
|
||||||
|
"end": 18.432
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "one",
|
||||||
|
"start": 18.432,
|
||||||
|
"end": 18.892
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "second",
|
||||||
|
"start": 18.892,
|
||||||
|
"end": 19.372
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "prior",
|
||||||
|
"start": 19.372,
|
||||||
|
"end": 19.792
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "to",
|
||||||
|
"start": 19.792,
|
||||||
|
"end": 20.032
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 20.032,
|
||||||
|
"end": 20.152
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "word",
|
||||||
|
"start": 20.152,
|
||||||
|
"end": 20.412
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "when",
|
||||||
|
"start": 20.412,
|
||||||
|
"end": 21.112
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "I",
|
||||||
|
"start": 21.112,
|
||||||
|
"end": 21.192
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "say",
|
||||||
|
"start": 21.192,
|
||||||
|
"end": 21.352
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "whoever",
|
||||||
|
"start": 21.352,
|
||||||
|
"end": 21.732
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "first",
|
||||||
|
"start": 21.732,
|
||||||
|
"end": 22.312
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "time.",
|
||||||
|
"start": 22.312,
|
||||||
|
"end": 22.592
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "This",
|
||||||
|
"start": 24.532,
|
||||||
|
"end": 24.872
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "is",
|
||||||
|
"start": 24.872,
|
||||||
|
"end": 25.032
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "me",
|
||||||
|
"start": 25.032,
|
||||||
|
"end": 25.252
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "taking,",
|
||||||
|
"start": 25.252,
|
||||||
|
"end": 25.652
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "talking",
|
||||||
|
"start": 26.092,
|
||||||
|
"end": 27.052
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "alongside",
|
||||||
|
"start": 27.052,
|
||||||
|
"end": 27.532
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "a",
|
||||||
|
"start": 27.532,
|
||||||
|
"end": 27.792
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "video.",
|
||||||
|
"start": 27.792,
|
||||||
|
"end": 28.052
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "The",
|
||||||
|
"start": 28.652,
|
||||||
|
"end": 28.832
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "video",
|
||||||
|
"start": 28.832,
|
||||||
|
"end": 29.092
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "is",
|
||||||
|
"start": 29.092,
|
||||||
|
"end": 29.272
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "constrained",
|
||||||
|
"start": 29.272,
|
||||||
|
"end": 29.712
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "within",
|
||||||
|
"start": 29.712,
|
||||||
|
"end": 30.052
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 30.052,
|
||||||
|
"end": 30.232
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "red",
|
||||||
|
"start": 30.232,
|
||||||
|
"end": 30.392
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "square.",
|
||||||
|
"start": 30.392,
|
||||||
|
"end": 30.792
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "Notice",
|
||||||
|
"start": 30.792,
|
||||||
|
"end": 31.352
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "how",
|
||||||
|
"start": 31.352,
|
||||||
|
"end": 31.652
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 31.652,
|
||||||
|
"end": 31.832
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "video",
|
||||||
|
"start": 31.832,
|
||||||
|
"end": 32.032
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "stops",
|
||||||
|
"start": 32.032,
|
||||||
|
"end": 32.372
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "immediately",
|
||||||
|
"start": 32.372,
|
||||||
|
"end": 32.852
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "when",
|
||||||
|
"start": 32.852,
|
||||||
|
"end": 33.272
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "we",
|
||||||
|
"start": 33.272,
|
||||||
|
"end": 33.452
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "make",
|
||||||
|
"start": 33.452,
|
||||||
|
"end": 33.632
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 33.632,
|
||||||
|
"end": 33.872
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "transition",
|
||||||
|
"start": 33.872,
|
||||||
|
"end": 34.292
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "to",
|
||||||
|
"start": 34.292,
|
||||||
|
"end": 34.592
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 34.592,
|
||||||
|
"end": 34.712
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "next",
|
||||||
|
"start": 34.712,
|
||||||
|
"end": 34.932
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "slide.",
|
||||||
|
"start": 34.932,
|
||||||
|
"end": 35.392
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "I",
|
||||||
|
"start": 37.112,
|
||||||
|
"end": 37.632
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "will",
|
||||||
|
"start": 37.632,
|
||||||
|
"end": 37.692
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "continue",
|
||||||
|
"start": 37.692,
|
||||||
|
"end": 37.992
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "to",
|
||||||
|
"start": 37.992,
|
||||||
|
"end": 38.212
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "talk",
|
||||||
|
"start": 38.212,
|
||||||
|
"end": 38.452
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "without",
|
||||||
|
"start": 38.452,
|
||||||
|
"end": 38.792
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "pause,",
|
||||||
|
"start": 38.792,
|
||||||
|
"end": 39.132
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "but",
|
||||||
|
"start": 39.372,
|
||||||
|
"end": 39.472
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "in",
|
||||||
|
"start": 39.472,
|
||||||
|
"end": 39.592
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 39.592,
|
||||||
|
"end": 39.652
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "finished",
|
||||||
|
"start": 39.652,
|
||||||
|
"end": 39.872
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "recording",
|
||||||
|
"start": 39.872,
|
||||||
|
"end": 40.352
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "there",
|
||||||
|
"start": 40.352,
|
||||||
|
"end": 41.092
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "will",
|
||||||
|
"start": 41.092,
|
||||||
|
"end": 41.252
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "be",
|
||||||
|
"start": 41.252,
|
||||||
|
"end": 41.452
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "a",
|
||||||
|
"start": 41.452,
|
||||||
|
"end": 41.552
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "pause",
|
||||||
|
"start": 41.552,
|
||||||
|
"end": 41.812
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "before",
|
||||||
|
"start": 41.812,
|
||||||
|
"end": 42.192
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 42.192,
|
||||||
|
"end": 42.392
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "narration",
|
||||||
|
"start": 42.392,
|
||||||
|
"end": 42.932
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "continues.",
|
||||||
|
"start": 42.932,
|
||||||
|
"end": 43.552
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "Now",
|
||||||
|
"start": 44.232,
|
||||||
|
"end": 44.432
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "a",
|
||||||
|
"start": 44.432,
|
||||||
|
"end": 44.572
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "video",
|
||||||
|
"start": 44.572,
|
||||||
|
"end": 44.812
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "will",
|
||||||
|
"start": 44.812,
|
||||||
|
"end": 44.972
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "play",
|
||||||
|
"start": 44.972,
|
||||||
|
"end": 45.272
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "that",
|
||||||
|
"start": 45.272,
|
||||||
|
"end": 45.672
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "pauses",
|
||||||
|
"start": 45.672,
|
||||||
|
"end": 46.412
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 46.412,
|
||||||
|
"end": 46.672
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "narration.",
|
||||||
|
"start": 46.672,
|
||||||
|
"end": 47.092
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "Notice",
|
||||||
|
"start": 48.352,
|
||||||
|
"end": 49.092
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "how",
|
||||||
|
"start": 49.092,
|
||||||
|
"end": 49.312
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "my",
|
||||||
|
"start": 49.312,
|
||||||
|
"end": 49.492
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "voice",
|
||||||
|
"start": 49.492,
|
||||||
|
"end": 49.752
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "continues",
|
||||||
|
"start": 49.752,
|
||||||
|
"end": 50.272
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "after",
|
||||||
|
"start": 50.272,
|
||||||
|
"end": 50.752
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "the",
|
||||||
|
"start": 50.752,
|
||||||
|
"end": 50.932
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "video",
|
||||||
|
"start": 50.932,
|
||||||
|
"end": 51.152
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"word": "finished.",
|
||||||
|
"start": 51.152,
|
||||||
|
"end": 51.652
|
||||||
|
}
|
||||||
|
]
|
||||||
@@ -25,15 +25,14 @@
|
|||||||
"source_file": "Logo.mov",
|
"source_file": "Logo.mov",
|
||||||
"is_shared": true,
|
"is_shared": true,
|
||||||
"cutout": "fullscreen",
|
"cutout": "fullscreen",
|
||||||
"pause_narration": 17,
|
"pause_narration": 14,
|
||||||
"take": 25,
|
"take": 25,
|
||||||
"skip": 0
|
"skip": 0
|
||||||
},
|
},
|
||||||
"Zoomin_MontageZoom": {
|
"zoomin_montagezoom": {
|
||||||
"description": "Montage zoom",
|
"description": "Montage zoom",
|
||||||
"source_file": "MontageZoom.mp4",
|
"source_file": "MontageZoom.mp4",
|
||||||
"output_file": "MontageZoom.mp4",
|
"output_file": "MontageZoom.mp4",
|
||||||
"pause_narration": 5,
|
|
||||||
"cutout": "square",
|
"cutout": "square",
|
||||||
"is_shared": true,
|
"is_shared": true,
|
||||||
"filter": []
|
"filter": []
|
||||||
|
|||||||
@@ -9,7 +9,7 @@
|
|||||||
"keynote_file": "media/example.key",
|
"keynote_file": "media/example.key",
|
||||||
"transcript": "media/videos/talking_head.transcript.json",
|
"transcript": "media/videos/talking_head.transcript.json",
|
||||||
"narration": "media/narration/narration.json",
|
"narration": "media/narration/narration.json",
|
||||||
"background": "shared_assets/solarpunk.png",
|
"background": "blackbackground",
|
||||||
"videos": "media/videos/videos.json",
|
"videos": "media/videos/videos.json",
|
||||||
"slides": "media/slides/Example/slides.json",
|
"slides": "media/slides/Example/slides.json",
|
||||||
"audio": "media/audio/audio.json",
|
"audio": "media/audio/audio.json",
|
||||||
@@ -18,6 +18,7 @@
|
|||||||
"talkinghead": [
|
"talkinghead": [
|
||||||
{
|
{
|
||||||
"type": "audio_normalize",
|
"type": "audio_normalize",
|
||||||
|
"enable":false,
|
||||||
"eq_bands": [
|
"eq_bands": [
|
||||||
{"freq": 47, "gain": -15, "type": "lowshelf"},
|
{"freq": 47, "gain": -15, "type": "lowshelf"},
|
||||||
{"freq": 107, "gain": -1.3, "q": 1.2},
|
{"freq": 107, "gain": -1.3, "q": 1.2},
|
||||||
@@ -47,10 +48,12 @@
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
"type": "color_grade",
|
"type": "color_grade",
|
||||||
"saturation": 1.15,
|
"saturation": 1.02,
|
||||||
"contrast": 1.05,
|
"contrast": 1.05,
|
||||||
"bm": -0.10,
|
"brightness": 0.04,
|
||||||
"rm": 0.04
|
"bm": 0.0,
|
||||||
|
"gm": 0.02,
|
||||||
|
"rm": -0.07
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"type": "gnommokey",
|
"type": "gnommokey",
|
||||||
@@ -76,7 +79,7 @@
|
|||||||
"talkinghead": {
|
"talkinghead": {
|
||||||
"x": "-10%",
|
"x": "-10%",
|
||||||
"y": "40%",
|
"y": "40%",
|
||||||
"height": "60%"
|
"height": "80%"
|
||||||
},
|
},
|
||||||
"square": {
|
"square": {
|
||||||
"x": "45%",
|
"x": "45%",
|
||||||
|
|||||||
@@ -0,0 +1,13 @@
|
|||||||
|
# Tasks: example
|
||||||
|
_Generated: 2026-07-15_
|
||||||
|
|
||||||
|
## Slide Alignment Issues (7)
|
||||||
|
Slide markers that could not be matched to the spoken narration (likely adlibbed).
|
||||||
|
|
||||||
|
- [ ] `S6` — _"(end of: voice continues after the video finished)"_
|
||||||
|
- [ ] `S7` — _"This is the first slide. It appears immediately."_
|
||||||
|
- [ ] `S8` — _"However, this is the second slide. It should appea"_
|
||||||
|
- [ ] `S9` — _"This is me talking alongside a video. The video is"_
|
||||||
|
- [ ] `S10` — _"I will continue to talk without pause, but in the"_
|
||||||
|
- [ ] `S11` — _"Notice how my voice continues after the video fini"_
|
||||||
|
- [ ] `S12` — _"(repaired: voice continues after the video finished)"_
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
{
|
||||||
|
"talkinghead": [
|
||||||
|
{
|
||||||
|
"type": "audio_normalize",
|
||||||
|
"enabled": true,
|
||||||
|
"highpass": 85,
|
||||||
|
"eq_bands": [
|
||||||
|
{
|
||||||
|
"type": "peak",
|
||||||
|
"freq": 200,
|
||||||
|
"gain": -3.5,
|
||||||
|
"q": 1.2
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"compress": false,
|
||||||
|
"normalize": true,
|
||||||
|
"target_lufs": -14,
|
||||||
|
"target_lra": 11,
|
||||||
|
"target_tp": -1.5
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "gnommokey",
|
||||||
|
"screen_color": [
|
||||||
|
81,
|
||||||
|
137,
|
||||||
|
65
|
||||||
|
],
|
||||||
|
"screen_gain": 175,
|
||||||
|
"screen_balance": 58,
|
||||||
|
"despill_bias": [
|
||||||
|
235,
|
||||||
|
222,
|
||||||
|
210
|
||||||
|
],
|
||||||
|
"despill_strength": 7.0,
|
||||||
|
"spill_suppress": 1.3,
|
||||||
|
"yellow_protect": 0.9,
|
||||||
|
"edge_erode": 1.0,
|
||||||
|
"clip_black": 0,
|
||||||
|
"clip_white": 100
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "color_grade",
|
||||||
|
"saturation": 1.02,
|
||||||
|
"contrast": 1.05,
|
||||||
|
"brightness": 0.04,
|
||||||
|
"bm": 0.0,
|
||||||
|
"gm": 0.02,
|
||||||
|
"rm": -0.07
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "mask",
|
||||||
|
"left": 0.05,
|
||||||
|
"right": 0.1,
|
||||||
|
"top": 0.1,
|
||||||
|
"bottom": 0.0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
+104
-34
@@ -16,6 +16,7 @@ from pathlib import Path
|
|||||||
from typing import Optional, Tuple
|
from typing import Optional, Tuple
|
||||||
|
|
||||||
_cache_config: Optional[dict] = None
|
_cache_config: Optional[dict] = None
|
||||||
|
_assets_config: Optional[dict] = None
|
||||||
_perf_config: Optional[dict] = None
|
_perf_config: Optional[dict] = None
|
||||||
|
|
||||||
|
|
||||||
@@ -38,7 +39,9 @@ def get_ffmpeg_thread_count() -> int:
|
|||||||
cfg.read(config_path)
|
cfg.read(config_path)
|
||||||
if cfg.has_option("performance", "cpu_limit"):
|
if cfg.has_option("performance", "cpu_limit"):
|
||||||
try:
|
try:
|
||||||
_perf_config["cpu_limit"] = float(cfg.get("performance", "cpu_limit"))
|
_perf_config["cpu_limit"] = float(
|
||||||
|
cfg.get("performance", "cpu_limit")
|
||||||
|
)
|
||||||
except ValueError:
|
except ValueError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@@ -100,41 +103,107 @@ def load_cache_config() -> Optional[Path]:
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def load_assets_process_cache() -> Optional[Path]:
|
||||||
|
"""Return the process-cache path on the [assets] disk, or None if not configured.
|
||||||
|
|
||||||
|
Derived by replacing the last component of the [assets] path with
|
||||||
|
that name + "cache". E.g.:
|
||||||
|
[assets] path = /Volumes/LaCie Jens/Projects/gnommo
|
||||||
|
→ process cache = /Volumes/LaCie Jens/Projects/gnommocache
|
||||||
|
|
||||||
|
This mirrors the GnommoDisk convention where the asset root is
|
||||||
|
/Volumes/GnommoDisk/gnommo and the process cache is /Volumes/GnommoDisk/gnommocache.
|
||||||
|
"""
|
||||||
|
assets_path = load_assets_config()
|
||||||
|
if assets_path is None:
|
||||||
|
return None
|
||||||
|
return assets_path.parent / (assets_path.name + "cache")
|
||||||
|
|
||||||
|
|
||||||
|
def load_assets_config() -> Optional[Path]:
|
||||||
|
"""Load gnommo.conf and return the [assets] path if configured.
|
||||||
|
|
||||||
|
The assets path is a second external fallback (e.g. a LaCie drive) with
|
||||||
|
the same directory layout as the gnommo project root. Resolution order is:
|
||||||
|
local → cache ([cache] path) → assets ([assets] path).
|
||||||
|
|
||||||
|
Example ~/.gnommo.conf:
|
||||||
|
[assets]
|
||||||
|
path = /Volumes/LaCie Jens/Projects/gnommo
|
||||||
|
"""
|
||||||
|
global _assets_config
|
||||||
|
if _assets_config is not None:
|
||||||
|
return _assets_config.get("path")
|
||||||
|
|
||||||
|
config_path = Path.home() / ".gnommo.conf"
|
||||||
|
if not config_path.exists():
|
||||||
|
_assets_config = {}
|
||||||
|
return None
|
||||||
|
|
||||||
|
config = configparser.ConfigParser()
|
||||||
|
config.read(config_path)
|
||||||
|
|
||||||
|
if config.has_option("assets", "path"):
|
||||||
|
assets_path = Path(config.get("assets", "path"))
|
||||||
|
_assets_config = {"path": assets_path}
|
||||||
|
return assets_path
|
||||||
|
|
||||||
|
_assets_config = {}
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_against_base(
|
||||||
|
local_path: Path, project_path: Path, base: Path
|
||||||
|
) -> Optional[Path]:
|
||||||
|
"""Try to find local_path mirrored under base.
|
||||||
|
|
||||||
|
Tries two mappings:
|
||||||
|
1. project-relative: base / project_name / relative_to_project
|
||||||
|
2. gnommo-root-relative: base / relative_to_project_parent (e.g. shared_assets/…)
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
relative = local_path.relative_to(project_path)
|
||||||
|
p = base / project_path.name / relative
|
||||||
|
if p.exists():
|
||||||
|
return p
|
||||||
|
except ValueError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
try:
|
||||||
|
relative = local_path.relative_to(project_path.parent)
|
||||||
|
p = base / relative
|
||||||
|
if p.exists():
|
||||||
|
return p
|
||||||
|
except ValueError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
def resolve_with_cache(
|
def resolve_with_cache(
|
||||||
local_path: Path,
|
local_path: Path,
|
||||||
project_path: Path,
|
project_path: Path,
|
||||||
) -> Tuple[Path, bool]:
|
) -> Tuple[Path, bool]:
|
||||||
"""
|
"""Resolve a file path with external-disk fallback (read-only).
|
||||||
Resolve a file path with cache fallback (read-only).
|
|
||||||
|
|
||||||
Checks the local path first. If not found and cache is configured,
|
Resolution order:
|
||||||
checks the cache directory which mirrors the project structure.
|
1. local_path (always checked first)
|
||||||
|
2. [cache] path — typically GnommoDisk
|
||||||
Args:
|
3. [assets] path — optional second drive (e.g. LaCie)
|
||||||
local_path: The expected local path to the file
|
|
||||||
project_path: The project root directory
|
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Tuple of (resolved_path, is_cached) where is_cached=True if
|
Tuple of (resolved_path, is_from_external) where is_from_external=True
|
||||||
the file was found in the external cache instead of locally.
|
when the file was found on an external drive rather than locally.
|
||||||
"""
|
"""
|
||||||
# Check local path first
|
|
||||||
if local_path.exists():
|
if local_path.exists():
|
||||||
return local_path, False
|
return local_path, False
|
||||||
|
|
||||||
# Check cache
|
for base in (load_cache_config(), load_assets_config()):
|
||||||
cache_base = load_cache_config()
|
if base is None:
|
||||||
if cache_base is None:
|
continue
|
||||||
return local_path, False # No cache configured
|
resolved = _resolve_against_base(local_path, project_path, base)
|
||||||
|
if resolved is not None:
|
||||||
# Build cache path: {cache_base}/{project_name}/{relative_path}
|
return resolved, True
|
||||||
try:
|
|
||||||
relative = local_path.relative_to(project_path)
|
|
||||||
cache_path = cache_base / project_path.name / relative
|
|
||||||
if cache_path.exists():
|
|
||||||
return cache_path, True
|
|
||||||
except ValueError:
|
|
||||||
pass # local_path is not relative to project_path
|
|
||||||
|
|
||||||
return local_path, False
|
return local_path, False
|
||||||
|
|
||||||
@@ -173,15 +242,16 @@ def load_server_config() -> Optional[dict]:
|
|||||||
|
|
||||||
|
|
||||||
def is_cache_configured() -> bool:
|
def is_cache_configured() -> bool:
|
||||||
"""Check if cache is configured (for status messages)."""
|
"""Check if any external fallback is configured."""
|
||||||
return load_cache_config() is not None
|
return load_cache_config() is not None or load_assets_config() is not None
|
||||||
|
|
||||||
|
|
||||||
def get_cache_info() -> Optional[str]:
|
def get_cache_info() -> Optional[str]:
|
||||||
"""Get a human-readable cache configuration string."""
|
"""Get a human-readable string of all configured external paths."""
|
||||||
cache_path = load_cache_config()
|
parts = []
|
||||||
if cache_path is None:
|
for label, path in (("cache", load_cache_config()), ("assets", load_assets_config())):
|
||||||
return None
|
if path is None:
|
||||||
if cache_path.exists():
|
continue
|
||||||
return f"{cache_path} (connected)"
|
status = "connected" if path.exists() else "not connected"
|
||||||
return f"{cache_path} (not connected)"
|
parts.append(f"{path} ({status})")
|
||||||
|
return "; ".join(parts) if parts else None
|
||||||
|
|||||||
+3510
-1225
File diff suppressed because it is too large
Load Diff
+31
-46
@@ -1,27 +1,27 @@
|
|||||||
"""Hand off a finished video to MinIO storage via gnommoeditor (prod) or gnommoweb (local).
|
"""Hand off a finished video to gnommoweb (the review app) — MinIO upload + version bump.
|
||||||
|
|
||||||
Works for any gnommo project type: parent videos and shorts alike.
|
Works for any gnommo project type: parent videos and shorts alike.
|
||||||
|
|
||||||
Usage:
|
Usage:
|
||||||
gnommo handoff -p video1
|
gnommo handoff -p video1 # → local gnommoweb
|
||||||
gnommo handoff -p short_pixelated_universe
|
gnommo handoff -p video1 --prod # → production gnommoweb (glitch.university)
|
||||||
gnommo handoff -p video1 --file /path/to/render.mp4
|
gnommo handoff -p video1 --file /path/to/render.mp4
|
||||||
|
|
||||||
Reads project.json for the 'output_video' field (path relative to the
|
Reads project.json for the 'output_video' field (path relative to the
|
||||||
project directory). Override with --file.
|
project directory). Override with --file.
|
||||||
|
|
||||||
On success (production):
|
On success (both local and --prod):
|
||||||
- Uploads the video to MinIO via POST /api/assets/upload on gnommoeditor
|
- Uploads the video via POST /api/projects/:id/handoff on gnommoweb, which
|
||||||
- Updates .gnommo_sync.prod.json with asset URL
|
stores it in MinIO and bumps the project's video_version (so it shows up on
|
||||||
|
the review page).
|
||||||
On success (local):
|
- Updates .gnommo_sync.json (local) / .gnommo_sync.prod.json (--prod) with the
|
||||||
- Uploads via POST /api/projects/:handle/handoff on gnommoweb
|
new video_version.
|
||||||
- Updates .gnommo_sync.json with new video_version
|
|
||||||
|
|
||||||
Configuration (from .env or environment):
|
Configuration (from .env or environment):
|
||||||
GNOMMOEDITOR_URL Base URL for production (e.g. https://editor.glitch.university)
|
|
||||||
GNOMMOWEB_URL Base URL for local dev (e.g. http://localhost:3001)
|
GNOMMOWEB_URL Base URL for local dev (e.g. http://localhost:3001)
|
||||||
GNOMMOWEB_API_KEY Bearer token (CONTENT_API_KEY from gnommoweb)
|
GNOMMOWEB_API_KEY Bearer token for local (CONTENT_API_KEY from gnommoweb)
|
||||||
|
GNOMMOWEB_PROD_URL Base URL for production (e.g. https://glitch.university)
|
||||||
|
GNOMMOWEB_PROD_API_KEY Bearer token for production (CONTENT_API_KEY)
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import json
|
import json
|
||||||
@@ -85,24 +85,26 @@ def cmd_handoff(
|
|||||||
) -> int:
|
) -> int:
|
||||||
_load_env_file()
|
_load_env_file()
|
||||||
|
|
||||||
|
# Handoff always targets gnommoweb (the review app). --prod selects the
|
||||||
|
# production instance; without it, the local dev server.
|
||||||
if prod:
|
if prod:
|
||||||
api_url = os.environ.get("GNOMMOEDITOR_URL", "").rstrip("/")
|
api_url = os.environ.get("GNOMMOWEB_PROD_URL", "").rstrip("/")
|
||||||
if not api_url:
|
api_key = os.environ.get("GNOMMOWEB_PROD_API_KEY", "")
|
||||||
print("Error: GNOMMOEDITOR_URL is not set.", file=sys.stderr)
|
url_var, key_var = "GNOMMOWEB_PROD_URL", "GNOMMOWEB_PROD_API_KEY"
|
||||||
return 1
|
|
||||||
else:
|
else:
|
||||||
api_url = os.environ.get("GNOMMOWEB_URL", "").rstrip("/")
|
api_url = os.environ.get("GNOMMOWEB_URL", "").rstrip("/")
|
||||||
api_key = os.environ.get("GNOMMOWEB_API_KEY", "")
|
api_key = os.environ.get("GNOMMOWEB_API_KEY", "")
|
||||||
|
url_var, key_var = "GNOMMOWEB_URL", "GNOMMOWEB_API_KEY"
|
||||||
if not api_url:
|
if not api_url:
|
||||||
print("Error: GNOMMOWEB_URL is not set.", file=sys.stderr)
|
print(f"Error: {url_var} is not set.", file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
if not api_key:
|
if not api_key:
|
||||||
print("Error: GNOMMOWEB_API_KEY is not set.", file=sys.stderr)
|
print(f"Error: {key_var} is not set.", file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
|
|
||||||
if verbose:
|
if verbose:
|
||||||
target = "production (gnommoeditor)" if prod else "local"
|
target = "production" if prod else "local"
|
||||||
print(f" → {target}: {api_url}")
|
print(f" → gnommoweb {target}: {api_url}")
|
||||||
|
|
||||||
project_file = project_path / "project.json"
|
project_file = project_path / "project.json"
|
||||||
if not project_file.exists():
|
if not project_file.exists():
|
||||||
@@ -145,21 +147,18 @@ def cmd_handoff(
|
|||||||
print(f" File: {video_path} ({file_size_mb:.1f} MB)")
|
print(f" File: {video_path} ({file_size_mb:.1f} MB)")
|
||||||
|
|
||||||
# ── Upload ─────────────────────────────────────────────────────────────────
|
# ── Upload ─────────────────────────────────────────────────────────────────
|
||||||
|
# gnommoweb: POST /api/projects/:id/handoff — uploads to MinIO and bumps the
|
||||||
|
# project's video_version so it appears on the review page.
|
||||||
|
extra_data = {}
|
||||||
|
course = project.get("course")
|
||||||
|
if course:
|
||||||
|
extra_data["course"] = course
|
||||||
try:
|
try:
|
||||||
if prod:
|
|
||||||
# gnommoeditor: POST /api/assets/upload — field name is 'file', no auth
|
|
||||||
with open(video_path, "rb") as vf:
|
|
||||||
r = requests.post(
|
|
||||||
f"{api_url}/api/assets/upload",
|
|
||||||
files={"file": (video_path.name, vf, _mime_type(video_path))},
|
|
||||||
timeout=None,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
# gnommoweb: POST /api/projects/:id/handoff
|
|
||||||
with open(video_path, "rb") as vf:
|
with open(video_path, "rb") as vf:
|
||||||
r = requests.post(
|
r = requests.post(
|
||||||
f"{api_url}/api/projects/{project_id}/handoff",
|
f"{api_url}/api/projects/{project_id}/handoff",
|
||||||
files={"video": (video_path.name, vf, _mime_type(video_path))},
|
files={"video": (video_path.name, vf, _mime_type(video_path))},
|
||||||
|
data=extra_data or None,
|
||||||
headers={"Authorization": f"Bearer {api_key}"},
|
headers={"Authorization": f"Bearer {api_key}"},
|
||||||
timeout=None,
|
timeout=None,
|
||||||
)
|
)
|
||||||
@@ -178,23 +177,9 @@ def cmd_handoff(
|
|||||||
result = r.json()
|
result = r.json()
|
||||||
|
|
||||||
# ── Write sync state ───────────────────────────────────────────────────────
|
# ── Write sync state ───────────────────────────────────────────────────────
|
||||||
|
# gnommoweb response: { video_version, video_url, asset: { updated_at } }
|
||||||
now_iso = datetime.now(tz=timezone.utc).isoformat(timespec="seconds")
|
now_iso = datetime.now(tz=timezone.utc).isoformat(timespec="seconds")
|
||||||
existing_sync = _read_sync(project_path, prod)
|
existing_sync = _read_sync(project_path, prod)
|
||||||
|
|
||||||
if prod:
|
|
||||||
# gnommoeditor response: { asset: { id, url, minio_object_key, ... } }
|
|
||||||
asset = result.get("asset", {})
|
|
||||||
asset_url = asset.get("url", "")
|
|
||||||
_write_sync(
|
|
||||||
project_path,
|
|
||||||
{**existing_sync, "last_handoff_at": now_iso, "asset_url": asset_url},
|
|
||||||
prod,
|
|
||||||
)
|
|
||||||
print(f"✓ {project_id} → uploaded [asset #{asset.get('id')}]")
|
|
||||||
if asset_url:
|
|
||||||
print(f" {asset_url}")
|
|
||||||
else:
|
|
||||||
# gnommoweb response: { video_version, video_url, asset: { updated_at } }
|
|
||||||
video_version = result.get("video_version", "?")
|
video_version = result.get("video_version", "?")
|
||||||
video_url = result.get("video_url", "")
|
video_url = result.get("video_url", "")
|
||||||
_write_sync(
|
_write_sync(
|
||||||
@@ -209,7 +194,7 @@ def cmd_handoff(
|
|||||||
},
|
},
|
||||||
prod,
|
prod,
|
||||||
)
|
)
|
||||||
print(f"✓ {project_id} → v{video_version} [processed]")
|
print(f"✓ {project_id} → v{video_version} [uploaded]")
|
||||||
if video_url:
|
if video_url:
|
||||||
print(f" {video_url}")
|
print(f" {video_url}")
|
||||||
|
|
||||||
|
|||||||
+66
-3
@@ -26,6 +26,15 @@ class CutoutDefinition:
|
|||||||
height_percent: float = 0.0 # percentage (0.0-1.0) if height is -1
|
height_percent: float = 0.0 # percentage (0.0-1.0) if height is -1
|
||||||
width_percent: float = 0.0 # percentage (0.0-1.0) if width is -1
|
width_percent: float = 0.0 # percentage (0.0-1.0) if width is -1
|
||||||
|
|
||||||
|
# New center-based model (opt-in via cx/cy in project.json). When cx_percent is
|
||||||
|
# set: cx/cy are the CENTER as fractions of frame width/height; width_percent and
|
||||||
|
# height_percent are fractions of min(W,H) (so equal values → a true square on any
|
||||||
|
# aspect ratio); margin_percent is a per-side inset, also a fraction of min(W,H).
|
||||||
|
cx_percent: Optional[float] = None
|
||||||
|
cy_percent: Optional[float] = None
|
||||||
|
margin_percent: float = 0.0
|
||||||
|
size_relative_to_min: bool = False
|
||||||
|
|
||||||
|
|
||||||
# Backwards compatibility alias
|
# Backwards compatibility alias
|
||||||
TalkingHeadConfig = CutoutDefinition
|
TalkingHeadConfig = CutoutDefinition
|
||||||
@@ -49,6 +58,7 @@ class ProjectConfig:
|
|||||||
slides_path: str = "slides.json" # path to slides.json relative to project
|
slides_path: str = "slides.json" # path to slides.json relative to project
|
||||||
videos_path: str = "videos.json" # path to videos.json relative to project
|
videos_path: str = "videos.json" # path to videos.json relative to project
|
||||||
audio_path: str = "audio.json" # path to audio.json relative to project
|
audio_path: str = "audio.json" # path to audio.json relative to project
|
||||||
|
transcript_path: Optional[str] = None # path to transcript.json relative to project (always saved locally)
|
||||||
audio_source: Optional[str] = None # defaults to talking head
|
audio_source: Optional[str] = None # defaults to talking head
|
||||||
main_video: Optional[
|
main_video: Optional[
|
||||||
Union[str, list]
|
Union[str, list]
|
||||||
@@ -131,10 +141,35 @@ class GnommoKeyConfig:
|
|||||||
# How aggressively to apply despill (0-1)
|
# How aggressively to apply despill (0-1)
|
||||||
despill_strength: float = 0.5
|
despill_strength: float = 0.5
|
||||||
|
|
||||||
|
# Interior green-limiter (0.0-2.0, 0 = off). Suppresses green cast/spill
|
||||||
|
# across the whole frame even where green is NOT the dominant channel — the
|
||||||
|
# case the bias/edge despill misses (e.g. green bounce on skin/a bald head).
|
||||||
|
# Caps green at a reference through the other two channels: max(r,b) [0.0] ->
|
||||||
|
# average [1.0] -> min(r,b) [2.0]. 0.5-0.7 for light cast; >1.0 for heavy
|
||||||
|
# close-up spill (2.0 = green can never exceed the smallest channel).
|
||||||
|
spill_suppress: float = 0.0
|
||||||
|
|
||||||
|
# Protect saturated yellows/warm fabrics from spill_suppress (0.0-1.0, 0 = off).
|
||||||
|
# spill_suppress caps green everywhere, which turns legit yellow (high r+g,
|
||||||
|
# low b) into orange. Blue is the tell: skin keeps some blue, yellow fabric
|
||||||
|
# reflects almost none. This gates the green-limiter down where
|
||||||
|
# min(r,g)-b is high (yellow) while leaving skin/scalp spill fully suppressed.
|
||||||
|
# 1.0 = full protection for strong yellows. Only matters when spill_suppress>0.
|
||||||
|
yellow_protect: float = 0.0
|
||||||
|
|
||||||
# Alpha bias: influences edge treatment (RGB)
|
# Alpha bias: influences edge treatment (RGB)
|
||||||
# Can help with edge color contamination
|
# Can help with edge color contamination
|
||||||
alpha_bias: tuple[int, int, int] = None
|
alpha_bias: tuple[int, int, int] = None
|
||||||
|
|
||||||
|
# Luminance protection: pixels with luma above this stay fully opaque (0-255, -1 = off)
|
||||||
|
# Use ~220 to protect white objects (headphones, teeth) from being partially keyed.
|
||||||
|
protect_luma: int = -1
|
||||||
|
|
||||||
|
# Shadow boost: extra key strength for dark pixels (0.0-5.0, 0 = off)
|
||||||
|
# Ramps up key signal proportionally to how dark a pixel is, helping key dark greens
|
||||||
|
# without affecting bright foreground areas. Values 1.0-2.0 are typical.
|
||||||
|
shadow_boost: float = 0.0
|
||||||
|
|
||||||
# Edge refinement
|
# Edge refinement
|
||||||
edge_erode: int = 0 # Pixels to erode from alpha edge (0-5)
|
edge_erode: int = 0 # Pixels to erode from alpha edge (0-5)
|
||||||
edge_soften: float = 0.0 # Blur the alpha edge (0-5 pixels)
|
edge_soften: float = 0.0 # Blur the alpha edge (0-5 pixels)
|
||||||
@@ -170,6 +205,18 @@ class ColorGradeConfig:
|
|||||||
brightness: float = 0.0 # Brightness adjustment (-1.0 to 1.0, 0 = no change)
|
brightness: float = 0.0 # Brightness adjustment (-1.0 to 1.0, 0 = no change)
|
||||||
saturation: float = 1.0 # Saturation multiplier (0.0-3.0, 1.0 = no change)
|
saturation: float = 1.0 # Saturation multiplier (0.0-3.0, 1.0 = no change)
|
||||||
|
|
||||||
|
# Auto-levels: a fixed histogram stretch (crush blacks, lift whites) for
|
||||||
|
# punch, like Photoshop auto-levels. 0 = off, 1 = strong. Fixed (not
|
||||||
|
# per-frame adaptive) so it can't flicker; it remaps a constant [lo,hi]
|
||||||
|
# window to full range, so the keyed-out background never skews it.
|
||||||
|
auto_levels: float = 0.0
|
||||||
|
|
||||||
|
# Yellow tint: hue-selective nudge of ONLY the yellow range (leaves reds/skin
|
||||||
|
# alone), to counter the orange shift auto-levels/saturation gives a yellow
|
||||||
|
# costume. <0 pulls yellows back toward green/pure yellow (preserve), >0
|
||||||
|
# pushes them warmer/orange. Range roughly -1.0..1.0. 0 = off.
|
||||||
|
yellow_tint: float = 0.0
|
||||||
|
|
||||||
# Custom curves for lift/gamma/gain control
|
# Custom curves for lift/gamma/gain control
|
||||||
# Format: "0/0 0.5/0.56 1/1" means (input/output) control points
|
# Format: "0/0 0.5/0.56 1/1" means (input/output) control points
|
||||||
curves_r: str = "" # Red channel curve
|
curves_r: str = "" # Red channel curve
|
||||||
@@ -195,7 +242,7 @@ class AudioNormalizeConfig:
|
|||||||
Applies noise reduction, compression, and loudness normalization
|
Applies noise reduction, compression, and loudness normalization
|
||||||
to improve audio quality and consistency.
|
to improve audio quality and consistency.
|
||||||
"""
|
"""
|
||||||
|
enabled: bool = True # Master switch to enable/disable all audio processing
|
||||||
# Parametric EQ bands (applied before other processing)
|
# Parametric EQ bands (applied before other processing)
|
||||||
eq_bands: list[EQBand] = field(default_factory=list)
|
eq_bands: list[EQBand] = field(default_factory=list)
|
||||||
|
|
||||||
@@ -300,14 +347,15 @@ class VideoSource:
|
|||||||
False # If True, skip loudnorm during preprocessing (apply after concatenation)
|
False # If True, skip loudnorm during preprocessing (apply after concatenation)
|
||||||
)
|
)
|
||||||
volume: float = 1.0 # Volume multiplier (1.0=full, >1.0=boost, <1.0=reduce)
|
volume: float = 1.0 # Volume multiplier (1.0=full, >1.0=boost, <1.0=reduce)
|
||||||
layer: str = "above" # "above" = renders on top of slides; "below" = behind slides
|
layer: str = "above" # "above" = on top of slides; "mid" = above narrator/below slides; "below" = behind narrator
|
||||||
duration: Optional[
|
duration: Optional[
|
||||||
float
|
float
|
||||||
] = None # Pre-probed file duration in seconds (set by import)
|
] = None # Pre-probed file duration in seconds (set by import)
|
||||||
has_audio: Optional[bool] = None # Pre-detected audio presence (set by import)
|
has_audio: Optional[bool] = None # Pre-detected audio presence (set by import)
|
||||||
end_on: Optional[
|
end_on: Optional[
|
||||||
str
|
str
|
||||||
] = None # When video event ends: "next_slide" | "end" | "take" (None = marker-type default)
|
] = None # When video event ends: "end" (play once to natural length) | "loop" (loop to render end)
|
||||||
|
# | "next_slide" | "next_video" | "take" (None = marker-type default: next_slide for videos)
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
@@ -368,6 +416,11 @@ class AudioEvent:
|
|||||||
audio_id: str
|
audio_id: str
|
||||||
start_time: float # When to start playing (marker time - offset)
|
start_time: float # When to start playing (marker time - offset)
|
||||||
audio_def: AudioDefinition
|
audio_def: AudioDefinition
|
||||||
|
# Chunking v2 (docs/chunking_v2.md): when a clip began before this chunk's
|
||||||
|
# window it must resume mid-track, not restart at the seam. src_offset is the
|
||||||
|
# position (seconds) into the source stream to begin at — the loop phase for
|
||||||
|
# looping music, or a linear seek for one-shots. 0.0 = play from the start (v1).
|
||||||
|
src_offset: float = 0.0
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
@@ -381,6 +434,11 @@ class VideoEvent:
|
|||||||
cutout: "CutoutDefinition"
|
cutout: "CutoutDefinition"
|
||||||
cutout_name: str = "" # resolved cutout name (e.g. "fullscreen"), for display
|
cutout_name: str = "" # resolved cutout name (e.g. "fullscreen"), for display
|
||||||
layer: str = "above" # "above" = on top of slides; "below" = behind slides
|
layer: str = "above" # "above" = on top of slides; "below" = behind slides
|
||||||
|
# Chunking v2 seam (see docs/chunking_v2.md): when a clip began before this
|
||||||
|
# chunk's window, the render must seek into it so it resumes mid-clip instead of
|
||||||
|
# restarting at the boundary. None = play from video_source.skip (the v1/default).
|
||||||
|
# Set only by the v2 overlap-inclusion path; the renderer prefers it when present.
|
||||||
|
skip_override: Optional[float] = None
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
@@ -516,6 +574,11 @@ class RenderPlan:
|
|||||||
narration_pauses: list[NarrationPause] = field(
|
narration_pauses: list[NarrationPause] = field(
|
||||||
default_factory=list
|
default_factory=list
|
||||||
) # Gaps in narration for interstitial videos
|
) # Gaps in narration for interstitial videos
|
||||||
|
# Render-time narration concat: ordered segments (skip/take + offset) to
|
||||||
|
# concatenate directly at render time. Typed loosely (list of
|
||||||
|
# narration.NarrationSegment) to avoid a circular import between models and
|
||||||
|
# narration.
|
||||||
|
narration_segments: list = field(default_factory=list)
|
||||||
# Outro sequence (plays after narration ends)
|
# Outro sequence (plays after narration ends)
|
||||||
outro_events: list["OutroEvent"] = field(
|
outro_events: list["OutroEvent"] = field(
|
||||||
default_factory=list
|
default_factory=list
|
||||||
|
|||||||
@@ -0,0 +1,120 @@
|
|||||||
|
"""Deterministic narration scheduling for render-time segment concatenation.
|
||||||
|
|
||||||
|
Rather than pre-concatenating segments into one file, the render stage
|
||||||
|
concatenates the processed segments directly. From narration.json + the cached
|
||||||
|
per-segment transcripts this module computes two things:
|
||||||
|
|
||||||
|
1. an ordered segment schedule (processed file, skip, take, and offset in the
|
||||||
|
combined timeline) — this drives the ffmpeg concat at render time; and
|
||||||
|
2. the merged word-level transcript, with every word re-timed into the
|
||||||
|
combined timeline — this drives slide alignment, exactly what
|
||||||
|
re-transcribing a pre-concatenated narration file used to produce, but derived
|
||||||
|
deterministically (no re-transcription, no separate combined file).
|
||||||
|
|
||||||
|
The processed files share framerate and format and are uncompressed, so the
|
||||||
|
computed offsets match the concatenated audio timeline sample-for-sample.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Callable, Optional
|
||||||
|
|
||||||
|
from .models import VideoSource
|
||||||
|
from .preprocessor import get_preprocessed_path
|
||||||
|
from .transcriber import TranscribedWord, load_transcript
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class NarrationSegment:
|
||||||
|
"""One narration segment placed on the combined render timeline."""
|
||||||
|
|
||||||
|
seg_id: str
|
||||||
|
source_path: Path # processed file to concatenate
|
||||||
|
skip: float # seconds trimmed from the segment's start
|
||||||
|
take: Optional[float] # seconds kept from `skip` (None → to end)
|
||||||
|
duration: float # effective seconds contributed to the timeline
|
||||||
|
offset: float # start time of this segment in the combined timeline
|
||||||
|
|
||||||
|
|
||||||
|
def segment_order(narration: dict) -> list[str]:
|
||||||
|
"""Natural sort of segment ids (S2 before S10, s1-15 before s16-end)."""
|
||||||
|
return sorted(
|
||||||
|
narration.keys(),
|
||||||
|
key=lambda s: [int(t) if t.isdigit() else t.lower() for t in re.split(r"(\d+)", s)],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def build_narration_schedule(
|
||||||
|
narration: dict[str, VideoSource],
|
||||||
|
narration_dir: Path,
|
||||||
|
get_duration: Callable[[Path], float],
|
||||||
|
transcripts_dir: Optional[Path] = None,
|
||||||
|
verbose: bool = False,
|
||||||
|
) -> tuple[list[NarrationSegment], list[TranscribedWord]]:
|
||||||
|
"""Return (ordered segments with offsets, merged transcript in timeline).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
narration: seg_id -> VideoSource (from parse_narration).
|
||||||
|
narration_dir: base dir the processed files resolve against
|
||||||
|
(media/narration; output_file is like processed/…mov).
|
||||||
|
get_duration: callable(Path) -> float (ffprobe duration), used only
|
||||||
|
when a segment has no explicit take.
|
||||||
|
transcripts_dir: where per-segment {seg_id}.json transcripts live
|
||||||
|
(defaults to narration_dir/transcripts).
|
||||||
|
"""
|
||||||
|
if transcripts_dir is None:
|
||||||
|
transcripts_dir = narration_dir / "transcripts"
|
||||||
|
|
||||||
|
segments: list[NarrationSegment] = []
|
||||||
|
merged: list[TranscribedWord] = []
|
||||||
|
offset = 0.0
|
||||||
|
|
||||||
|
for seg_id in segment_order(narration):
|
||||||
|
vs = narration[seg_id]
|
||||||
|
source_path = get_preprocessed_path(narration_dir, vs)
|
||||||
|
|
||||||
|
skip = vs.skip or 0.0
|
||||||
|
take = vs.take
|
||||||
|
if take is not None:
|
||||||
|
eff = max(0.0, take)
|
||||||
|
else:
|
||||||
|
full_dur = get_duration(source_path) if source_path.exists() else 0.0
|
||||||
|
eff = max(0.0, full_dur - skip)
|
||||||
|
seg_end = skip + eff # kept window end in the segment's own timeline
|
||||||
|
|
||||||
|
segments.append(
|
||||||
|
NarrationSegment(
|
||||||
|
seg_id=seg_id,
|
||||||
|
source_path=source_path,
|
||||||
|
skip=skip,
|
||||||
|
take=take,
|
||||||
|
duration=eff,
|
||||||
|
offset=offset,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Re-time this segment's transcript into the combined timeline: keep only
|
||||||
|
# words inside [skip, seg_end], subtract skip, and add the running offset.
|
||||||
|
tpath = transcripts_dir / f"{seg_id}.json"
|
||||||
|
if tpath.exists():
|
||||||
|
kept = 0
|
||||||
|
for w in load_transcript(tpath):
|
||||||
|
if w.end <= skip or w.start >= seg_end:
|
||||||
|
continue # entirely outside the kept window
|
||||||
|
new_start = max(w.start, skip) - skip + offset
|
||||||
|
new_end = min(w.end, seg_end) - skip + offset
|
||||||
|
merged.append(
|
||||||
|
TranscribedWord(word=w.word, start=round(new_start, 3), end=round(new_end, 3))
|
||||||
|
)
|
||||||
|
kept += 1
|
||||||
|
if verbose:
|
||||||
|
print(f" {seg_id}: +{kept} words (skip={skip:.2f}s take={eff:.2f}s → offset {offset:.2f}s)")
|
||||||
|
elif verbose:
|
||||||
|
print(f" ⚠ {seg_id}: no transcript at {tpath} — words missing from merged transcript")
|
||||||
|
|
||||||
|
offset += eff
|
||||||
|
|
||||||
|
return segments, merged
|
||||||
+141
-11
@@ -54,6 +54,55 @@ def _resolve_case_insensitive(path: Path) -> Path:
|
|||||||
return resolved
|
return resolved
|
||||||
|
|
||||||
|
|
||||||
|
# Inline marker-override keys honored at build time. These are the per-occurrence
|
||||||
|
# presentation fields materialized onto events.json and resolved per-event
|
||||||
|
# (transformer.resolve_video_presentation). Global params (skip/zoom/volume/…) are not
|
||||||
|
# yet overridable inline; unknown keys are ignored.
|
||||||
|
_MARKER_OVERRIDE_KEYS = frozenset({"cutout", "layer", "end_on", "take"})
|
||||||
|
|
||||||
|
|
||||||
|
def _coerce_marker_value(key: str, raw: str):
|
||||||
|
"""Coerce an inline override value: `take` → float; everything else → string."""
|
||||||
|
v = raw.strip().strip('"').strip("'")
|
||||||
|
if key == "take":
|
||||||
|
try:
|
||||||
|
return float(v)
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
return v
|
||||||
|
|
||||||
|
|
||||||
|
def parse_marker(raw: str) -> "tuple[str, Optional[dict]]":
|
||||||
|
"""Split a bracket's contents into (marker_id, overrides).
|
||||||
|
|
||||||
|
Supports the inline-overload grammar `[prefix:handle, key=value, key=value]`:
|
||||||
|
the text before the first comma is the marker id (prefix:handle); the rest are
|
||||||
|
comma-separated key=value overrides. Only _MARKER_OVERRIDE_KEYS are kept (others
|
||||||
|
ignored). A plain marker (no comma) returns (marker_id, None).
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
"vsb:clip" -> ("vsb:clip", None)
|
||||||
|
"vsb:clip, end_on=next_video" -> ("vsb:clip", {"end_on": "next_video"})
|
||||||
|
"video:clip, cutout=square, layer=below"
|
||||||
|
-> ("video:clip", {"cutout": "square", "layer": "below"})
|
||||||
|
"""
|
||||||
|
if "," not in raw:
|
||||||
|
return raw.strip(), None
|
||||||
|
head, _, tail = raw.partition(",")
|
||||||
|
marker_id = head.strip()
|
||||||
|
overrides: dict = {}
|
||||||
|
for tok in tail.split(","):
|
||||||
|
if "=" not in tok:
|
||||||
|
continue
|
||||||
|
key, _, val = tok.partition("=")
|
||||||
|
key = key.strip().lower()
|
||||||
|
if key in _MARKER_OVERRIDE_KEYS:
|
||||||
|
coerced = _coerce_marker_value(key, val)
|
||||||
|
if coerced is not None:
|
||||||
|
overrides[key] = coerced
|
||||||
|
return marker_id, (overrides or None)
|
||||||
|
|
||||||
|
|
||||||
def parse_manuscript(
|
def parse_manuscript(
|
||||||
project_path: Path,
|
project_path: Path,
|
||||||
) -> tuple[str, list[str], list[tuple[int, str]], list[Citation]]:
|
) -> tuple[str, list[str], list[tuple[int, str]], list[Citation]]:
|
||||||
@@ -80,13 +129,18 @@ def parse_manuscript(
|
|||||||
# Strip [cite:...] markers from text so they don't pollute alignment
|
# Strip [cite:...] markers from text so they don't pollute alignment
|
||||||
text = re.sub(r"\[cite:[^\]]+\]", "", text)
|
text = re.sub(r"\[cite:[^\]]+\]", "", text)
|
||||||
|
|
||||||
# Strip [marker:...] and [cue:...] markers (personal recording cues, ignored by pipeline)
|
# Strip narrator cues (ignored by pipeline)
|
||||||
text = re.sub(r"\[marker:[^\]]+\]", "", text)
|
text = re.sub(r"\[marker:[^\]]+\]", "", text)
|
||||||
text = re.sub(r"\[cue:[^\]]+\]", "", text)
|
text = re.sub(r"\[cue:[^\]]+\]", "", text)
|
||||||
|
text = re.sub(r"\[pause\]", "", text)
|
||||||
|
text = re.sub(r"\[stop\]", "", text)
|
||||||
|
|
||||||
# Extract all valid markers like [S1], [video:demo], [Zoom2], etc.
|
# Extract all valid markers like [S1], [video:demo], [vf2m:pexels/clip-name], and
|
||||||
# Include . in pattern to catch markers with file extensions (so validator can warn about them)
|
# inline-override forms like [vsb:clip, end_on=next_video]. Include / and - for
|
||||||
markers = re.findall(r"\[([A-Za-z0-9_:.]+)\]", text)
|
# pexels/library video IDs; . for file extensions; an optional ",…" tail carries
|
||||||
|
# per-event overrides (parsed out by parse_marker, so `markers` holds bare ids).
|
||||||
|
raw_markers = re.findall(r"\[([A-Za-z0-9_:./\-]+(?:,[^\]\n]*)?)\]", text)
|
||||||
|
markers = [parse_marker(m)[0] for m in raw_markers]
|
||||||
|
|
||||||
# Find malformed markers (missing brackets, extra spaces, etc.)
|
# Find malformed markers (missing brackets, extra spaces, etc.)
|
||||||
malformed: list[tuple[int, str]] = []
|
malformed: list[tuple[int, str]] = []
|
||||||
@@ -221,6 +275,24 @@ def parse_project_config(project_path: Path) -> ProjectConfig:
|
|||||||
# override the built-ins above.
|
# override the built-ins above.
|
||||||
cutouts_data = data.get("cutouts", {})
|
cutouts_data = data.get("cutouts", {})
|
||||||
for cutout_name, cutout_data in cutouts_data.items():
|
for cutout_name, cutout_data in cutouts_data.items():
|
||||||
|
if "cx" in cutout_data or "cy" in cutout_data:
|
||||||
|
# New center-based model: cx/cy are the center (fractions of frame W/H);
|
||||||
|
# width/height/margin are fractions of min(W,H). width==height → square.
|
||||||
|
w_pct = _parse_percent(cutout_data.get("width", "100%"))
|
||||||
|
h_pct = _parse_percent(cutout_data.get("height", cutout_data.get("width", "100%")))
|
||||||
|
cutouts[cutout_name] = CutoutDefinition(
|
||||||
|
x=-1,
|
||||||
|
y=-1,
|
||||||
|
height=-1,
|
||||||
|
width=-1,
|
||||||
|
cx_percent=_parse_percent(cutout_data.get("cx", "50%")),
|
||||||
|
cy_percent=_parse_percent(cutout_data.get("cy", "50%")),
|
||||||
|
width_percent=w_pct,
|
||||||
|
height_percent=h_pct,
|
||||||
|
margin_percent=_parse_percent(cutout_data.get("margin", "0%")),
|
||||||
|
size_relative_to_min=True,
|
||||||
|
)
|
||||||
|
continue
|
||||||
x, x_pct = _parse_dimension(cutout_data.get("x", 0))
|
x, x_pct = _parse_dimension(cutout_data.get("x", 0))
|
||||||
y, y_pct = _parse_dimension(cutout_data.get("y", 0))
|
y, y_pct = _parse_dimension(cutout_data.get("y", 0))
|
||||||
height, height_pct = _parse_dimension(cutout_data.get("height", 200))
|
height, height_pct = _parse_dimension(cutout_data.get("height", 200))
|
||||||
@@ -258,6 +330,7 @@ def parse_project_config(project_path: Path) -> ProjectConfig:
|
|||||||
slides_path=data.get("slides", "slides.json"),
|
slides_path=data.get("slides", "slides.json"),
|
||||||
videos_path=data.get("videos", "videos.json"),
|
videos_path=data.get("videos", "videos.json"),
|
||||||
audio_path=data.get("audio", "audio.json"),
|
audio_path=data.get("audio", "audio.json"),
|
||||||
|
transcript_path=data.get("transcript"),
|
||||||
audio_source=data.get("audio_source"),
|
audio_source=data.get("audio_source"),
|
||||||
main_video=data.get("main_video"),
|
main_video=data.get("main_video"),
|
||||||
process_cache=data.get("process_cache"),
|
process_cache=data.get("process_cache"),
|
||||||
@@ -287,6 +360,20 @@ def _parse_dimension(value: Any) -> tuple[int, float]:
|
|||||||
return 200, 0.0 # default
|
return 200, 0.0 # default
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_percent(value: Any, default: float = 0.0) -> float:
|
||||||
|
"""Parse a percentage into a 0.0-based fraction. '177%' → 1.77; a bare number
|
||||||
|
is taken as an already-computed fraction (1.77 → 1.77). Used by the center-based
|
||||||
|
cutout model where values can exceed 100% (fullscreen width on 16:9)."""
|
||||||
|
if value is None:
|
||||||
|
return default
|
||||||
|
if isinstance(value, str):
|
||||||
|
v = value.strip()
|
||||||
|
if v.endswith("%"):
|
||||||
|
return float(v[:-1]) / 100.0
|
||||||
|
return float(v)
|
||||||
|
return float(value)
|
||||||
|
|
||||||
|
|
||||||
def parse_slides(
|
def parse_slides(
|
||||||
project_path: Path, config: ProjectConfig = None
|
project_path: Path, config: ProjectConfig = None
|
||||||
) -> dict[str, SlideDefinition]:
|
) -> dict[str, SlideDefinition]:
|
||||||
@@ -389,16 +476,25 @@ def parse_timestamp(value: str) -> float:
|
|||||||
- "2:54" → 2 minutes 54 seconds (174.0)
|
- "2:54" → 2 minutes 54 seconds (174.0)
|
||||||
- "1:23:45" → 1 hour 23 minutes 45 seconds
|
- "1:23:45" → 1 hour 23 minutes 45 seconds
|
||||||
- "2:54.5" → 2 minutes 54.5 seconds
|
- "2:54.5" → 2 minutes 54.5 seconds
|
||||||
|
- "2m:3.5s" → 2 minutes 3.5 seconds
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Time in seconds as a float.
|
Time in seconds as a float.
|
||||||
"""
|
"""
|
||||||
|
if value is None:
|
||||||
|
return 0.0
|
||||||
|
if isinstance(value, (int, float)):
|
||||||
|
return float(value)
|
||||||
|
|
||||||
|
value = value.strip()
|
||||||
if not value:
|
if not value:
|
||||||
return 0.0
|
return 0.0
|
||||||
|
|
||||||
value = value.strip()
|
|
||||||
|
|
||||||
# Remove trailing 's' if present (e.g., "3.5s")
|
# Remove trailing 's' if present (e.g., "3.5s")
|
||||||
|
if "h" in value:
|
||||||
|
value = value.replace("h", ":")
|
||||||
|
if "m" in value:
|
||||||
|
value = value.replace("m", ":")
|
||||||
if value.endswith("s"):
|
if value.endswith("s"):
|
||||||
value = value[:-1]
|
value = value[:-1]
|
||||||
|
|
||||||
@@ -464,6 +560,7 @@ def parse_videos(
|
|||||||
|
|
||||||
videos = {}
|
videos = {}
|
||||||
for video_id, video_data in data.items():
|
for video_id, video_data in data.items():
|
||||||
|
|
||||||
if "source_file" not in video_data:
|
if "source_file" not in video_data:
|
||||||
raise ParseError(
|
raise ParseError(
|
||||||
f"Video '{video_id}' missing required field 'source_file'", videos_path
|
f"Video '{video_id}' missing required field 'source_file'", videos_path
|
||||||
@@ -496,7 +593,11 @@ def parse_videos(
|
|||||||
|
|
||||||
# Handle skip/take - can use begin/end as user-friendly alternatives
|
# Handle skip/take - can use begin/end as user-friendly alternatives
|
||||||
skip = float(video_data.get("skip") or 0.0)
|
skip = float(video_data.get("skip") or 0.0)
|
||||||
take = float(video_data["take"]) if video_data.get("take") not in (None, "") else None
|
take = (
|
||||||
|
float(video_data["take"])
|
||||||
|
if video_data.get("take") not in (None, "")
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
|
||||||
# Convert begin/end to skip/take if provided
|
# Convert begin/end to skip/take if provided
|
||||||
if "begin" in video_data and video_data["begin"]:
|
if "begin" in video_data and video_data["begin"]:
|
||||||
@@ -576,12 +677,35 @@ def parse_narration(
|
|||||||
default_filters = config.default_filters if config else {}
|
default_filters = config.default_filters if config else {}
|
||||||
|
|
||||||
narration = {}
|
narration = {}
|
||||||
|
_narr_video_exts = {".mov", ".mp4", ".webm", ".avi", ".mkv", ".m4v"}
|
||||||
for segment_id, segment_data in data.items():
|
for segment_id, segment_data in data.items():
|
||||||
if "source_file" not in segment_data:
|
if not segment_data.get("source_file"):
|
||||||
|
# source_file can drift out of an entry (case/sync churn between
|
||||||
|
# machines). Recover the way import/prune/trim do: find the raw
|
||||||
|
# recording whose stem matches the segment id, case-insensitively.
|
||||||
|
recovered = None
|
||||||
|
for sub in ("raw_mov", "processed"):
|
||||||
|
sub_dir = narration_dir / sub
|
||||||
|
if not sub_dir.is_dir():
|
||||||
|
continue
|
||||||
|
for f in sorted(sub_dir.iterdir()):
|
||||||
|
if (
|
||||||
|
f.is_file()
|
||||||
|
and f.suffix.lower() in _narr_video_exts
|
||||||
|
and f.stem.lower() == segment_id.lower()
|
||||||
|
):
|
||||||
|
recovered = f"{sub}/{f.name}"
|
||||||
|
break
|
||||||
|
if recovered:
|
||||||
|
break
|
||||||
|
if recovered is None:
|
||||||
raise ParseError(
|
raise ParseError(
|
||||||
f"Narration segment '{segment_id}' missing required field 'source_file'",
|
f"Narration segment '{segment_id}' missing required field 'source_file' "
|
||||||
|
f"and no matching recording was found in raw_mov/ or processed/. "
|
||||||
|
f"Add a 'source_file' or run 'import' to repair narration.json.",
|
||||||
narration_path,
|
narration_path,
|
||||||
)
|
)
|
||||||
|
segment_data = {**segment_data, "source_file": recovered}
|
||||||
|
|
||||||
# Resolve filter - can be a list or a string reference to default_filters
|
# Resolve filter - can be a list or a string reference to default_filters
|
||||||
filter_value = segment_data.get("filter", [])
|
filter_value = segment_data.get("filter", [])
|
||||||
@@ -604,9 +728,11 @@ def parse_narration(
|
|||||||
skip = segment_data.get("skip", default_begin)
|
skip = segment_data.get("skip", default_begin)
|
||||||
take = segment_data.get("take")
|
take = segment_data.get("take")
|
||||||
|
|
||||||
# Explicit begin/end always override defaults
|
# Explicit begin/start/end always override defaults
|
||||||
if "begin" in segment_data and segment_data["begin"]:
|
if "begin" in segment_data and segment_data["begin"]:
|
||||||
skip = parse_timestamp(segment_data["begin"])
|
skip = parse_timestamp(segment_data["begin"])
|
||||||
|
elif "start" in segment_data and segment_data["start"]:
|
||||||
|
skip = parse_timestamp(segment_data["start"])
|
||||||
if "end" in segment_data and segment_data["end"]:
|
if "end" in segment_data and segment_data["end"]:
|
||||||
end_time = parse_timestamp(segment_data["end"])
|
end_time = parse_timestamp(segment_data["end"])
|
||||||
# take = end - begin (duration from begin to end)
|
# take = end - begin (duration from begin to end)
|
||||||
@@ -615,7 +741,11 @@ def parse_narration(
|
|||||||
narration[segment_id] = VideoSource(
|
narration[segment_id] = VideoSource(
|
||||||
source_file=segment_data["source_file"],
|
source_file=segment_data["source_file"],
|
||||||
filter=filter_list,
|
filter=filter_list,
|
||||||
output_file=segment_data.get("output_file"),
|
# New model stores the preprocessed output under "processed_file";
|
||||||
|
# "output_file" is the legacy key. Either maps to output_file, which
|
||||||
|
# render uses (preferring it when it exists, else source_file).
|
||||||
|
output_file=segment_data.get("processed_file")
|
||||||
|
or segment_data.get("output_file"),
|
||||||
take=take,
|
take=take,
|
||||||
skip=skip,
|
skip=skip,
|
||||||
zoom=segment_data.get("zoom", 1.0),
|
zoom=segment_data.get("zoom", 1.0),
|
||||||
|
|||||||
@@ -0,0 +1,479 @@
|
|||||||
|
"""Pexels video downloader for gnommo shared_assets.
|
||||||
|
|
||||||
|
Configure API key in ~/.gnommo.conf:
|
||||||
|
|
||||||
|
[pexels]
|
||||||
|
api_key = YOUR_KEY_HERE
|
||||||
|
|
||||||
|
Get a free key at https://www.pexels.com/api/
|
||||||
|
"""
|
||||||
|
|
||||||
|
import configparser
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
|
||||||
|
def get_pexels_api_key() -> Optional[str]:
|
||||||
|
config_path = Path.home() / ".gnommo.conf"
|
||||||
|
if not config_path.exists():
|
||||||
|
return None
|
||||||
|
cfg = configparser.ConfigParser()
|
||||||
|
cfg.read(config_path)
|
||||||
|
return cfg.get("pexels", "api_key", fallback=None)
|
||||||
|
|
||||||
|
|
||||||
|
def extract_pexels_id(source_file: str) -> Optional[str]:
|
||||||
|
"""Extract the numeric Pexels video ID from a source_file path.
|
||||||
|
|
||||||
|
Handles names like 'pexels/11868263-hd_1920_1080_24fps.mp4'
|
||||||
|
and 'pexels/12136677_1080_1920_30fps.mp4'.
|
||||||
|
"""
|
||||||
|
name = Path(source_file).stem.split("/")[-1]
|
||||||
|
m = re.match(r"^(\d+)", name)
|
||||||
|
return m.group(1) if m else None
|
||||||
|
|
||||||
|
|
||||||
|
def _fetch_video_info(pexels_id: str, api_key: str) -> Optional[dict]:
|
||||||
|
url = f"https://api.pexels.com/videos/videos/{pexels_id}"
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url,
|
||||||
|
headers={"Authorization": api_key, "User-Agent": "Mozilla/5.0 gnommo/1.0"},
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req, timeout=15) as resp:
|
||||||
|
return json.loads(resp.read())
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
print(f" [{pexels_id}] Pexels API error {e.code} — video may have been deleted", flush=True)
|
||||||
|
return None
|
||||||
|
except Exception as e:
|
||||||
|
print(f" [{pexels_id}] Pexels API error: {e}", flush=True)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def description_from_url(video_url: str) -> str:
|
||||||
|
"""Extract human-readable description from a Pexels video URL slug.
|
||||||
|
|
||||||
|
'https://www.pexels.com/video/abstract-television-noise-11868263/'
|
||||||
|
→ 'Abstract Television Noise'
|
||||||
|
"""
|
||||||
|
m = re.search(r"/video/([a-z0-9][a-z0-9-]+?)-\d+/?$", video_url)
|
||||||
|
if m:
|
||||||
|
return m.group(1).replace("-", " ").title()
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def _pick_best_video_file(video_files: list, source_file: str) -> Optional[dict]:
|
||||||
|
"""Select the video_files entry that best matches the hints in source_file."""
|
||||||
|
stem = Path(source_file).stem.split("/")[-1]
|
||||||
|
|
||||||
|
width_hint = height_hint = fps_hint = quality_hint = None
|
||||||
|
m = re.search(r"[_-](\d{3,4})[_-](\d{3,4})[_-](\d+)fps", stem)
|
||||||
|
if m:
|
||||||
|
width_hint = int(m.group(1))
|
||||||
|
height_hint = int(m.group(2))
|
||||||
|
fps_hint = int(m.group(3))
|
||||||
|
for q in ("uhd", "hd", "sd"):
|
||||||
|
if q in stem.lower():
|
||||||
|
quality_hint = q
|
||||||
|
break
|
||||||
|
|
||||||
|
mp4s = [f for f in video_files if f.get("file_type") == "video/mp4"]
|
||||||
|
if not mp4s:
|
||||||
|
mp4s = video_files # fall back to any format
|
||||||
|
|
||||||
|
def score(vf: dict) -> int:
|
||||||
|
s = 0
|
||||||
|
if quality_hint and vf.get("quality", "").lower() == quality_hint:
|
||||||
|
s += 10
|
||||||
|
if width_hint and vf.get("width") == width_hint:
|
||||||
|
s += 5
|
||||||
|
if height_hint and vf.get("height") == height_hint:
|
||||||
|
s += 5
|
||||||
|
if fps_hint and round(float(vf.get("fps") or 0)) == fps_hint:
|
||||||
|
s += 3
|
||||||
|
return s
|
||||||
|
|
||||||
|
return max(mp4s, key=score)
|
||||||
|
|
||||||
|
|
||||||
|
def download_video(
|
||||||
|
source_file: str,
|
||||||
|
shared_assets_dir: Path,
|
||||||
|
api_key: str,
|
||||||
|
) -> Optional[dict]:
|
||||||
|
"""Download one Pexels video to shared_assets_dir/<source_file>.
|
||||||
|
|
||||||
|
Returns a metadata dict {description, duration, has_audio=False} on
|
||||||
|
success, or None on failure.
|
||||||
|
"""
|
||||||
|
pexels_id = extract_pexels_id(source_file)
|
||||||
|
if not pexels_id:
|
||||||
|
print(f" Cannot extract Pexels ID from: {source_file}", file=sys.stderr)
|
||||||
|
return None
|
||||||
|
|
||||||
|
target_path = shared_assets_dir / source_file
|
||||||
|
target_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
print(f" [{pexels_id}] Fetching video info...", flush=True)
|
||||||
|
info = _fetch_video_info(pexels_id, api_key)
|
||||||
|
if not info:
|
||||||
|
return None
|
||||||
|
|
||||||
|
description = description_from_url(info.get("url", ""))
|
||||||
|
duration = float(info.get("duration") or 0) or None
|
||||||
|
|
||||||
|
video_files = info.get("video_files", [])
|
||||||
|
if not video_files:
|
||||||
|
print(f" [{pexels_id}] No video files in API response", flush=True)
|
||||||
|
return None
|
||||||
|
|
||||||
|
best = _pick_best_video_file(video_files, source_file)
|
||||||
|
if not best:
|
||||||
|
return None
|
||||||
|
|
||||||
|
download_url = best["link"]
|
||||||
|
w, h, fps = best.get("width", "?"), best.get("height", "?"), best.get("fps", "?")
|
||||||
|
q = best.get("quality", "?")
|
||||||
|
label = f'"{description}" — ' if description else ""
|
||||||
|
print(f" [{pexels_id}] {label}{q} {w}x{h} @ {fps}fps", flush=True)
|
||||||
|
print(f" → {target_path}", flush=True)
|
||||||
|
|
||||||
|
try:
|
||||||
|
req = urllib.request.Request(
|
||||||
|
download_url, headers={"User-Agent": "Mozilla/5.0 gnommo/1.0"}
|
||||||
|
)
|
||||||
|
with urllib.request.urlopen(req, timeout=300) as resp:
|
||||||
|
total = int(resp.headers.get("Content-Length") or 0)
|
||||||
|
downloaded = 0
|
||||||
|
chunks: list[bytes] = []
|
||||||
|
chunk_size = 1024 * 512 # 512 KB
|
||||||
|
while True:
|
||||||
|
chunk = resp.read(chunk_size)
|
||||||
|
if not chunk:
|
||||||
|
break
|
||||||
|
chunks.append(chunk)
|
||||||
|
downloaded += len(chunk)
|
||||||
|
if total:
|
||||||
|
pct = downloaded * 100 // total
|
||||||
|
mb_done = downloaded / 1024 / 1024
|
||||||
|
mb_total = total / 1024 / 1024
|
||||||
|
print(f" {pct:3d}% {mb_done:.1f}/{mb_total:.1f} MB\r", end="", flush=True)
|
||||||
|
print(f" Done — {downloaded / 1024 / 1024:.1f} MB ", flush=True)
|
||||||
|
target_path.write_bytes(b"".join(chunks))
|
||||||
|
except Exception as e:
|
||||||
|
print(f"\n Download failed: {e}", flush=True)
|
||||||
|
return None
|
||||||
|
|
||||||
|
return {
|
||||||
|
"description": description,
|
||||||
|
"duration": duration,
|
||||||
|
"has_audio": False, # conservative; renderer probes when needed
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def update_videos_json(
|
||||||
|
json_path: Path,
|
||||||
|
video_id: str,
|
||||||
|
metadata: dict,
|
||||||
|
) -> None:
|
||||||
|
"""Write description (and other metadata) into an existing videos.json entry."""
|
||||||
|
if not json_path.exists():
|
||||||
|
return
|
||||||
|
with open(json_path, "r", encoding="utf-8") as f:
|
||||||
|
raw = json.load(f)
|
||||||
|
if video_id not in raw:
|
||||||
|
return
|
||||||
|
changed = False
|
||||||
|
for key, value in metadata.items():
|
||||||
|
if value and raw[video_id].get(key) != value:
|
||||||
|
raw[video_id][key] = value
|
||||||
|
changed = True
|
||||||
|
if changed:
|
||||||
|
with open(json_path, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(raw, f, indent=2, ensure_ascii=False)
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_metadata(pexels_id: str, api_key: str) -> Optional[dict]:
|
||||||
|
"""Fetch only description and duration for a Pexels video (no download)."""
|
||||||
|
info = _fetch_video_info(pexels_id, api_key)
|
||||||
|
if not info:
|
||||||
|
return None
|
||||||
|
return {
|
||||||
|
"description": description_from_url(info.get("url", "")),
|
||||||
|
"duration": float(info.get("duration") or 0) or None,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def enrich_missing_descriptions(
|
||||||
|
shared_assets_dir: Path,
|
||||||
|
api_key: str,
|
||||||
|
) -> int:
|
||||||
|
"""Fetch descriptions from Pexels API for entries that have a file on disk but no description.
|
||||||
|
|
||||||
|
Scans shared_assets/videos.json for pexels/* entries where:
|
||||||
|
- description is absent or empty
|
||||||
|
- source_file exists on disk (locally or via cache)
|
||||||
|
|
||||||
|
Returns number of entries updated.
|
||||||
|
"""
|
||||||
|
from .cache import resolve_with_cache
|
||||||
|
|
||||||
|
videos_json = shared_assets_dir / "videos.json"
|
||||||
|
if not videos_json.exists():
|
||||||
|
return 0
|
||||||
|
|
||||||
|
with open(videos_json, "r", encoding="utf-8") as f:
|
||||||
|
raw = json.load(f)
|
||||||
|
|
||||||
|
candidates = [
|
||||||
|
(vid_id, entry)
|
||||||
|
for vid_id, entry in raw.items()
|
||||||
|
if vid_id.startswith("pexels/") and not entry.get("description")
|
||||||
|
]
|
||||||
|
|
||||||
|
# Filter to those whose file exists on disk
|
||||||
|
to_enrich = []
|
||||||
|
for vid_id, entry in candidates:
|
||||||
|
sf = entry.get("source_file", "")
|
||||||
|
if not sf:
|
||||||
|
continue
|
||||||
|
path = shared_assets_dir / sf
|
||||||
|
resolved, _ = resolve_with_cache(path, shared_assets_dir)
|
||||||
|
if resolved.exists():
|
||||||
|
pexels_id = extract_pexels_id(sf)
|
||||||
|
if pexels_id:
|
||||||
|
to_enrich.append((vid_id, pexels_id))
|
||||||
|
|
||||||
|
if not to_enrich:
|
||||||
|
return 0
|
||||||
|
|
||||||
|
print(f" Enriching descriptions for {len(to_enrich)} existing pexels video(s)...", flush=True)
|
||||||
|
|
||||||
|
updated = 0
|
||||||
|
for vid_id, pexels_id in to_enrich:
|
||||||
|
meta = fetch_metadata(pexels_id, api_key)
|
||||||
|
if meta and meta.get("description"):
|
||||||
|
print(f" [{pexels_id}] \"{meta['description']}\"", flush=True)
|
||||||
|
update_videos_json(videos_json, vid_id, meta)
|
||||||
|
updated += 1
|
||||||
|
else:
|
||||||
|
print(f" [{pexels_id}] not found or no description — skipped", flush=True)
|
||||||
|
|
||||||
|
return updated
|
||||||
|
|
||||||
|
|
||||||
|
def _search_videos(
|
||||||
|
query: str, api_key: str, per_page: int = 80, page: int = 1
|
||||||
|
) -> Optional[dict]:
|
||||||
|
"""Call the Pexels video search API and return the raw response."""
|
||||||
|
import urllib.parse
|
||||||
|
|
||||||
|
params = urllib.parse.urlencode({"query": query, "per_page": per_page, "page": page})
|
||||||
|
url = f"https://api.pexels.com/videos/search?{params}"
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url,
|
||||||
|
headers={"Authorization": api_key, "User-Agent": "Mozilla/5.0 gnommo/1.0"},
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req, timeout=15) as resp:
|
||||||
|
return json.loads(resp.read())
|
||||||
|
except Exception as e:
|
||||||
|
print(f" Pexels search error: {e}", flush=True)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _pick_best_quality(video_files: list) -> Optional[dict]:
|
||||||
|
"""Pick the highest-resolution MP4 from a search result's video_files list."""
|
||||||
|
mp4s = [f for f in video_files if f.get("file_type") == "video/mp4"]
|
||||||
|
if not mp4s:
|
||||||
|
mp4s = video_files
|
||||||
|
if not mp4s:
|
||||||
|
return None
|
||||||
|
return max(mp4s, key=lambda f: f.get("width", 0) * f.get("height", 0))
|
||||||
|
|
||||||
|
|
||||||
|
def _make_source_filename(pexels_id: str, video_file: dict) -> str:
|
||||||
|
"""Build a canonical filename like 12345678_1920_1080_30fps.mp4."""
|
||||||
|
w = video_file.get("width", 0)
|
||||||
|
h = video_file.get("height", 0)
|
||||||
|
fps = round(float(video_file.get("fps") or 0))
|
||||||
|
return f"{pexels_id}_{w}_{h}_{fps}fps.mp4"
|
||||||
|
|
||||||
|
|
||||||
|
def _download_bytes(url: str, target_path: Path) -> bool:
|
||||||
|
"""Stream-download url to target_path with a progress indicator. Returns True on success."""
|
||||||
|
try:
|
||||||
|
req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0 gnommo/1.0"})
|
||||||
|
with urllib.request.urlopen(req, timeout=300) as resp:
|
||||||
|
total = int(resp.headers.get("Content-Length") or 0)
|
||||||
|
done = 0
|
||||||
|
chunks: list[bytes] = []
|
||||||
|
while True:
|
||||||
|
chunk = resp.read(524288) # 512 KB
|
||||||
|
if not chunk:
|
||||||
|
break
|
||||||
|
chunks.append(chunk)
|
||||||
|
done += len(chunk)
|
||||||
|
if total:
|
||||||
|
pct = done * 100 // total
|
||||||
|
print(
|
||||||
|
f" {pct:3d}% {done/1048576:.1f}/{total/1048576:.1f} MB\r",
|
||||||
|
end="",
|
||||||
|
flush=True,
|
||||||
|
)
|
||||||
|
print(f" Done — {done/1048576:.1f} MB ", flush=True)
|
||||||
|
target_path.write_bytes(b"".join(chunks))
|
||||||
|
return True
|
||||||
|
except Exception as e:
|
||||||
|
print(f"\n Download failed: {e}", flush=True)
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def search_and_download(
|
||||||
|
query: str,
|
||||||
|
pexels_dir: Path,
|
||||||
|
shared_videos_json: Path,
|
||||||
|
api_key: str,
|
||||||
|
max_results: int = 200,
|
||||||
|
) -> tuple[int, int]:
|
||||||
|
"""Search Pexels for *query* and download all results to pexels_dir.
|
||||||
|
|
||||||
|
Each video is saved as ``pexels_dir/{pexels_id}_{w}_{h}_{fps}fps.mp4`` and
|
||||||
|
registered in *shared_videos_json* so the renderer can find it.
|
||||||
|
|
||||||
|
Returns (downloaded_count, skipped_count).
|
||||||
|
"""
|
||||||
|
print(f"Searching Pexels for '{query}'...", flush=True)
|
||||||
|
pexels_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# Load existing registry so we can skip already-downloaded videos
|
||||||
|
existing: dict = {}
|
||||||
|
if shared_videos_json.exists():
|
||||||
|
with open(shared_videos_json, "r", encoding="utf-8") as f:
|
||||||
|
existing = json.load(f)
|
||||||
|
|
||||||
|
downloaded = 0
|
||||||
|
skipped = 0
|
||||||
|
page = 1
|
||||||
|
|
||||||
|
while downloaded + skipped < max_results:
|
||||||
|
per_page = min(80, max_results - downloaded - skipped)
|
||||||
|
result = _search_videos(query, api_key, per_page=per_page, page=page)
|
||||||
|
if not result:
|
||||||
|
break
|
||||||
|
|
||||||
|
videos = result.get("videos", [])
|
||||||
|
if not videos:
|
||||||
|
break
|
||||||
|
|
||||||
|
total_results = result.get("total_results", 0)
|
||||||
|
print(
|
||||||
|
f" Page {page}: {len(videos)} result(s) (Pexels total: {total_results})",
|
||||||
|
flush=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
for video in videos:
|
||||||
|
pexels_id = str(video.get("id", ""))
|
||||||
|
video_files = video.get("video_files", [])
|
||||||
|
if not pexels_id or not video_files:
|
||||||
|
continue
|
||||||
|
|
||||||
|
best = _pick_best_quality(video_files)
|
||||||
|
if not best:
|
||||||
|
continue
|
||||||
|
|
||||||
|
filename = _make_source_filename(pexels_id, best)
|
||||||
|
video_id = f"pexels/{Path(filename).stem}"
|
||||||
|
target_path = pexels_dir / filename
|
||||||
|
source_file = f"pexels/{filename}"
|
||||||
|
|
||||||
|
# Skip if already registered or file already on disk
|
||||||
|
if video_id in existing or target_path.exists():
|
||||||
|
if video_id not in existing:
|
||||||
|
# File exists but not registered — register it
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
skipped += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
description = description_from_url(video.get("url", ""))
|
||||||
|
duration = float(video.get("duration") or 0) or None
|
||||||
|
w = best.get("width", "?")
|
||||||
|
h = best.get("height", "?")
|
||||||
|
fps = best.get("fps", "?")
|
||||||
|
q = best.get("quality", "?")
|
||||||
|
label = f'"{description}" — ' if description else ""
|
||||||
|
print(f" [{pexels_id}] {label}{q} {w}x{h} @ {fps}fps", flush=True)
|
||||||
|
print(f" → {target_path}", flush=True)
|
||||||
|
|
||||||
|
if not target_path.exists():
|
||||||
|
if not _download_bytes(best["link"], target_path):
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Register in shared videos.json
|
||||||
|
existing[video_id] = {
|
||||||
|
"source_file": source_file,
|
||||||
|
"description": description,
|
||||||
|
"duration": duration,
|
||||||
|
"has_audio": False,
|
||||||
|
}
|
||||||
|
with open(shared_videos_json, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(existing, f, indent=2, ensure_ascii=False)
|
||||||
|
|
||||||
|
downloaded += 1
|
||||||
|
|
||||||
|
if not result.get("next_page"):
|
||||||
|
break
|
||||||
|
page += 1
|
||||||
|
|
||||||
|
return downloaded, skipped
|
||||||
|
|
||||||
|
|
||||||
|
def find_missing_pexels_videos(
|
||||||
|
manuscript_markers: list[str],
|
||||||
|
videos: dict,
|
||||||
|
shared_assets_dir: Path,
|
||||||
|
) -> list[tuple[str, str]]:
|
||||||
|
"""Return [(video_id, source_file)] for pexels videos referenced but not on disk."""
|
||||||
|
from .cache import resolve_with_cache
|
||||||
|
|
||||||
|
_VIDEO_PREFIXES = (
|
||||||
|
"video:", "narration:",
|
||||||
|
"vft:", "vfb:", "vfm:",
|
||||||
|
"vf2t:", "vf2b:", "vf2m:",
|
||||||
|
"vst:", "vsb:", "vsm:",
|
||||||
|
"vftp:", "vfbp:", "vfmp:",
|
||||||
|
"vf2tp:", "vf2bp:", "vf2mp:",
|
||||||
|
"vstp:", "vsbp:", "vsmp:",
|
||||||
|
)
|
||||||
|
|
||||||
|
seen: set[str] = set()
|
||||||
|
missing: list[tuple[str, str]] = []
|
||||||
|
|
||||||
|
for marker in manuscript_markers:
|
||||||
|
prefix = next((p for p in _VIDEO_PREFIXES if marker.startswith(p)), None)
|
||||||
|
if prefix is None:
|
||||||
|
continue
|
||||||
|
video_id = marker[len(prefix):].lower()
|
||||||
|
if video_id in seen or not video_id.startswith("pexels/"):
|
||||||
|
continue
|
||||||
|
seen.add(video_id)
|
||||||
|
|
||||||
|
source_file = videos.get(video_id, None)
|
||||||
|
if source_file is None:
|
||||||
|
# Not in videos.json yet — synthesize expected path from the ID
|
||||||
|
sf = video_id + ".mp4"
|
||||||
|
else:
|
||||||
|
sf = source_file.source_file if hasattr(source_file, "source_file") else source_file
|
||||||
|
|
||||||
|
candidate = shared_assets_dir / sf
|
||||||
|
resolved, _ = resolve_with_cache(candidate, shared_assets_dir)
|
||||||
|
if not resolved.exists():
|
||||||
|
missing.append((video_id, sf))
|
||||||
|
|
||||||
|
return missing
|
||||||
+434
-272
@@ -1,5 +1,6 @@
|
|||||||
"""Preprocessing stage: apply filters to source videos."""
|
"""Preprocessing stage: apply filters to source videos."""
|
||||||
|
|
||||||
|
import json
|
||||||
import os
|
import os
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
@@ -18,9 +19,11 @@ from .models import (
|
|||||||
)
|
)
|
||||||
from typing import Union, Optional
|
from typing import Union, Optional
|
||||||
|
|
||||||
|
|
||||||
def _tc() -> str:
|
def _tc() -> str:
|
||||||
"""Return FFmpeg thread count string from ~/.gnommo.conf [performance] cpu_limit."""
|
"""Return FFmpeg thread count string from ~/.gnommo.conf [performance] cpu_limit."""
|
||||||
from .cache import get_ffmpeg_thread_count
|
from .cache import get_ffmpeg_thread_count
|
||||||
|
|
||||||
return str(get_ffmpeg_thread_count())
|
return str(get_ffmpeg_thread_count())
|
||||||
|
|
||||||
|
|
||||||
@@ -85,6 +88,42 @@ def _video_has_alpha(video_path: Path) -> bool:
|
|||||||
return "yuva" in pix_fmt or "rgba" in pix_fmt or "bgra" in pix_fmt
|
return "yuva" in pix_fmt or "rgba" in pix_fmt or "bgra" in pix_fmt
|
||||||
|
|
||||||
|
|
||||||
|
# Codecs where every frame is independently coded (all-intra).
|
||||||
|
# Stream copy with -ss is frame-accurate for these — no re-encode needed.
|
||||||
|
_INTRA_ONLY_CODECS = frozenset(
|
||||||
|
{
|
||||||
|
"prores",
|
||||||
|
"prores_ks",
|
||||||
|
"mjpeg",
|
||||||
|
"dnxhd",
|
||||||
|
"dvvideo",
|
||||||
|
"huffyuv",
|
||||||
|
"ffv1",
|
||||||
|
"rawvideo",
|
||||||
|
"png",
|
||||||
|
"tiff",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _get_video_codec(video_path: Path) -> str:
|
||||||
|
"""Return the codec name of the first video stream (lowercase)."""
|
||||||
|
cmd = [
|
||||||
|
"ffprobe",
|
||||||
|
"-v",
|
||||||
|
"error",
|
||||||
|
"-select_streams",
|
||||||
|
"v:0",
|
||||||
|
"-show_entries",
|
||||||
|
"stream=codec_name",
|
||||||
|
"-of",
|
||||||
|
"default=noprint_wrappers=1:nokey=1",
|
||||||
|
str(video_path),
|
||||||
|
]
|
||||||
|
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||||
|
return result.stdout.strip().lower() if result.returncode == 0 else ""
|
||||||
|
|
||||||
|
|
||||||
def format_time(seconds: float) -> str:
|
def format_time(seconds: float) -> str:
|
||||||
"""Format seconds as human-readable time string."""
|
"""Format seconds as human-readable time string."""
|
||||||
if seconds < 60:
|
if seconds < 60:
|
||||||
@@ -113,6 +152,10 @@ def create_downscaled_video(
|
|||||||
if out_path.exists() and not force:
|
if out_path.exists() and not force:
|
||||||
return out_path
|
return out_path
|
||||||
|
|
||||||
|
# Write to a .partial sibling and atomically rename on success, so a run that
|
||||||
|
# crashed mid-encode (e.g. laptop battery died) never leaves a truncated file
|
||||||
|
# at out_path that a resuming run would silently reuse as if complete.
|
||||||
|
partial = out_path.with_name(out_path.stem + ".partial" + out_path.suffix)
|
||||||
cmd = [
|
cmd = [
|
||||||
"ffmpeg",
|
"ffmpeg",
|
||||||
"-y",
|
"-y",
|
||||||
@@ -132,16 +175,18 @@ def create_downscaled_video(
|
|||||||
"aac", # re-encode audio so both streams share the same PTS origin,
|
"aac", # re-encode audio so both streams share the same PTS origin,
|
||||||
"-ar", # avoiding the lip-sync drift caused by libx264 encoder delay
|
"-ar", # avoiding the lip-sync drift caused by libx264 encoder delay
|
||||||
"48000", # when audio is copied with its original timestamps
|
"48000", # when audio is copied with its original timestamps
|
||||||
str(out_path),
|
str(partial),
|
||||||
]
|
]
|
||||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||||
if result.returncode != 0:
|
if result.returncode != 0:
|
||||||
|
partial.unlink(missing_ok=True)
|
||||||
raise PreprocessError(
|
raise PreprocessError(
|
||||||
f"Failed to downscale {source_path.name} to {width}x{height}",
|
f"Failed to downscale {source_path.name} to {width}x{height}",
|
||||||
filter_type="downscale",
|
filter_type="downscale",
|
||||||
command=" ".join(cmd),
|
command=" ".join(cmd),
|
||||||
stderr=result.stderr,
|
stderr=result.stderr,
|
||||||
)
|
)
|
||||||
|
os.replace(partial, out_path)
|
||||||
return out_path
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
@@ -267,9 +312,164 @@ def ensure_proxy_files_exist(
|
|||||||
import selectors, time, sys, subprocess
|
import selectors, time, sys, subprocess
|
||||||
|
|
||||||
|
|
||||||
def run_ffmpeg_with_progress(cmd, duration, description="Processing"):
|
# Module-level FFmpeg log level. None → the quiet progress-bar path (-loglevel
|
||||||
|
# warning). A level string (e.g. "verbose", "debug") → stream FFmpeg's full
|
||||||
|
# output live instead of the bar — useful when debugging filter graphs.
|
||||||
|
_FFMPEG_LOGLEVEL: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
def set_ffmpeg_verbose(enabled: bool) -> None:
|
||||||
|
"""Turn on (verbose) / off streamed FFmpeg logging for every run in this process."""
|
||||||
|
global _FFMPEG_LOGLEVEL
|
||||||
|
_FFMPEG_LOGLEVEL = "verbose" if enabled else None
|
||||||
|
|
||||||
|
|
||||||
|
def set_ffmpeg_loglevel(level: Optional[str]) -> None:
|
||||||
|
"""Set an explicit FFmpeg -loglevel (info/verbose/debug/trace…) and stream its
|
||||||
|
output; pass None to restore the quiet progress bar."""
|
||||||
|
global _FFMPEG_LOGLEVEL
|
||||||
|
_FFMPEG_LOGLEVEL = level
|
||||||
|
|
||||||
|
|
||||||
|
def _mem_snapshot():
|
||||||
|
"""(available_bytes, total_bytes) of physical RAM, or None. Lightweight; used
|
||||||
|
only on the OOM error path so it can be duplicated from cli._system_mem without
|
||||||
|
a circular import."""
|
||||||
|
try:
|
||||||
|
import psutil
|
||||||
|
vm = psutil.virtual_memory()
|
||||||
|
return vm.available, vm.total
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
if sys.platform.startswith("linux"):
|
||||||
|
try:
|
||||||
|
info = {}
|
||||||
|
with open("/proc/meminfo") as f:
|
||||||
|
for line in f:
|
||||||
|
k, _, v = line.partition(":")
|
||||||
|
info[k.strip()] = int(v.strip().split()[0]) * 1024
|
||||||
|
return info.get("MemAvailable", info.get("MemFree", 0)), info.get("MemTotal", 0)
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _kernel_oom_lines():
|
||||||
|
"""Best-effort: the kernel's own OOM-killer log lines (Linux). Empty when not
|
||||||
|
Linux or when dmesg is restricted (dmesg_restrict=1 without privileges) — in
|
||||||
|
that case the signal/memory evidence still tells the story."""
|
||||||
|
if not sys.platform.startswith("linux"):
|
||||||
|
return []
|
||||||
|
for probe in (
|
||||||
|
["dmesg", "--ctime"],
|
||||||
|
["dmesg"],
|
||||||
|
["journalctl", "-k", "--no-pager", "-n", "300"],
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
r = subprocess.run(probe, capture_output=True, text=True, timeout=5)
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
if r.returncode != 0 or not r.stdout:
|
||||||
|
continue
|
||||||
|
hits = [
|
||||||
|
ln.strip()[:200]
|
||||||
|
for ln in r.stdout.splitlines()
|
||||||
|
if any(s in ln.lower() for s in ("out of memory", "oom-kill", "killed process"))
|
||||||
|
]
|
||||||
|
if hits:
|
||||||
|
return hits[-4:]
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def _oom_postmortem(returncode: int, log_text: str) -> None:
|
||||||
|
"""When ffmpeg dies from memory pressure, print a diagnosis instead of a bare
|
||||||
|
exit code. Detects the OS OOM killer's signature (SIGKILL) and explicit
|
||||||
|
allocation failures, reports current RAM, pulls the kernel oom-kill line when
|
||||||
|
readable, and lists mitigations. No-op for ordinary (non-OOM) failures."""
|
||||||
|
low = (log_text or "").lower()
|
||||||
|
killed = returncode in (-9, 137) # SIGKILL, or 128+9 via a shell wrapper
|
||||||
|
alloc_fail = (
|
||||||
|
"cannot allocate memory" in low
|
||||||
|
or "out of memory" in low
|
||||||
|
or "error while allocating" in low
|
||||||
|
or (returncode in (-6, 134) and "memory" in low)
|
||||||
|
)
|
||||||
|
if not (killed or alloc_fail):
|
||||||
|
return
|
||||||
|
|
||||||
|
out = ["", " " + "=" * 66,
|
||||||
|
" OOM DIAGNOSIS — ffmpeg was killed by memory pressure, not a normal error."]
|
||||||
|
if killed:
|
||||||
|
out += [
|
||||||
|
" Signal: SIGKILL (-9) — the signature of the OS out-of-memory killer.",
|
||||||
|
" The kernel terminated ffmpeg instantly, so ffmpeg logged no error of its",
|
||||||
|
" own; that is why the only visible symptom was 'process terminated'.",
|
||||||
|
]
|
||||||
|
if alloc_fail:
|
||||||
|
out.append(" ffmpeg reported an allocation failure (could not allocate memory).")
|
||||||
|
mem = _mem_snapshot()
|
||||||
|
if mem and mem[1]:
|
||||||
|
avail, total = mem
|
||||||
|
out.append(
|
||||||
|
f" Memory now: {avail / 1e9:.2f} GB free of {total / 1e9:.1f} GB "
|
||||||
|
f"({100 * (total - avail) / total:.0f}% used)."
|
||||||
|
)
|
||||||
|
for kline in _kernel_oom_lines():
|
||||||
|
out.append(f" kernel: {kline}")
|
||||||
|
out += [
|
||||||
|
" Mitigations (most effective first):",
|
||||||
|
" - render --res low smaller frames -> much less RAM per input",
|
||||||
|
" - render in slide chunks: raise chunk_slides / render_chunk_slides",
|
||||||
|
" - lower ffmpeg thread count in ~/.gnommo.conf (fewer parallel buffers)",
|
||||||
|
" - add RAM or swap; on WSL2 raise memory=/swap= in .wslconfig",
|
||||||
|
" " + "=" * 66, ""]
|
||||||
|
msg = "\n".join(out)
|
||||||
|
sys.stdout.write(msg + "\n")
|
||||||
|
sys.stdout.flush()
|
||||||
|
|
||||||
|
|
||||||
|
def run_ffmpeg_with_progress(cmd, duration, description="Processing", loglevel=None,
|
||||||
|
progress_hook=None):
|
||||||
|
"""progress_hook(seconds): optional callback invoked ~every 5s with the current
|
||||||
|
output timestamp. The render passes one that logs the active slide/asset + RAM,
|
||||||
|
so the last line flushed before a hard kill (even a whole-VM WSL crash) names
|
||||||
|
where in the timeline memory blew up."""
|
||||||
|
from collections import deque
|
||||||
|
|
||||||
cmd = cmd.copy()
|
cmd = cmd.copy()
|
||||||
|
|
||||||
|
# Retain only a bounded tail of FFmpeg's output. FFmpeg emits a -progress block
|
||||||
|
# every second (plus any per-frame warnings), so on a long or stuck-at-99% run an
|
||||||
|
# unbounded list grows in Python until it eats all RAM — which is what OOM'd the
|
||||||
|
# 66 GB render box while ffmpeg sat at the last frame. The last N lines are all
|
||||||
|
# the error path ever needs.
|
||||||
|
_LOG_TAIL = 5000
|
||||||
|
|
||||||
|
# Explicit call arg wins, else the module-level setting, else the quiet bar.
|
||||||
|
level = loglevel if loglevel is not None else _FFMPEG_LOGLEVEL
|
||||||
|
stream = level is not None
|
||||||
|
|
||||||
|
# Streamed mode: show FFmpeg's full output live (no progress bar), still
|
||||||
|
# capturing it so the error path keeps the full log.
|
||||||
|
if stream:
|
||||||
|
insert_pos = cmd.index("-y") + 1 if "-y" in cmd else 1
|
||||||
|
cmd[insert_pos:insert_pos] = ["-loglevel", level, "-stats"]
|
||||||
|
print(f" {description} — FFmpeg (-loglevel {level}):")
|
||||||
|
print(f" $ {' '.join(cmd)}")
|
||||||
|
p = subprocess.Popen(
|
||||||
|
cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
|
||||||
|
text=True, bufsize=1, universal_newlines=True,
|
||||||
|
)
|
||||||
|
logs = deque(maxlen=_LOG_TAIL)
|
||||||
|
for line in iter(p.stdout.readline, ""):
|
||||||
|
logs.append(line)
|
||||||
|
sys.stdout.write(line)
|
||||||
|
sys.stdout.flush()
|
||||||
|
p.wait()
|
||||||
|
if p.returncode != 0:
|
||||||
|
_oom_postmortem(p.returncode, "".join(logs))
|
||||||
|
return subprocess.CompletedProcess(cmd, p.returncode, stdout="", stderr="".join(logs))
|
||||||
|
|
||||||
insert_pos = cmd.index("-y") + 1 if "-y" in cmd else 1
|
insert_pos = cmd.index("-y") + 1 if "-y" in cmd else 1
|
||||||
cmd[insert_pos:insert_pos] = [
|
cmd[insert_pos:insert_pos] = [
|
||||||
"-progress",
|
"-progress",
|
||||||
@@ -297,7 +497,8 @@ def run_ffmpeg_with_progress(cmd, duration, description="Processing"):
|
|||||||
last_percent = 0
|
last_percent = 0
|
||||||
seen_any_progress = False
|
seen_any_progress = False
|
||||||
last_log_line = ""
|
last_log_line = ""
|
||||||
logs = []
|
last_hook = 0.0
|
||||||
|
logs = deque(maxlen=_LOG_TAIL)
|
||||||
|
|
||||||
def draw(percent, suffix=""):
|
def draw(percent, suffix=""):
|
||||||
filled = int(bar_width * percent / 100)
|
filled = int(bar_width * percent / 100)
|
||||||
@@ -358,6 +559,12 @@ def run_ffmpeg_with_progress(cmd, duration, description="Processing"):
|
|||||||
last_update = time.time()
|
last_update = time.time()
|
||||||
seen_any_progress = True
|
seen_any_progress = True
|
||||||
draw(last_percent, "")
|
draw(last_percent, "")
|
||||||
|
if progress_hook is not None and (time.time() - last_hook) >= 5.0:
|
||||||
|
last_hook = time.time()
|
||||||
|
try:
|
||||||
|
progress_hook(t_s)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
except ValueError:
|
except ValueError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@@ -372,6 +579,7 @@ def run_ffmpeg_with_progress(cmd, duration, description="Processing"):
|
|||||||
)
|
)
|
||||||
sys.stdout.write(f"\n FFmpeg exited with code {code}{signal_hint}\n")
|
sys.stdout.write(f"\n FFmpeg exited with code {code}{signal_hint}\n")
|
||||||
sys.stdout.flush()
|
sys.stdout.flush()
|
||||||
|
_oom_postmortem(code, "".join(logs))
|
||||||
|
|
||||||
return subprocess.CompletedProcess(
|
return subprocess.CompletedProcess(
|
||||||
cmd, p.returncode, stdout="", stderr="".join(logs)
|
cmd, p.returncode, stdout="", stderr="".join(logs)
|
||||||
@@ -582,11 +790,15 @@ def preprocess_video(
|
|||||||
# No filters defined, return original file
|
# No filters defined, return original file
|
||||||
return videos_dir / video_source.source_file
|
return videos_dir / video_source.source_file
|
||||||
|
|
||||||
# Use custom intermediate dir if provided, otherwise default to videos_dir/intermediate
|
# Use custom intermediate dir if provided, otherwise default to videos_dir/intermediate.
|
||||||
|
# Always namespace by video_id so segments processed in parallel (or across
|
||||||
|
# overlapping runs) never share the chunks/ dir and concat.txt — otherwise
|
||||||
|
# they overwrite each other's chunk_NNNN.mov and one segment's footage bleeds
|
||||||
|
# into another's processed output (duplicated/inflated narration).
|
||||||
if custom_gnommo_scratch:
|
if custom_gnommo_scratch:
|
||||||
gnommo_scratch = custom_gnommo_scratch / video_id
|
gnommo_scratch = custom_gnommo_scratch / video_id
|
||||||
else:
|
else:
|
||||||
gnommo_scratch = videos_dir / "intermediate"
|
gnommo_scratch = videos_dir / "intermediate" / video_id
|
||||||
gnommo_scratch.mkdir(parents=True, exist_ok=True)
|
gnommo_scratch.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
# Start with the source file (relative to videos_dir)
|
# Start with the source file (relative to videos_dir)
|
||||||
@@ -615,6 +827,10 @@ def preprocess_video(
|
|||||||
if channel == "auto":
|
if channel == "auto":
|
||||||
channel = _resolve_auto_channel(current_input)
|
channel = _resolve_auto_channel(current_input)
|
||||||
print(f" Auto channel detection: using '{channel}'")
|
print(f" Auto channel detection: using '{channel}'")
|
||||||
|
# Persist the resolved channel on the source so the caller can store it back
|
||||||
|
# to narration.json — the auto-detect (a full-file volumedetect) then runs
|
||||||
|
# ONCE here at preprocess, never again at render time.
|
||||||
|
video_source.use_audio_channels = channel
|
||||||
elif channel in ("left", "right"):
|
elif channel in ("left", "right"):
|
||||||
is_silent, max_vol = check_audio_channel_silent(current_input, channel)
|
is_silent, max_vol = check_audio_channel_silent(current_input, channel)
|
||||||
if is_silent:
|
if is_silent:
|
||||||
@@ -655,6 +871,8 @@ def preprocess_video(
|
|||||||
for batch in filter_batches:
|
for batch in filter_batches:
|
||||||
first_filter_type = batch[0].get("type")
|
first_filter_type = batch[0].get("type")
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
if first_filter_type in VIDEO_FILTER_TYPES:
|
if first_filter_type in VIDEO_FILTER_TYPES:
|
||||||
# Combined video filter batch - use chunked processing for large files
|
# Combined video filter batch - use chunked processing for large files
|
||||||
filter_names = "+".join(f.get("type") for f in batch)
|
filter_names = "+".join(f.get("type") for f in batch)
|
||||||
@@ -713,7 +931,7 @@ def preprocess_video(
|
|||||||
filter_type=first_filter_type,
|
filter_type=first_filter_type,
|
||||||
)
|
)
|
||||||
|
|
||||||
# If output_file is specified, copy/rename to final location and clean up
|
# If output_file is specified, move/copy to final location and clean up
|
||||||
if video_source.output_file:
|
if video_source.output_file:
|
||||||
import shutil
|
import shutil
|
||||||
|
|
||||||
@@ -721,8 +939,32 @@ def preprocess_video(
|
|||||||
|
|
||||||
final_output.parent.mkdir(parents=True, exist_ok=True)
|
final_output.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
# Copy the final intermediate to the output location
|
# Stage on a .tmp on the destination filesystem, then atomically swap in —
|
||||||
shutil.copy2(current_input, final_output)
|
# an interrupted transfer never leaves a partial file at the final path.
|
||||||
|
tmp_output = final_output.with_suffix(".tmp")
|
||||||
|
|
||||||
|
# current_input is the last batch's scratch intermediate (about to be
|
||||||
|
# deleted anyway), so MOVE it into place rather than copying — concat has
|
||||||
|
# already written this multi-GB ProRes once, and a rename is instant on the
|
||||||
|
# same filesystem. Only fall back to a copy across filesystems (e.g. scratch
|
||||||
|
# on an external SSD, output on the internal disk). Never move the original
|
||||||
|
# source (transcribe-only path leaves current_input == source).
|
||||||
|
moved = False
|
||||||
|
if current_input in intermediate_files:
|
||||||
|
try:
|
||||||
|
os.replace(current_input, tmp_output)
|
||||||
|
moved = True
|
||||||
|
except OSError:
|
||||||
|
moved = False # cross-filesystem — copy instead
|
||||||
|
if not moved:
|
||||||
|
try:
|
||||||
|
size_gb = current_input.stat().st_size / (1024**3)
|
||||||
|
print(f" Finalizing: copying {size_gb:.1f} GB to output (cross-disk)...")
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
shutil.copy2(current_input, tmp_output)
|
||||||
|
|
||||||
|
os.replace(tmp_output, final_output) # atomic on the destination filesystem
|
||||||
|
|
||||||
if verbose:
|
if verbose:
|
||||||
print(f" Final output: {final_output}")
|
print(f" Final output: {final_output}")
|
||||||
@@ -784,7 +1026,10 @@ def apply_combined_video_filters(
|
|||||||
|
|
||||||
cmd.extend(
|
cmd.extend(
|
||||||
[
|
[
|
||||||
"-probesize", "50000000", "-analyzeduration", "50000000",
|
"-probesize",
|
||||||
|
"50000000",
|
||||||
|
"-analyzeduration",
|
||||||
|
"50000000",
|
||||||
"-i",
|
"-i",
|
||||||
str(input_path),
|
str(input_path),
|
||||||
"-vf",
|
"-vf",
|
||||||
@@ -912,6 +1157,19 @@ def build_color_grade_filter(config: dict) -> str:
|
|||||||
# Start with format conversion to RGBA for color operations
|
# Start with format conversion to RGBA for color operations
|
||||||
parts.append("format=rgba")
|
parts.append("format=rgba")
|
||||||
|
|
||||||
|
# Auto-levels: a fixed contrast/levels stretch for punch. Crushes blacks and
|
||||||
|
# lifts whites by remapping [lo, hi] → [0, 1] equally per channel (luminance
|
||||||
|
# levels, no colour shift). Fixed points → no per-frame flicker, and no
|
||||||
|
# histogram is measured so the transparent green background can't skew it.
|
||||||
|
if grade_config.auto_levels > 0:
|
||||||
|
al = min(max(grade_config.auto_levels, 0.0), 1.0)
|
||||||
|
lo = round(0.10 * al, 4)
|
||||||
|
hi = round(1.0 - 0.10 * al, 4)
|
||||||
|
parts.append(
|
||||||
|
f"colorlevels=rimin={lo}:gimin={lo}:bimin={lo}:"
|
||||||
|
f"rimax={hi}:gimax={hi}:bimax={hi}"
|
||||||
|
)
|
||||||
|
|
||||||
# Color balance (only add if any value is non-zero)
|
# Color balance (only add if any value is non-zero)
|
||||||
colorbalance_parts = []
|
colorbalance_parts = []
|
||||||
if grade_config.rs != 0:
|
if grade_config.rs != 0:
|
||||||
@@ -952,6 +1210,16 @@ def build_color_grade_filter(config: dict) -> str:
|
|||||||
if eq_parts:
|
if eq_parts:
|
||||||
parts.append(f"eq={':'.join(eq_parts)}")
|
parts.append(f"eq={':'.join(eq_parts)}")
|
||||||
|
|
||||||
|
# Yellow tint: hue-selective correction of the yellow range only (skin, in
|
||||||
|
# the reds, is untouched). In CMYK terms a yellow's "orange-ness" is its
|
||||||
|
# magenta content, so we push magenta up (warmer/orange) or down (greener,
|
||||||
|
# preserving the costume yellow) proportional to yellow_tint.
|
||||||
|
if grade_config.yellow_tint != 0:
|
||||||
|
yt = max(-1.0, min(1.0, grade_config.yellow_tint))
|
||||||
|
magenta = round(0.6 * yt, 3) # red content of the yellow range
|
||||||
|
yellow = round(0.2 * yt, 3) # a little saturation follow so it reads
|
||||||
|
parts.append(f"selectivecolor=yellows=0 {magenta} {yellow} 0")
|
||||||
|
|
||||||
# Custom curves (if specified)
|
# Custom curves (if specified)
|
||||||
custom_curves = []
|
custom_curves = []
|
||||||
if grade_config.curves_r:
|
if grade_config.curves_r:
|
||||||
@@ -993,6 +1261,8 @@ def parse_color_grade_config(config: dict) -> ColorGradeConfig:
|
|||||||
contrast=float(config.get("contrast", 1.0)),
|
contrast=float(config.get("contrast", 1.0)),
|
||||||
brightness=float(config.get("brightness", 0.0)),
|
brightness=float(config.get("brightness", 0.0)),
|
||||||
saturation=float(config.get("saturation", 1.0)),
|
saturation=float(config.get("saturation", 1.0)),
|
||||||
|
auto_levels=float(config.get("auto_levels", 0.0)),
|
||||||
|
yellow_tint=float(config.get("yellow_tint", 0.0)),
|
||||||
# Custom curves
|
# Custom curves
|
||||||
curves_r=config.get("curves_r", ""),
|
curves_r=config.get("curves_r", ""),
|
||||||
curves_g=config.get("curves_g", ""),
|
curves_g=config.get("curves_g", ""),
|
||||||
@@ -1060,6 +1330,14 @@ def build_gnommokey_filter(config: dict) -> str:
|
|||||||
scale_factor = gain * 2.5
|
scale_factor = gain * 2.5
|
||||||
key_expr = f"({key_expr})*{scale_factor:.3f}"
|
key_expr = f"({key_expr})*{scale_factor:.3f}"
|
||||||
|
|
||||||
|
# Shadow boost: amplify key signal for dark pixels so dark greens key out fully.
|
||||||
|
# shadow_factor = 1 - luma/255 (high for dark pixels, 0 for bright pixels)
|
||||||
|
# extra multiplier = 1 + shadow_boost * shadow_factor
|
||||||
|
if cfg.shadow_boost > 0:
|
||||||
|
luma_expr = f"(0.299*r(X,Y)+0.587*g(X,Y)+0.114*b(X,Y))"
|
||||||
|
shadow_factor = f"(1-{luma_expr}/255)"
|
||||||
|
key_expr = f"({key_expr})*(1+{cfg.shadow_boost:.3f}*{shadow_factor})"
|
||||||
|
|
||||||
# Apply clip_black and clip_white to compress the matte
|
# Apply clip_black and clip_white to compress the matte
|
||||||
# clip_black: key values below this become 0 (those pixels stay opaque)
|
# clip_black: key values below this become 0 (those pixels stay opaque)
|
||||||
# clip_white: key values above this become 255 (fully transparent)
|
# clip_white: key values above this become 255 (fully transparent)
|
||||||
@@ -1077,6 +1355,13 @@ def build_gnommokey_filter(config: dict) -> str:
|
|||||||
# Invert: high key value (green) = low alpha (transparent)
|
# Invert: high key value (green) = low alpha (transparent)
|
||||||
alpha_expr = f"255-{key_expr}"
|
alpha_expr = f"255-{key_expr}"
|
||||||
|
|
||||||
|
# Luminance protection: lock bright pixels to fully opaque so white objects
|
||||||
|
# (headphones, teeth) are never accidentally keyed or jitter.
|
||||||
|
# protect_luma=-1 disables this. Use ~220 for typical white protection.
|
||||||
|
if cfg.protect_luma >= 0:
|
||||||
|
luma_expr = f"(0.299*r(X,Y)+0.587*g(X,Y)+0.114*b(X,Y))"
|
||||||
|
alpha_expr = f"if(gt({luma_expr},{cfg.protect_luma}),255,{alpha_expr})"
|
||||||
|
|
||||||
# Build the geq filter for alpha (in RGBA mode)
|
# Build the geq filter for alpha (in RGBA mode)
|
||||||
parts.append(f"geq=r='r(X,Y)':g='g(X,Y)':b='b(X,Y)':a='{alpha_expr}'")
|
parts.append(f"geq=r='r(X,Y)':g='g(X,Y)':b='b(X,Y)':a='{alpha_expr}'")
|
||||||
|
|
||||||
@@ -1103,6 +1388,41 @@ def build_gnommokey_filter(config: dict) -> str:
|
|||||||
|
|
||||||
parts.append(f"geq=r='{new_r}':g='{new_g}':b='{new_b}':a='alpha(X,Y)'")
|
parts.append(f"geq=r='{new_r}':g='{new_g}':b='{new_b}':a='alpha(X,Y)'")
|
||||||
|
|
||||||
|
# Interior spill suppression: cap the spill channel across the WHOLE frame,
|
||||||
|
# including fully-opaque interior pixels the bias/edge despill can't reach
|
||||||
|
# (green bounce on skin, a bald head, etc.). Caps the channel at a reference
|
||||||
|
# extrapolated through the other two channels: max [t=0] -> average [t=1] ->
|
||||||
|
# min [t=2] (since r+b-max == min). Values >1 push green below the r/b
|
||||||
|
# average for heavy close-up spill. Green is only ever reduced, never
|
||||||
|
# boosted, so non-spilled pixels are untouched.
|
||||||
|
if cfg.spill_suppress > 0:
|
||||||
|
t = min(max(cfg.spill_suppress, 0.0), 2.0)
|
||||||
|
yp = min(max(cfg.yellow_protect, 0.0), 1.0)
|
||||||
|
|
||||||
|
def _protected(spill_ch: str, capped: str, warm_ch: str, cold_ch: str) -> str:
|
||||||
|
"""Blend the capped spill channel back toward its original value for
|
||||||
|
saturated hues that legitimately pair the spill channel with a warm
|
||||||
|
channel while the cold channel is low (e.g. yellow = green+red, no
|
||||||
|
blue). Skin/scalp keeps some cold channel, so it stays suppressed."""
|
||||||
|
if yp <= 0:
|
||||||
|
return capped
|
||||||
|
# "yellowness": high when spill & warm are both high but cold is low.
|
||||||
|
hue = f"max(0,min({spill_ch},{warm_ch})-{cold_ch})"
|
||||||
|
gate = f"clip(({hue}-40)/120,0,1)"
|
||||||
|
protect = f"({gate}*{yp:.3f})"
|
||||||
|
return f"({spill_ch}*{protect}+({capped})*(1-{protect}))"
|
||||||
|
|
||||||
|
if is_green_screen:
|
||||||
|
ref = f"((1-{t:.3f})*max(r(X,Y),b(X,Y))+{t:.3f}*(r(X,Y)+b(X,Y))/2)"
|
||||||
|
capped = f"min(g(X,Y),{ref})"
|
||||||
|
new_g = _protected("g(X,Y)", capped, "r(X,Y)", "b(X,Y)")
|
||||||
|
parts.append(f"geq=r='r(X,Y)':g='{new_g}':b='b(X,Y)':a='alpha(X,Y)'")
|
||||||
|
else:
|
||||||
|
ref = f"((1-{t:.3f})*max(r(X,Y),g(X,Y))+{t:.3f}*(r(X,Y)+g(X,Y))/2)"
|
||||||
|
capped = f"min(b(X,Y),{ref})"
|
||||||
|
new_b = _protected("b(X,Y)", capped, "r(X,Y)", "g(X,Y)")
|
||||||
|
parts.append(f"geq=r='r(X,Y)':g='g(X,Y)':b='{new_b}':a='alpha(X,Y)'")
|
||||||
|
|
||||||
# Edge-aware despill: aggressively suppress green at semi-transparent edges
|
# Edge-aware despill: aggressively suppress green at semi-transparent edges
|
||||||
# This targets the 2-4px green fringe that regular despill misses
|
# This targets the 2-4px green fringe that regular despill misses
|
||||||
# edge_factor is high (1.0) at alpha=128, low (0) at alpha=0 or 255
|
# edge_factor is high (1.0) at alpha=128, low (0) at alpha=0 or 255
|
||||||
@@ -1189,12 +1509,45 @@ def parse_gnommokey_config(config: dict) -> GnommoKeyConfig:
|
|||||||
clip_white=float(config.get("clip_white", 100.0)),
|
clip_white=float(config.get("clip_white", 100.0)),
|
||||||
despill_bias=despill_bias,
|
despill_bias=despill_bias,
|
||||||
despill_strength=float(config.get("despill_strength", 0.5)),
|
despill_strength=float(config.get("despill_strength", 0.5)),
|
||||||
|
spill_suppress=float(config.get("spill_suppress", 0.0)),
|
||||||
|
yellow_protect=float(config.get("yellow_protect", 0.0)),
|
||||||
alpha_bias=alpha_bias,
|
alpha_bias=alpha_bias,
|
||||||
|
protect_luma=int(config.get("protect_luma", -1)),
|
||||||
|
shadow_boost=float(config.get("shadow_boost", 0.0)),
|
||||||
edge_erode=int(config.get("edge_erode", 0)),
|
edge_erode=int(config.get("edge_erode", 0)),
|
||||||
edge_soften=float(config.get("edge_soften", 0.0)),
|
edge_soften=float(config.get("edge_soften", 0.0)),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _chunk_is_valid(chunk_path: Path, expected_dur: float) -> bool:
|
||||||
|
"""True if a chunk file exists and looks fully written (readable, ~right length).
|
||||||
|
|
||||||
|
Used to decide which chunks a resuming run can keep. A chunk that was mid-write
|
||||||
|
when the process died has no moov atom / a short duration, so ffprobe fails or
|
||||||
|
the duration is well under expected — either way it's rejected and redone.
|
||||||
|
"""
|
||||||
|
if not chunk_path.exists() or chunk_path.stat().st_size == 0:
|
||||||
|
return False
|
||||||
|
dur = get_video_duration(chunk_path) # 0.0 if unreadable
|
||||||
|
return dur > 0 and abs(dur - expected_dur) <= max(1.0, expected_dur * 0.05)
|
||||||
|
|
||||||
|
|
||||||
|
def _chunk_fingerprint(input_path: Path, filters: list[dict], take) -> str:
|
||||||
|
"""Identity of a chunk set: filters + source + chunk length. If any of these
|
||||||
|
change, previously written chunks are stale and must not be resumed."""
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
|
||||||
|
try:
|
||||||
|
st = input_path.stat()
|
||||||
|
src = f"{input_path.name}:{st.st_size}:{int(st.st_mtime)}"
|
||||||
|
except OSError:
|
||||||
|
src = input_path.name
|
||||||
|
payload = json.dumps(filters, sort_keys=True, default=str)
|
||||||
|
payload += f"|{src}|{CHUNK_DURATION}|{take}"
|
||||||
|
return hashlib.sha1(payload.encode()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
def apply_combined_video_filters_chunked(
|
def apply_combined_video_filters_chunked(
|
||||||
input_path: Path,
|
input_path: Path,
|
||||||
output_path: Path,
|
output_path: Path,
|
||||||
@@ -1228,6 +1581,22 @@ def apply_combined_video_filters_chunked(
|
|||||||
scratch_dir = output_path.parent / "chunks"
|
scratch_dir = output_path.parent / "chunks"
|
||||||
scratch_dir.mkdir(parents=True, exist_ok=True)
|
scratch_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# Resume guard: if the filters or the source changed since the leftover chunks
|
||||||
|
# were written, they're stale — drop them and start clean. Otherwise we keep
|
||||||
|
# whatever completed chunks are on disk and only redo the missing ones.
|
||||||
|
fingerprint = _chunk_fingerprint(input_path, filters, take)
|
||||||
|
manifest_path = scratch_dir / ".chunks.json"
|
||||||
|
prev_fp = None
|
||||||
|
if manifest_path.exists():
|
||||||
|
try:
|
||||||
|
prev_fp = json.loads(manifest_path.read_text()).get("fingerprint")
|
||||||
|
except (json.JSONDecodeError, OSError):
|
||||||
|
prev_fp = None
|
||||||
|
if prev_fp != fingerprint:
|
||||||
|
for old in scratch_dir.glob("chunk_*.mov"):
|
||||||
|
old.unlink(missing_ok=True)
|
||||||
|
manifest_path.write_text(json.dumps({"fingerprint": fingerprint}))
|
||||||
|
|
||||||
num_chunks = int(duration / CHUNK_DURATION) + 1
|
num_chunks = int(duration / CHUNK_DURATION) + 1
|
||||||
chunk_files: list[Path] = []
|
chunk_files: list[Path] = []
|
||||||
chunk_tasks: list[tuple] = [] # (index, chunk_path, start_time, chunk_duration)
|
chunk_tasks: list[tuple] = [] # (index, chunk_path, start_time, chunk_duration)
|
||||||
@@ -1244,9 +1613,28 @@ def apply_combined_video_filters_chunked(
|
|||||||
chunk_files.append(chunk_path)
|
chunk_files.append(chunk_path)
|
||||||
chunk_tasks.append((i, chunk_path, start_time, chunk_duration))
|
chunk_tasks.append((i, chunk_path, start_time, chunk_duration))
|
||||||
|
|
||||||
num_workers = min(DEFAULT_CHUNK_WORKERS, len(chunk_tasks))
|
# Resume: keep chunks a previous run already finished; only redo the rest.
|
||||||
|
pending_tasks: list[tuple] = []
|
||||||
|
for task in chunk_tasks:
|
||||||
|
i, chunk_path, start_time, chunk_dur = task
|
||||||
|
if _chunk_is_valid(chunk_path, chunk_dur):
|
||||||
|
continue
|
||||||
|
chunk_path.unlink(missing_ok=True) # drop a half-written chunk before redo
|
||||||
|
pending_tasks.append(task)
|
||||||
|
|
||||||
|
resumed = len(chunk_tasks) - len(pending_tasks)
|
||||||
|
if resumed and pending_tasks:
|
||||||
print(
|
print(
|
||||||
f" Processing {len(chunk_tasks)} chunks in parallel ({num_workers} workers)"
|
f" Resuming: {resumed}/{len(chunk_tasks)} chunk(s) already complete, "
|
||||||
|
f"processing the remaining {len(pending_tasks)}"
|
||||||
|
)
|
||||||
|
elif not pending_tasks:
|
||||||
|
print(f" All {len(chunk_tasks)} chunks already complete — concatenating")
|
||||||
|
|
||||||
|
num_workers = min(DEFAULT_CHUNK_WORKERS, len(pending_tasks)) if pending_tasks else 1
|
||||||
|
if pending_tasks and resumed == 0:
|
||||||
|
print(
|
||||||
|
f" Processing {len(pending_tasks)} chunks in parallel ({num_workers} workers)"
|
||||||
)
|
)
|
||||||
|
|
||||||
# Process chunks in parallel
|
# Process chunks in parallel
|
||||||
@@ -1263,10 +1651,10 @@ def apply_combined_video_filters_chunked(
|
|||||||
)
|
)
|
||||||
return i, chunk_path
|
return i, chunk_path
|
||||||
|
|
||||||
completed = 0
|
completed = resumed
|
||||||
with ThreadPoolExecutor(max_workers=num_workers) as executor:
|
with ThreadPoolExecutor(max_workers=num_workers) as executor:
|
||||||
futures = {
|
futures = {
|
||||||
executor.submit(process_chunk_task, task): task for task in chunk_tasks
|
executor.submit(process_chunk_task, task): task for task in pending_tasks
|
||||||
}
|
}
|
||||||
for future in as_completed(futures):
|
for future in as_completed(futures):
|
||||||
i, chunk_path = future.result()
|
i, chunk_path = future.result()
|
||||||
@@ -1306,11 +1694,13 @@ def apply_combined_video_filters_chunked(
|
|||||||
stderr=concat_result.stderr,
|
stderr=concat_result.stderr,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Clean up chunk files and concat list
|
# Clean up chunk files, concat list, and the resume manifest — only now that
|
||||||
|
# the batch output is safely produced.
|
||||||
for chunk_path in chunk_files:
|
for chunk_path in chunk_files:
|
||||||
if chunk_path.exists():
|
if chunk_path.exists():
|
||||||
chunk_path.unlink()
|
chunk_path.unlink()
|
||||||
concat_list.unlink(missing_ok=True)
|
concat_list.unlink(missing_ok=True)
|
||||||
|
manifest_path.unlink(missing_ok=True)
|
||||||
|
|
||||||
# Remove chunks directory if empty
|
# Remove chunks directory if empty
|
||||||
try:
|
try:
|
||||||
@@ -1407,7 +1797,11 @@ def _process_chunk_to_prores4444(
|
|||||||
else:
|
else:
|
||||||
cmd.append("-an")
|
cmd.append("-an")
|
||||||
|
|
||||||
cmd.append(str(output_path))
|
# Encode to a .partial sibling and atomically rename only after validation, so
|
||||||
|
# a chunk that appears at its final name is always complete. A crash mid-encode
|
||||||
|
# leaves the .partial behind (ignored by resume), never a half-written chunk.
|
||||||
|
partial = output_path.with_name(output_path.stem + ".partial" + output_path.suffix)
|
||||||
|
cmd.append(str(partial))
|
||||||
|
|
||||||
if verbose:
|
if verbose:
|
||||||
print(f" Filter: {video_filter}")
|
print(f" Filter: {video_filter}")
|
||||||
@@ -1416,6 +1810,7 @@ def _process_chunk_to_prores4444(
|
|||||||
result = run_ffmpeg_with_progress(cmd, actual_take or chunk_duration, "Encoding")
|
result = run_ffmpeg_with_progress(cmd, actual_take or chunk_duration, "Encoding")
|
||||||
|
|
||||||
if result.returncode != 0:
|
if result.returncode != 0:
|
||||||
|
partial.unlink(missing_ok=True)
|
||||||
raise PreprocessError(
|
raise PreprocessError(
|
||||||
"Chunk processing failed",
|
"Chunk processing failed",
|
||||||
filter_type="chunk",
|
filter_type="chunk",
|
||||||
@@ -1435,12 +1830,13 @@ def _process_chunk_to_prores4444(
|
|||||||
"format=duration",
|
"format=duration",
|
||||||
"-of",
|
"-of",
|
||||||
"csv=p=0",
|
"csv=p=0",
|
||||||
str(output_path),
|
str(partial),
|
||||||
],
|
],
|
||||||
capture_output=True,
|
capture_output=True,
|
||||||
text=True,
|
text=True,
|
||||||
)
|
)
|
||||||
if probe.returncode != 0 or not probe.stdout.strip():
|
if probe.returncode != 0 or not probe.stdout.strip():
|
||||||
|
partial.unlink(missing_ok=True)
|
||||||
raise PreprocessError(
|
raise PreprocessError(
|
||||||
f"Chunk output file is unreadable or missing moov atom: {output_path.name}",
|
f"Chunk output file is unreadable or missing moov atom: {output_path.name}",
|
||||||
filter_type="chunk",
|
filter_type="chunk",
|
||||||
@@ -1448,6 +1844,8 @@ def _process_chunk_to_prores4444(
|
|||||||
stderr=probe.stderr,
|
stderr=probe.stderr,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
os.replace(partial, output_path)
|
||||||
|
|
||||||
|
|
||||||
def _process_chunk_to_webm(
|
def _process_chunk_to_webm(
|
||||||
input_path: Path,
|
input_path: Path,
|
||||||
@@ -1954,7 +2352,12 @@ def apply_audio_normalize(
|
|||||||
channel_map -> eq_bands -> highpass -> lowpass -> room_eq -> dereverb -> denoise -> gate -> compress -> normalize
|
channel_map -> eq_bands -> highpass -> lowpass -> room_eq -> dereverb -> denoise -> gate -> compress -> normalize
|
||||||
"""
|
"""
|
||||||
cfg = parse_audio_normalize_config(config)
|
cfg = parse_audio_normalize_config(config)
|
||||||
|
if not cfg.enabled:
|
||||||
|
# No audio processing, just copy
|
||||||
|
import shutil
|
||||||
|
|
||||||
|
shutil.copy2(input_path, output_path)
|
||||||
|
return
|
||||||
# Build audio filter chain (order matters!)
|
# Build audio filter chain (order matters!)
|
||||||
audio_filters: list[str] = []
|
audio_filters: list[str] = []
|
||||||
|
|
||||||
@@ -2104,6 +2507,7 @@ def parse_audio_normalize_config(config: dict[str, Any]) -> AudioNormalizeConfig
|
|||||||
)
|
)
|
||||||
|
|
||||||
return AudioNormalizeConfig(
|
return AudioNormalizeConfig(
|
||||||
|
enabled=bool(config.get("enabled", True)),
|
||||||
# Parametric EQ
|
# Parametric EQ
|
||||||
eq_bands=eq_bands,
|
eq_bands=eq_bands,
|
||||||
# Room treatment
|
# Room treatment
|
||||||
@@ -2178,12 +2582,23 @@ def parse_chroma_key_config(config: dict[str, Any]) -> ChromaKeyConfig:
|
|||||||
|
|
||||||
def get_preprocessed_path(videos_dir: Path, video_source: VideoSource) -> Path:
|
def get_preprocessed_path(videos_dir: Path, video_source: VideoSource) -> Path:
|
||||||
"""
|
"""
|
||||||
Get the path to the preprocessed video file.
|
Get the file to feed into render for this segment.
|
||||||
|
|
||||||
Returns output_file if specified, otherwise returns source_file.
|
Prefers the preprocessed output (output_file / processed_file) once it has
|
||||||
|
actually been produced on disk, so a project can be rendered straight from
|
||||||
|
the raw source_file before the (heavy) preprocess stage has run. Falls back
|
||||||
|
to source_file whenever no processed output exists yet.
|
||||||
"""
|
"""
|
||||||
if video_source.output_file:
|
if video_source.output_file:
|
||||||
return videos_dir / video_source.output_file
|
processed = videos_dir / video_source.output_file
|
||||||
|
if processed.exists():
|
||||||
|
return processed
|
||||||
|
# preprocess may emit the compressed variant alongside the recorded name
|
||||||
|
variant = processed.with_suffix(
|
||||||
|
".webm" if processed.suffix.lower() == ".mov" else ".mov"
|
||||||
|
)
|
||||||
|
if variant.exists():
|
||||||
|
return variant
|
||||||
return videos_dir / video_source.source_file
|
return videos_dir / video_source.source_file
|
||||||
|
|
||||||
|
|
||||||
@@ -2203,256 +2618,3 @@ def needs_preprocessing(videos_dir: Path, video_source: VideoSource) -> bool:
|
|||||||
return True
|
return True
|
||||||
|
|
||||||
return True
|
return True
|
||||||
|
|
||||||
|
|
||||||
def stitch_narration_segments(
|
|
||||||
videos_dir: Path,
|
|
||||||
segment_ids: list[str],
|
|
||||||
videos: dict[str, VideoSource],
|
|
||||||
output_path: Path,
|
|
||||||
verbose: bool = False,
|
|
||||||
default_end_trim: float = 0.0,
|
|
||||||
loudnorm_config: Optional[dict] = None,
|
|
||||||
) -> Path:
|
|
||||||
"""
|
|
||||||
Stitch multiple narration video segments into a single file.
|
|
||||||
|
|
||||||
Each segment's skip and take values are applied to trim dead video at the
|
|
||||||
start/end of each recording. The segments are concatenated in the order
|
|
||||||
specified by segment_ids.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
videos_dir: Directory containing video files
|
|
||||||
segment_ids: Ordered list of video IDs from videos.json
|
|
||||||
videos: Dict of video ID -> VideoSource from videos.json
|
|
||||||
output_path: Path for the concatenated output file
|
|
||||||
verbose: Enable verbose output
|
|
||||||
default_end_trim: Seconds to trim from the end when no explicit end/take is set
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Path to the stitched video file.
|
|
||||||
"""
|
|
||||||
if len(segment_ids) == 1:
|
|
||||||
# Single segment - just return its processed path
|
|
||||||
video_source = videos[segment_ids[0]]
|
|
||||||
return get_preprocessed_path(videos_dir, video_source)
|
|
||||||
|
|
||||||
print(f" Concatenating {len(segment_ids)} narration segments...")
|
|
||||||
|
|
||||||
# Create temp directory for trimmed segments
|
|
||||||
temp_dir = output_path.parent / "concat_temp"
|
|
||||||
temp_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
trimmed_segments: list[Path] = []
|
|
||||||
|
|
||||||
for i, video_id in enumerate(segment_ids):
|
|
||||||
if video_id not in videos:
|
|
||||||
raise PreprocessError(
|
|
||||||
f"Narration segment '{video_id}' not found in videos.json",
|
|
||||||
filter_type=None,
|
|
||||||
)
|
|
||||||
|
|
||||||
video_source = videos[video_id]
|
|
||||||
source_path = get_preprocessed_path(videos_dir, video_source)
|
|
||||||
|
|
||||||
if not source_path.exists():
|
|
||||||
raise PreprocessError(
|
|
||||||
f"Narration segment not found: {source_path}",
|
|
||||||
filter_type=None,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Get segment duration
|
|
||||||
full_duration = get_video_duration(source_path)
|
|
||||||
skip = video_source.skip or 0.0
|
|
||||||
take = video_source.take
|
|
||||||
|
|
||||||
# Apply default end trim if no explicit take/end was set
|
|
||||||
if take is None and default_end_trim > 0:
|
|
||||||
take = max(0.0, full_duration - skip - default_end_trim)
|
|
||||||
|
|
||||||
# Calculate effective duration
|
|
||||||
if take is not None:
|
|
||||||
effective_duration = min(take, full_duration - skip)
|
|
||||||
else:
|
|
||||||
effective_duration = full_duration - skip
|
|
||||||
|
|
||||||
if verbose:
|
|
||||||
print(f" Segment {i+1}: {video_id}")
|
|
||||||
print(f" Source: {source_path.name}")
|
|
||||||
print(
|
|
||||||
f" Skip: {skip}s, Take: {take or 'all'}s, Duration: {effective_duration:.1f}s"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Always re-encode every segment to normalize fps and timestamps.
|
|
||||||
# Mixing un-normalized source files (e.g. 60fps camera) with
|
|
||||||
# trimmed-and-re-encoded 30fps segments causes cumulative A/V drift
|
|
||||||
# in the final concat.
|
|
||||||
|
|
||||||
# Trim/normalize the segment
|
|
||||||
trimmed_path = temp_dir / f"segment_{i:03d}.mov"
|
|
||||||
|
|
||||||
# Check if source has alpha channel (for ProRes 4444, etc.)
|
|
||||||
has_alpha = _video_has_alpha(source_path)
|
|
||||||
|
|
||||||
# Re-encode to normalize framerate and fix timestamps
|
|
||||||
# Different segments may have different framerates which breaks concatenation
|
|
||||||
cmd = ["ffmpeg", "-y"]
|
|
||||||
if skip > 0:
|
|
||||||
cmd.extend(["-ss", str(skip)])
|
|
||||||
cmd.extend(["-i", str(source_path)])
|
|
||||||
if take is not None:
|
|
||||||
cmd.extend(["-t", str(take)])
|
|
||||||
|
|
||||||
if has_alpha:
|
|
||||||
# Preserve alpha with ProRes 4444
|
|
||||||
cmd.extend(
|
|
||||||
[
|
|
||||||
"-vf",
|
|
||||||
"fps=30,format=yuva444p10le",
|
|
||||||
"-c:v",
|
|
||||||
"prores_ks",
|
|
||||||
"-profile:v",
|
|
||||||
"4",
|
|
||||||
"-pix_fmt",
|
|
||||||
"yuva444p10le",
|
|
||||||
"-c:a",
|
|
||||||
"pcm_s16le",
|
|
||||||
"-avoid_negative_ts",
|
|
||||||
"make_zero",
|
|
||||||
str(trimmed_path),
|
|
||||||
]
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
# No alpha - use fast h264 encoding
|
|
||||||
cmd.extend(
|
|
||||||
[
|
|
||||||
"-vf",
|
|
||||||
"fps=30",
|
|
||||||
"-c:v",
|
|
||||||
"libx264",
|
|
||||||
"-preset",
|
|
||||||
"fast",
|
|
||||||
"-crf",
|
|
||||||
"18",
|
|
||||||
"-c:a",
|
|
||||||
"aac",
|
|
||||||
"-b:a",
|
|
||||||
"192k",
|
|
||||||
"-avoid_negative_ts",
|
|
||||||
"make_zero",
|
|
||||||
"-movflags",
|
|
||||||
"+faststart",
|
|
||||||
str(trimmed_path),
|
|
||||||
]
|
|
||||||
)
|
|
||||||
|
|
||||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise PreprocessError(
|
|
||||||
f"Failed to trim segment {video_id}",
|
|
||||||
filter_type="concat",
|
|
||||||
command=" ".join(cmd),
|
|
||||||
stderr=result.stderr,
|
|
||||||
)
|
|
||||||
|
|
||||||
trimmed_segments.append(trimmed_path)
|
|
||||||
|
|
||||||
# Build concat file list
|
|
||||||
concat_list = temp_dir / "concat_list.txt"
|
|
||||||
with open(concat_list, "w", encoding="utf-8") as f:
|
|
||||||
for segment in trimmed_segments:
|
|
||||||
f.write(f"file '{segment.resolve()}'\n")
|
|
||||||
|
|
||||||
# Concatenate all segments
|
|
||||||
print(f" Stitching {len(trimmed_segments)} segments -> {output_path.name}")
|
|
||||||
|
|
||||||
cmd = [
|
|
||||||
"ffmpeg",
|
|
||||||
"-y",
|
|
||||||
"-f",
|
|
||||||
"concat",
|
|
||||||
"-safe",
|
|
||||||
"0",
|
|
||||||
"-i",
|
|
||||||
str(concat_list),
|
|
||||||
"-c:v",
|
|
||||||
"copy",
|
|
||||||
"-c:a",
|
|
||||||
"copy",
|
|
||||||
"-movflags",
|
|
||||||
"+faststart",
|
|
||||||
str(output_path),
|
|
||||||
]
|
|
||||||
|
|
||||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise PreprocessError(
|
|
||||||
"Segment concatenation failed",
|
|
||||||
filter_type="concat",
|
|
||||||
command=" ".join(cmd),
|
|
||||||
stderr=result.stderr,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Apply loudnorm if any segment had defer_loudnorm=True
|
|
||||||
needs_loudnorm = any(
|
|
||||||
videos[seg_id].defer_loudnorm for seg_id in segment_ids if seg_id in videos
|
|
||||||
)
|
|
||||||
if needs_loudnorm:
|
|
||||||
print(" Applying loudness normalization to stitched output...")
|
|
||||||
normalized_path = (
|
|
||||||
output_path.parent / f"{output_path.stem}_normalized{output_path.suffix}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Build loudnorm filter string from project config (or fall back to defaults)
|
|
||||||
_cfg = loudnorm_config or {}
|
|
||||||
_lufs = float(_cfg.get("target_lufs", -14))
|
|
||||||
_lra = float(_cfg.get("target_lra", 11))
|
|
||||||
_tp = float(_cfg.get("target_tp", -1.5))
|
|
||||||
loudnorm_filter = f"loudnorm=I={_lufs:.1f}:LRA={_lra:.1f}:TP={_tp:.1f}"
|
|
||||||
|
|
||||||
loudnorm_cmd = [
|
|
||||||
"ffmpeg",
|
|
||||||
"-y",
|
|
||||||
"-i",
|
|
||||||
str(output_path),
|
|
||||||
"-c:v",
|
|
||||||
"copy",
|
|
||||||
"-af",
|
|
||||||
loudnorm_filter,
|
|
||||||
"-c:a",
|
|
||||||
"aac",
|
|
||||||
"-b:a",
|
|
||||||
"192k",
|
|
||||||
"-movflags",
|
|
||||||
"+faststart",
|
|
||||||
str(normalized_path),
|
|
||||||
]
|
|
||||||
|
|
||||||
result = subprocess.run(loudnorm_cmd, capture_output=True, text=True)
|
|
||||||
if result.returncode != 0:
|
|
||||||
raise PreprocessError(
|
|
||||||
"Loudness normalization failed",
|
|
||||||
filter_type="loudnorm",
|
|
||||||
command=" ".join(loudnorm_cmd),
|
|
||||||
stderr=result.stderr,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Replace original with normalized version
|
|
||||||
output_path.unlink()
|
|
||||||
normalized_path.rename(output_path)
|
|
||||||
print(" Loudness normalization complete.")
|
|
||||||
|
|
||||||
# Clean up temp files
|
|
||||||
for segment in trimmed_segments:
|
|
||||||
if segment.parent == temp_dir and segment.exists():
|
|
||||||
segment.unlink()
|
|
||||||
concat_list.unlink()
|
|
||||||
try:
|
|
||||||
temp_dir.rmdir()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
total_duration = get_video_duration(output_path)
|
|
||||||
print(f" Stitched duration: {format_time(total_duration)}")
|
|
||||||
|
|
||||||
return output_path
|
|
||||||
|
|||||||
+29
-204
@@ -1,19 +1,18 @@
|
|||||||
"""Push project metadata to gnommoeditor (prod) or gnommoweb (local).
|
"""Push project metadata to gnommoweb (the review app).
|
||||||
|
|
||||||
Usage:
|
Usage:
|
||||||
gnommo push -p video1 # push parent video project
|
gnommo push -p video1 # push to local gnommoweb
|
||||||
|
gnommo push -p video1 --prod # push to production gnommoweb
|
||||||
gnommo push -p short_pixelated_universe # push a short project
|
gnommo push -p short_pixelated_universe # push a short project
|
||||||
gnommo push -p myproject --force # force push, overwrite server
|
gnommo push -p myproject --force # force push, overwrite server
|
||||||
|
|
||||||
Reads project.json and companion JSON files, then POSTs to:
|
POSTs the project metadata to gnommoweb's POST /api/projects/push.
|
||||||
Production: POST /api/ingest (gnommoeditor, uses INGEST_API_KEY)
|
|
||||||
Local: POST /api/projects/push (gnommoweb, uses GNOMMOWEB_API_KEY)
|
|
||||||
|
|
||||||
Configuration (from .env or environment):
|
Configuration (from .env or environment):
|
||||||
GNOMMOEDITOR_URL Base URL for production (e.g. https://editor.glitch.university)
|
|
||||||
INGEST_API_KEY Bearer token for gnommoeditor ingest endpoint
|
|
||||||
GNOMMOWEB_URL Base URL for local dev (e.g. http://localhost:3001)
|
GNOMMOWEB_URL Base URL for local dev (e.g. http://localhost:3001)
|
||||||
GNOMMOWEB_API_KEY Bearer token for local gnommoweb
|
GNOMMOWEB_API_KEY Bearer token for local (CONTENT_API_KEY)
|
||||||
|
GNOMMOWEB_PROD_URL Base URL for production (e.g. https://glitch.university)
|
||||||
|
GNOMMOWEB_PROD_API_KEY Bearer token for production (CONTENT_API_KEY)
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import json
|
import json
|
||||||
@@ -68,130 +67,6 @@ def _write_sync(project_path: Path, data: dict, prod: bool = False):
|
|||||||
json.dump(data, f, indent=2)
|
json.dump(data, f, indent=2)
|
||||||
|
|
||||||
|
|
||||||
def _load_json_file(path: Path, label: str, verbose: bool) -> dict | list | None:
|
|
||||||
"""Load a JSON file, returning None if it doesn't exist."""
|
|
||||||
if not path.exists():
|
|
||||||
if verbose:
|
|
||||||
print(f" {label}: not found at {path}")
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
with open(path) as f:
|
|
||||||
return json.load(f)
|
|
||||||
except json.JSONDecodeError as e:
|
|
||||||
print(f" Warning: could not parse {label} ({path}): {e}", file=sys.stderr)
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _load_text_file(path: Path, label: str) -> str | None:
|
|
||||||
"""Load a text file, returning None if it doesn't exist."""
|
|
||||||
if not path.exists():
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
return path.read_text(encoding="utf-8")
|
|
||||||
except UnicodeDecodeError:
|
|
||||||
return path.read_text(encoding="latin-1")
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_seconds(value) -> float | None:
|
|
||||||
"""Convert a time value like '30s', '1:30', or 30 into a plain float of seconds."""
|
|
||||||
if value is None:
|
|
||||||
return None
|
|
||||||
if isinstance(value, (int, float)):
|
|
||||||
return float(value)
|
|
||||||
value = str(value).strip()
|
|
||||||
if value.endswith("s"):
|
|
||||||
value = value[:-1]
|
|
||||||
if ":" in value:
|
|
||||||
parts = value.split(":")
|
|
||||||
if len(parts) == 2:
|
|
||||||
return float(parts[0]) * 60 + float(parts[1])
|
|
||||||
elif len(parts) == 3:
|
|
||||||
return float(parts[0]) * 3600 + float(parts[1]) * 60 + float(parts[2])
|
|
||||||
return float(value)
|
|
||||||
|
|
||||||
|
|
||||||
def _sanitize_time_fields(data: dict | None, fields: list[str]) -> dict | None:
|
|
||||||
"""Return a copy of dict with the given fields converted to plain floats."""
|
|
||||||
if not data:
|
|
||||||
return data
|
|
||||||
result = dict(data)
|
|
||||||
for field in fields:
|
|
||||||
if field in result and result[field] is not None:
|
|
||||||
try:
|
|
||||||
result[field] = _parse_seconds(result[field])
|
|
||||||
except (ValueError, TypeError):
|
|
||||||
pass # leave invalid values for the server to reject with a clear error
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def _build_ingest_payload(project: dict, project_path: Path, verbose: bool) -> dict:
|
|
||||||
"""Build the rich ingest payload for gnommoeditor POST /api/ingest."""
|
|
||||||
|
|
||||||
# ── slides ────────────────────────────────────────────────────────────────
|
|
||||||
slides_path_str = project.get("slides", "slides.json")
|
|
||||||
slides_path = project_path / slides_path_str
|
|
||||||
slides = _load_json_file(slides_path, "slides", verbose)
|
|
||||||
if slides and verbose:
|
|
||||||
print(f" slides: {len(slides)} entries")
|
|
||||||
|
|
||||||
# ── manuscript ────────────────────────────────────────────────────────────
|
|
||||||
manuscript_path_str = project.get("manuscript", "manuscript.txt")
|
|
||||||
manuscript_path = project_path / manuscript_path_str
|
|
||||||
manuscript = _load_text_file(manuscript_path, "manuscript")
|
|
||||||
if manuscript:
|
|
||||||
print(f" manuscript: {len(manuscript)} chars")
|
|
||||||
elif verbose:
|
|
||||||
print(f" manuscript: not found at {manuscript_path}")
|
|
||||||
|
|
||||||
# ── narration ─────────────────────────────────────────────────────────────
|
|
||||||
narration_path_str = project.get("narration", "narration.json")
|
|
||||||
narration_path = project_path / narration_path_str
|
|
||||||
narration = _load_json_file(narration_path, "narration", verbose)
|
|
||||||
|
|
||||||
# ── audio ─────────────────────────────────────────────────────────────────
|
|
||||||
audio_path_str = project.get("audio_tracks", "audio.json")
|
|
||||||
audio_path = project_path / audio_path_str
|
|
||||||
audio = _load_json_file(audio_path, "audio", verbose)
|
|
||||||
|
|
||||||
# ── videos ────────────────────────────────────────────────────────────────
|
|
||||||
videos_path_str = project.get("videos", "videos.json")
|
|
||||||
videos_path = project_path / videos_path_str
|
|
||||||
videos = _load_json_file(videos_path, "videos", verbose)
|
|
||||||
|
|
||||||
# ── citations ─────────────────────────────────────────────────────────────
|
|
||||||
citations_path = project_path / "citations.json"
|
|
||||||
citations = _load_json_file(citations_path, "citations", verbose)
|
|
||||||
|
|
||||||
# Sanitize time fields — convert "30s", "1:30" etc. to plain floats
|
|
||||||
_VIDEO_TIME_FIELDS = ["duration", "pause_narration", "skip", "take"]
|
|
||||||
_NARRATION_TIME_FIELDS = ["skip", "take"]
|
|
||||||
_AUDIO_TIME_FIELDS = ["overlap", "duration"]
|
|
||||||
|
|
||||||
if videos:
|
|
||||||
videos = {
|
|
||||||
k: _sanitize_time_fields(v, _VIDEO_TIME_FIELDS) for k, v in videos.items()
|
|
||||||
}
|
|
||||||
if narration:
|
|
||||||
narration = {
|
|
||||||
k: _sanitize_time_fields(v, _NARRATION_TIME_FIELDS)
|
|
||||||
for k, v in narration.items()
|
|
||||||
}
|
|
||||||
if audio:
|
|
||||||
audio = {
|
|
||||||
k: _sanitize_time_fields(v, _AUDIO_TIME_FIELDS) for k, v in audio.items()
|
|
||||||
}
|
|
||||||
|
|
||||||
return {
|
|
||||||
"project": project,
|
|
||||||
"slides": slides,
|
|
||||||
"manuscript": manuscript,
|
|
||||||
"narration": narration,
|
|
||||||
"audio": audio,
|
|
||||||
"videos": videos,
|
|
||||||
"citations": citations,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def cmd_push(
|
def cmd_push(
|
||||||
project_path: Path, verbose: bool = False, force: bool = False, prod: bool = False
|
project_path: Path, verbose: bool = False, force: bool = False, prod: bool = False
|
||||||
) -> int:
|
) -> int:
|
||||||
@@ -211,86 +86,34 @@ def cmd_push(
|
|||||||
print("Error: project.json must have 'id' and 'name' fields.", file=sys.stderr)
|
print("Error: project.json must have 'id' and 'name' fields.", file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
|
|
||||||
|
return _push_gnommoweb(project, project_path, verbose, force, prod)
|
||||||
|
|
||||||
|
|
||||||
|
# ── gnommoweb POST /api/projects/push ─────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
def _push_gnommoweb(
|
||||||
|
project: dict, project_path: Path, verbose: bool, force: bool, prod: bool
|
||||||
|
) -> int:
|
||||||
|
# --prod selects the production gnommoweb instance; without it, local dev.
|
||||||
if prod:
|
if prod:
|
||||||
return _push_prod(project, project_path, verbose)
|
api_url = os.environ.get("GNOMMOWEB_PROD_URL", "").rstrip("/")
|
||||||
|
api_key = os.environ.get("GNOMMOWEB_PROD_API_KEY", "")
|
||||||
|
url_var, key_var = "GNOMMOWEB_PROD_URL", "GNOMMOWEB_PROD_API_KEY"
|
||||||
else:
|
else:
|
||||||
return _push_local(project, project_path, verbose, force)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Production: gnommoeditor POST /api/ingest ─────────────────────────────────
|
|
||||||
|
|
||||||
|
|
||||||
def _push_prod(project: dict, project_path: Path, verbose: bool) -> int:
|
|
||||||
api_url = os.environ.get("GNOMMOEDITOR_URL", "").rstrip("/")
|
|
||||||
api_key = os.environ.get("INGEST_API_KEY", "")
|
|
||||||
if not api_url:
|
|
||||||
print("Error: GNOMMOEDITOR_URL is not set.", file=sys.stderr)
|
|
||||||
return 1
|
|
||||||
if not api_key:
|
|
||||||
print("Error: INGEST_API_KEY is not set.", file=sys.stderr)
|
|
||||||
return 1
|
|
||||||
|
|
||||||
project_id = project["id"]
|
|
||||||
payload = _build_ingest_payload(project, project_path, verbose)
|
|
||||||
|
|
||||||
# Attach sync state so the server can record it
|
|
||||||
sync = _read_sync(project_path, prod=True)
|
|
||||||
if sync:
|
|
||||||
payload["sync"] = sync
|
|
||||||
|
|
||||||
print(f" → {api_url}/api/ingest")
|
|
||||||
|
|
||||||
try:
|
|
||||||
r = requests.post(
|
|
||||||
f"{api_url}/api/ingest",
|
|
||||||
json=payload,
|
|
||||||
headers={"Authorization": f"Bearer {api_key}"},
|
|
||||||
timeout=30,
|
|
||||||
)
|
|
||||||
except requests.exceptions.ConnectionError:
|
|
||||||
print(f"✗ Could not connect to {api_url}")
|
|
||||||
return 1
|
|
||||||
|
|
||||||
if not r.ok:
|
|
||||||
try:
|
|
||||||
body = r.json()
|
|
||||||
except Exception:
|
|
||||||
body = r.text[:500]
|
|
||||||
print(f"✗ Server returned {r.status_code}: {body}")
|
|
||||||
return 1
|
|
||||||
|
|
||||||
result = r.json()
|
|
||||||
video_id = result.get("video_id")
|
|
||||||
slides_upserted = result.get("slides_upserted", 0)
|
|
||||||
|
|
||||||
# Update sync state
|
|
||||||
now_iso = datetime.now(tz=timezone.utc).isoformat(timespec="seconds")
|
|
||||||
existing_sync = _read_sync(project_path, prod=True)
|
|
||||||
_write_sync(
|
|
||||||
project_path,
|
|
||||||
{**existing_sync, "last_pushed_at": now_iso},
|
|
||||||
prod=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
print(f"✓ {project_id} → video #{video_id} ({slides_upserted} slides)")
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
# ── Local dev: gnommoweb POST /api/projects/push ──────────────────────────────
|
|
||||||
|
|
||||||
|
|
||||||
def _push_local(project: dict, project_path: Path, verbose: bool, force: bool) -> int:
|
|
||||||
api_url = os.environ.get("GNOMMOWEB_URL", "").rstrip("/")
|
api_url = os.environ.get("GNOMMOWEB_URL", "").rstrip("/")
|
||||||
api_key = os.environ.get("GNOMMOWEB_API_KEY", "")
|
api_key = os.environ.get("GNOMMOWEB_API_KEY", "")
|
||||||
|
url_var, key_var = "GNOMMOWEB_URL", "GNOMMOWEB_API_KEY"
|
||||||
if not api_url:
|
if not api_url:
|
||||||
print("Error: GNOMMOWEB_URL is not set.", file=sys.stderr)
|
print(f"Error: {url_var} is not set.", file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
if not api_key:
|
if not api_key:
|
||||||
print("Error: GNOMMOWEB_API_KEY is not set.", file=sys.stderr)
|
print(f"Error: {key_var} is not set.", file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
|
|
||||||
if verbose:
|
if verbose:
|
||||||
print(f" → local: {api_url}")
|
target = "production" if prod else "local"
|
||||||
|
print(f" → gnommoweb {target}: {api_url}")
|
||||||
|
|
||||||
project_id = project["id"]
|
project_id = project["id"]
|
||||||
parent_project = project.get("parent_project")
|
parent_project = project.get("parent_project")
|
||||||
@@ -327,7 +150,7 @@ def _push_local(project: dict, project_path: Path, verbose: bool, force: bool) -
|
|||||||
server_updated_at = result.get("server_updated_at")
|
server_updated_at = result.get("server_updated_at")
|
||||||
|
|
||||||
now_iso = datetime.now(tz=timezone.utc).isoformat(timespec="seconds")
|
now_iso = datetime.now(tz=timezone.utc).isoformat(timespec="seconds")
|
||||||
existing_sync = _read_sync(project_path, prod=False)
|
existing_sync = _read_sync(project_path, prod=prod)
|
||||||
_write_sync(
|
_write_sync(
|
||||||
project_path,
|
project_path,
|
||||||
{
|
{
|
||||||
@@ -335,7 +158,7 @@ def _push_local(project: dict, project_path: Path, verbose: bool, force: bool) -
|
|||||||
"last_pushed_at": now_iso,
|
"last_pushed_at": now_iso,
|
||||||
"server_updated_at": server_updated_at,
|
"server_updated_at": server_updated_at,
|
||||||
},
|
},
|
||||||
prod=False,
|
prod=prod,
|
||||||
)
|
)
|
||||||
|
|
||||||
asset = result.get("asset", {})
|
asset = result.get("asset", {})
|
||||||
@@ -376,6 +199,7 @@ def _build_parent_payload(project: dict, project_path: Path, verbose: bool) -> d
|
|||||||
"project_id": project["id"],
|
"project_id": project["id"],
|
||||||
"name": project["name"],
|
"name": project["name"],
|
||||||
"description": project.get("description"),
|
"description": project.get("description"),
|
||||||
|
"course": project.get("course"),
|
||||||
"coursecode": project.get("coursecode"),
|
"coursecode": project.get("coursecode"),
|
||||||
"script_content": script_content,
|
"script_content": script_content,
|
||||||
"resolution": project.get("resolution"),
|
"resolution": project.get("resolution"),
|
||||||
@@ -405,6 +229,7 @@ def _build_short_payload(project: dict, project_path: Path, verbose: bool) -> di
|
|||||||
"project_id": project["id"],
|
"project_id": project["id"],
|
||||||
"name": project["name"],
|
"name": project["name"],
|
||||||
"description": project.get("description"),
|
"description": project.get("description"),
|
||||||
|
"course": project.get("course"),
|
||||||
"parent_project": project["parent_project"],
|
"parent_project": project["parent_project"],
|
||||||
"hook": project.get("hook"),
|
"hook": project.get("hook"),
|
||||||
"script_content": script_content,
|
"script_content": script_content,
|
||||||
|
|||||||
+424
-110
@@ -159,7 +159,64 @@ def _build_crossfade_loop_filter(
|
|||||||
return filters
|
return filters
|
||||||
|
|
||||||
|
|
||||||
def render(plan: RenderPlan, output_path: Path, verbose: bool = False) -> None:
|
def _fmt_t(s: float) -> str:
|
||||||
|
s = max(0, int(s))
|
||||||
|
return f"{s // 60:02d}:{s % 60:02d}"
|
||||||
|
|
||||||
|
|
||||||
|
def _active_slide_at(plan: RenderPlan, t: float):
|
||||||
|
"""slide_id enabled at output time t (last slide whose window has opened)."""
|
||||||
|
cur = None
|
||||||
|
for e in plan.slide_events:
|
||||||
|
if e.start_time <= t < e.end_time:
|
||||||
|
return e.slide_id
|
||||||
|
if e.start_time <= t:
|
||||||
|
cur = e.slide_id
|
||||||
|
return cur
|
||||||
|
|
||||||
|
|
||||||
|
def _active_videos_at(plan: RenderPlan, t: float) -> list:
|
||||||
|
"""source_file of every video/outro event playing at output time t."""
|
||||||
|
names = []
|
||||||
|
for e in list(plan.video_events) + list(plan.outro_events):
|
||||||
|
if e.start_time <= t < e.end_time:
|
||||||
|
src = getattr(getattr(e, "video_source", None), "source_file", None)
|
||||||
|
names.append(src or "?")
|
||||||
|
return names
|
||||||
|
|
||||||
|
|
||||||
|
def _log_render_timeline(plan: RenderPlan, log) -> None:
|
||||||
|
"""Static output-time -> asset map (log only). Lets any render position — incl.
|
||||||
|
the last one before a hard kill — be mapped to a slide/asset after the fact."""
|
||||||
|
log("[timeline] video/outro assets by output time (RAM-relevant inputs):")
|
||||||
|
rows = [
|
||||||
|
(e.start_time, e.end_time, getattr(getattr(e, "video_source", None), "source_file", "?"))
|
||||||
|
for e in list(plan.video_events) + list(plan.outro_events)
|
||||||
|
]
|
||||||
|
for start, end, src in sorted(rows):
|
||||||
|
log(f" {_fmt_t(start)}-{_fmt_t(end)} {src}")
|
||||||
|
|
||||||
|
|
||||||
|
def _make_render_progress_hook(plan: RenderPlan, log):
|
||||||
|
from .preprocessor import _mem_snapshot
|
||||||
|
|
||||||
|
def hook(t: float) -> None:
|
||||||
|
mem = _mem_snapshot()
|
||||||
|
memstr = ""
|
||||||
|
if mem and mem[1]:
|
||||||
|
avail, total = mem
|
||||||
|
memstr = f"{100 * (total - avail) / total:.0f}% mem, {avail / 1e9:.1f}GB free | "
|
||||||
|
vids = _active_videos_at(plan, t)
|
||||||
|
vidstr = ", ".join(vids) if vids else "(none)"
|
||||||
|
log(
|
||||||
|
f"[render {_fmt_t(t)}/{_fmt_t(plan.total_duration)}] {memstr}"
|
||||||
|
f"slide={_active_slide_at(plan, t)} | videos: {vidstr}"
|
||||||
|
)
|
||||||
|
|
||||||
|
return hook
|
||||||
|
|
||||||
|
|
||||||
|
def render(plan: RenderPlan, output_path: Path, verbose: bool = False, log=None) -> None:
|
||||||
"""
|
"""
|
||||||
Render the final video using FFmpeg.
|
Render the final video using FFmpeg.
|
||||||
|
|
||||||
@@ -167,6 +224,11 @@ def render(plan: RenderPlan, output_path: Path, verbose: bool = False) -> None:
|
|||||||
1. Scales background video (if present) or creates solid color
|
1. Scales background video (if present) or creates solid color
|
||||||
2. Overlays talking head at configured position
|
2. Overlays talking head at configured position
|
||||||
3. Overlays slides at their configured positions with time-based enable
|
3. Overlays slides at their configured positions with time-based enable
|
||||||
|
|
||||||
|
`log`: optional callback (line -> None) writing to the render log only. When
|
||||||
|
given, a static timeline map is logged and the render position + active
|
||||||
|
slide/asset + RAM are logged ~every 5s, so a hard kill's last flushed line
|
||||||
|
pinpoints which asset the memory blew up on.
|
||||||
"""
|
"""
|
||||||
# Ensure output directory exists
|
# Ensure output directory exists
|
||||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
@@ -179,9 +241,17 @@ def render(plan: RenderPlan, output_path: Path, verbose: bool = False) -> None:
|
|||||||
print(" ".join(cmd))
|
print(" ".join(cmd))
|
||||||
print()
|
print()
|
||||||
|
|
||||||
|
hook = None
|
||||||
|
if log is not None:
|
||||||
|
try:
|
||||||
|
_log_render_timeline(plan, log)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
hook = _make_render_progress_hook(plan, log)
|
||||||
|
|
||||||
# Run with progress bar and ETA
|
# Run with progress bar and ETA
|
||||||
result = run_ffmpeg_with_progress(
|
result = run_ffmpeg_with_progress(
|
||||||
cmd, duration=plan.total_duration, description="Rendering"
|
cmd, duration=plan.total_duration, description="Rendering", progress_hook=hook
|
||||||
)
|
)
|
||||||
|
|
||||||
if result.returncode != 0:
|
if result.returncode != 0:
|
||||||
@@ -237,20 +307,45 @@ def _resolve_video_path(
|
|||||||
source_path = base_dir / video_source.source_file
|
source_path = base_dir / video_source.source_file
|
||||||
if project_path:
|
if project_path:
|
||||||
resolved, _ = resolve_with_cache(source_path, project_path)
|
resolved, _ = resolve_with_cache(source_path, project_path)
|
||||||
|
else:
|
||||||
|
resolved = source_path
|
||||||
|
|
||||||
|
if not resolved.exists():
|
||||||
|
# File not found anywhere — substitute PlaceholderVideo so FFmpeg doesn't crash
|
||||||
|
placeholder = None
|
||||||
|
if shared_assets_dir:
|
||||||
|
p = shared_assets_dir / "PlaceholderVideo.mp4"
|
||||||
|
if project_path:
|
||||||
|
p, _ = resolve_with_cache(p, project_path)
|
||||||
|
if p.exists():
|
||||||
|
placeholder = p
|
||||||
|
if placeholder:
|
||||||
|
import sys
|
||||||
|
print(
|
||||||
|
f" Warning: {video_source.source_file} not found — using PlaceholderVideo",
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
return placeholder
|
||||||
|
|
||||||
return resolved
|
return resolved
|
||||||
return source_path
|
|
||||||
|
|
||||||
|
|
||||||
def _has_audio_stream(video_path: Path) -> bool:
|
def _has_audio_stream(video_path: Path) -> bool:
|
||||||
"""Check if a video file contains a non-empty audio stream.
|
"""Check if a video file contains a non-empty, decodable audio stream.
|
||||||
|
|
||||||
Uses -analyzeduration 0 to avoid the slow avformat_find_stream_info() scan
|
Uses -analyzeduration 0 to avoid the slow avformat_find_stream_info() scan
|
||||||
that happens when an MP4 has a declared audio track with no actual frames —
|
that happens when an MP4 has a declared audio track with no actual frames —
|
||||||
ffprobe would otherwise scan the entire file looking for audio packets.
|
ffprobe would otherwise scan the entire file looking for audio packets.
|
||||||
|
|
||||||
Also checks nb_frames to reject ghost audio tracks (stream header exists in
|
Rejects:
|
||||||
the moov atom but no sample data in stsc/stsz).
|
- Ghost audio tracks (stream header exists but no sample data, nb_frames=0)
|
||||||
|
- Codecs FFmpeg cannot decode (e.g. 'apac' = Apple Packed Audio Codec used
|
||||||
|
on Apple Silicon Macs; causes "matches no streams" in the filtergraph)
|
||||||
"""
|
"""
|
||||||
|
# Codecs that are declared in the container but FFmpeg cannot decode.
|
||||||
|
# These appear as "Audio: none (apac / 0x63617061)" in FFmpeg output.
|
||||||
|
_UNSUPPORTED_CODECS = {"apac"}
|
||||||
|
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
[
|
[
|
||||||
"ffprobe",
|
"ffprobe",
|
||||||
@@ -263,7 +358,7 @@ def _has_audio_stream(video_path: Path) -> bool:
|
|||||||
"-select_streams",
|
"-select_streams",
|
||||||
"a:0",
|
"a:0",
|
||||||
"-show_entries",
|
"-show_entries",
|
||||||
"stream=index,nb_frames",
|
"stream=index,nb_frames,codec_name",
|
||||||
"-of",
|
"-of",
|
||||||
"csv=p=0",
|
"csv=p=0",
|
||||||
str(video_path),
|
str(video_path),
|
||||||
@@ -274,8 +369,12 @@ def _has_audio_stream(video_path: Path) -> bool:
|
|||||||
output = result.stdout.strip()
|
output = result.stdout.strip()
|
||||||
if not output:
|
if not output:
|
||||||
return False
|
return False
|
||||||
# output is "index" or "index,nb_frames"
|
# output is "index,nb_frames,codec_name" (or subset if fields are missing)
|
||||||
parts = output.split(",")
|
parts = output.split(",")
|
||||||
|
if len(parts) >= 3:
|
||||||
|
codec_name = parts[2].strip()
|
||||||
|
if codec_name in _UNSUPPORTED_CODECS:
|
||||||
|
return False # Codec declared but not decodable by FFmpeg
|
||||||
if len(parts) >= 2:
|
if len(parts) >= 2:
|
||||||
nb_frames = parts[1].strip()
|
nb_frames = parts[1].strip()
|
||||||
if nb_frames == "0":
|
if nb_frames == "0":
|
||||||
@@ -290,25 +389,68 @@ def _build_audio_channel_filter(use_audio_channels: str) -> str:
|
|||||||
use_audio_channels: "both", "left", or "right"
|
use_audio_channels: "both", "left", or "right"
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Filter string (e.g., "pan=mono|c0=c1") or empty string for "both"
|
Filter string, or empty string for "both".
|
||||||
|
|
||||||
|
Matches preprocess (apply_audio_normalize): a selected single channel is
|
||||||
|
duplicated to BOTH output channels (centered stereo), not collapsed to mono.
|
||||||
"""
|
"""
|
||||||
if use_audio_channels == "left":
|
if use_audio_channels == "left":
|
||||||
return "pan=mono|c0=c0"
|
return "pan=stereo|c0=c0|c1=c0"
|
||||||
elif use_audio_channels == "right":
|
elif use_audio_channels == "right":
|
||||||
return "pan=mono|c0=c1"
|
return "pan=stereo|c0=c1|c1=c1"
|
||||||
return "" # "both" - no filter needed
|
return "" # "both" - no filter needed
|
||||||
|
|
||||||
|
|
||||||
|
def _build_narration_concat_prefilter(seg_input_idxs: list[int], config) -> tuple:
|
||||||
|
"""Normalize + concatenate narration segment inputs into one narration stream.
|
||||||
|
|
||||||
|
Each segment is scaled to the project resolution with alpha-safe transparent
|
||||||
|
padding and setsar=1 (and its audio resampled to a canonical format) so that
|
||||||
|
mismatched source formats can't crash the concat. Returns
|
||||||
|
(video_label, audio_label, filter_lines).
|
||||||
|
"""
|
||||||
|
width, height = config.resolution
|
||||||
|
fps = config.fps
|
||||||
|
lines: list[str] = []
|
||||||
|
pairs: list[tuple[str, str]] = []
|
||||||
|
for j, idx in enumerate(seg_input_idxs):
|
||||||
|
v, a = f"ncv{j}", f"nca{j}"
|
||||||
|
lines.append(
|
||||||
|
f"[{idx}:v]fps={fps},setpts=PTS-STARTPTS,format=yuva444p10le,"
|
||||||
|
f"scale={width}:{height}:force_original_aspect_ratio=decrease,"
|
||||||
|
f"pad={width}:{height}:(ow-iw)/2:(oh-ih)/2:color=0x00000000,setsar=1[{v}]"
|
||||||
|
)
|
||||||
|
lines.append(
|
||||||
|
f"[{idx}:a]aresample=async=1,"
|
||||||
|
f"aformat=sample_rates=48000:channel_layouts=stereo,"
|
||||||
|
f"asetpts=PTS-STARTPTS[{a}]"
|
||||||
|
)
|
||||||
|
pairs.append((v, a))
|
||||||
|
joins = "".join(f"[{v}][{a}]" for v, a in pairs)
|
||||||
|
lines.append(f"{joins}concat=n={len(pairs)}:v=1:a=1[narrsrc_v][narrsrc_a]")
|
||||||
|
return "narrsrc_v", "narrsrc_a", lines
|
||||||
|
|
||||||
|
|
||||||
def build_ffmpeg_command(plan: RenderPlan, output_path: Path) -> list[str]:
|
def build_ffmpeg_command(plan: RenderPlan, output_path: Path) -> list[str]:
|
||||||
"""Build the complete FFmpeg command as a list of arguments."""
|
"""Build the complete FFmpeg command as a list of arguments."""
|
||||||
cmd = ["ffmpeg", "-y"] # -y to overwrite output
|
cmd = ["ffmpeg", "-y"] # -y to overwrite output
|
||||||
|
|
||||||
# Global thread limits before any -i. Without this, each format=rgba conversion
|
# Global thread limits before any -i. Without this, each format=rgba conversion
|
||||||
# in the filter graph (one per video layer) spawns one swscaler thread per CPU core,
|
# in the filter graph (one per video layer) spawns one swscaler thread per CPU core,
|
||||||
# causing OOM on Apple Silicon where av_cpu_count() returns 10-11.
|
# causing OOM on Apple Silicon where av_cpu_count() returns 10-11 — and OOM-killing
|
||||||
|
# the whole WSL2 VM on Windows (Wsl/Service/E_UNEXPECTED), since the VM's RAM is
|
||||||
|
# capped well below the host's.
|
||||||
|
#
|
||||||
|
# CRITICAL: this render uses -filter_complex, whose parallelism is governed by
|
||||||
|
# -filter_complex_threads. -filter_threads only applies to SIMPLE (-vf) graphs and
|
||||||
|
# is silently ignored here, so without -filter_complex_threads the graph runs one
|
||||||
|
# thread per core no matter what — the real cause of the render-stage memory blowup.
|
||||||
from .cache import get_ffmpeg_thread_count
|
from .cache import get_ffmpeg_thread_count
|
||||||
|
|
||||||
_tc = str(get_ffmpeg_thread_count())
|
_tc = str(get_ffmpeg_thread_count())
|
||||||
cmd.extend(["-threads", _tc, "-filter_threads", _tc])
|
cmd.extend(
|
||||||
|
["-threads", _tc, "-filter_threads", _tc, "-filter_complex_threads", _tc]
|
||||||
|
)
|
||||||
|
|
||||||
# Resolve paths to absolute
|
# Resolve paths to absolute
|
||||||
project_path = plan.project_path.resolve()
|
project_path = plan.project_path.resolve()
|
||||||
@@ -321,9 +463,30 @@ def build_ffmpeg_command(plan: RenderPlan, output_path: Path) -> list[str]:
|
|||||||
# Track input indices
|
# Track input indices
|
||||||
input_idx = 0
|
input_idx = 0
|
||||||
|
|
||||||
# Input: always_visible videos (like talking head)
|
# Input: narration (talking head).
|
||||||
# Add -ss seek BEFORE -i for skip parameter and/or partial rendering
|
# Concat mode: when plan.narration_segments is set, add each processed segment
|
||||||
|
# as its own input (trimmed by skip/take) and concatenate them in-graph into a
|
||||||
|
# single normalized narration stream. Otherwise, the single always-visible
|
||||||
|
# input path. Add -ss seek BEFORE -i for skip parameter and/or partial rendering.
|
||||||
always_visible_inputs: list[int] = []
|
always_visible_inputs: list[int] = []
|
||||||
|
narration_concat = None # (video_label, audio_label) when concat mode is active
|
||||||
|
if plan.narration_segments:
|
||||||
|
seg_input_idxs: list[int] = []
|
||||||
|
for seg in plan.narration_segments:
|
||||||
|
total_seek = (seg.skip or 0.0) + plan.input_seek_time
|
||||||
|
if total_seek > 0:
|
||||||
|
cmd.extend(["-ss", f"{total_seek:.3f}"])
|
||||||
|
if seg.take is not None:
|
||||||
|
cmd.extend(["-t", f"{seg.take:.3f}"])
|
||||||
|
cmd.extend(["-analyzeduration", "0", "-probesize", "1000"])
|
||||||
|
cmd.extend(["-i", str(seg.source_path)])
|
||||||
|
seg_input_idxs.append(input_idx)
|
||||||
|
input_idx += 1
|
||||||
|
_nv_label, _na_label, _narr_concat_lines = _build_narration_concat_prefilter(
|
||||||
|
seg_input_idxs, plan.config
|
||||||
|
)
|
||||||
|
narration_concat = (_nv_label, _na_label, _narr_concat_lines)
|
||||||
|
else:
|
||||||
for video_id, video_source, cutout in plan.narration_videos:
|
for video_id, video_source, cutout in plan.narration_videos:
|
||||||
video_path = _resolve_video_path(
|
video_path = _resolve_video_path(
|
||||||
videos_dir, video_source, shared_assets_dir, project_path
|
videos_dir, video_source, shared_assets_dir, project_path
|
||||||
@@ -361,15 +524,20 @@ def build_ffmpeg_command(plan: RenderPlan, output_path: Path) -> list[str]:
|
|||||||
f"Background handle '{bg_handle}' not found in shared_assets/videos.json"
|
f"Background handle '{bg_handle}' not found in shared_assets/videos.json"
|
||||||
)
|
)
|
||||||
bg_path = shared_assets_dir / bg_videos[bg_handle]["source_file"]
|
bg_path = shared_assets_dir / bg_videos[bg_handle]["source_file"]
|
||||||
|
bg_path, _ = resolve_with_cache(bg_path, plan.project_path)
|
||||||
if not bg_path.exists():
|
if not bg_path.exists():
|
||||||
raise RenderError(
|
raise RenderError(
|
||||||
f"Background file not found: {bg_path} (from handle '{bg_handle}')"
|
f"Background file not found: {bg_path} (from handle '{bg_handle}')"
|
||||||
)
|
)
|
||||||
image_extensions = {".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".webp"}
|
image_extensions = {".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".webp"}
|
||||||
bg_is_image = bg_path.suffix.lower() in image_extensions
|
bg_is_image = bg_path.suffix.lower() in image_extensions
|
||||||
# Loop background videos infinitely
|
# Loop background videos, but BOUND the loop to the output length. An
|
||||||
|
# unbounded -stream_loop -1 input can make ffmpeg read/buffer ahead without
|
||||||
|
# limit, which balloons memory on long renders — cap it with an input -t so
|
||||||
|
# it EOFs cleanly at total_duration.
|
||||||
if not bg_is_image:
|
if not bg_is_image:
|
||||||
cmd.extend(["-stream_loop", "-1"])
|
cmd.extend(["-stream_loop", "-1"])
|
||||||
|
cmd.extend(["-t", f"{plan.total_duration:.3f}"])
|
||||||
# Duration of background video is irrelevant (looped or image) — skip analysis
|
# Duration of background video is irrelevant (looped or image) — skip analysis
|
||||||
cmd.extend(["-analyzeduration", "0", "-probesize", "1000"])
|
cmd.extend(["-analyzeduration", "0", "-probesize", "1000"])
|
||||||
cmd.extend(["-i", str(bg_path)])
|
cmd.extend(["-i", str(bg_path)])
|
||||||
@@ -398,31 +566,71 @@ def build_ffmpeg_command(plan: RenderPlan, output_path: Path) -> list[str]:
|
|||||||
video_inputs: dict[int, int] = {} # event_index -> input_idx
|
video_inputs: dict[int, int] = {} # event_index -> input_idx
|
||||||
video_events_with_audio: set[int] = set() # event indices whose files have audio
|
video_events_with_audio: set[int] = set() # event indices whose files have audio
|
||||||
|
|
||||||
|
# Live-probe cache: avoids probing the same file twice within one render call.
|
||||||
|
# Always probe rather than trusting cached has_audio — stale values (e.g. from when a
|
||||||
|
# file was local but is now only on an external disk with different content) cause
|
||||||
|
# "[N:a] matches no streams" errors that crash the render.
|
||||||
|
_audio_probe_cache: dict[Path, bool] = {}
|
||||||
|
|
||||||
|
def _has_audio(video_source, path: Path) -> bool:
|
||||||
|
"""Trust the import-time `has_audio` in videos.json so we don't spawn an
|
||||||
|
ffprobe per video at render time — that per-file probing is what can hang or
|
||||||
|
crash the render rig on large/remote inputs. Only live-probe when the stored
|
||||||
|
value is absent (unimported entry). Keep videos.json fresh by re-running
|
||||||
|
import after swapping a file, so a stale value can't cause '[N:a] matches no
|
||||||
|
streams'."""
|
||||||
|
if video_source.has_audio is not None:
|
||||||
|
return bool(video_source.has_audio)
|
||||||
|
if path not in _audio_probe_cache:
|
||||||
|
_audio_probe_cache[path] = _has_audio_stream(path)
|
||||||
|
return _audio_probe_cache[path]
|
||||||
|
|
||||||
for i, event in enumerate(plan.video_events):
|
for i, event in enumerate(plan.video_events):
|
||||||
video_path = _resolve_video_path(
|
video_path = _resolve_video_path(
|
||||||
videos_dir, event.video_source, shared_assets_dir, project_path
|
videos_dir, event.video_source, shared_assets_dir, project_path
|
||||||
)
|
)
|
||||||
skip = event.video_source.skip or 0.0
|
# Chunking v2 (docs/chunking_v2.md): a clip that began before this chunk
|
||||||
|
# resumes mid-clip via skip_override. None today (v1) → the source's own skip.
|
||||||
|
skip = event.skip_override if getattr(event, "skip_override", None) is not None \
|
||||||
|
else (event.video_source.skip or 0.0)
|
||||||
|
|
||||||
|
# How long this clip needs to play in the output
|
||||||
|
clip_duration = event.end_time - event.start_time
|
||||||
|
if event.video_source.take is not None:
|
||||||
|
clip_duration = min(clip_duration, event.video_source.take)
|
||||||
|
|
||||||
|
# Loop the clip if the file is shorter than the display window.
|
||||||
|
# Don't loop pause-narration videos — they intentionally play once and stop.
|
||||||
|
needs_loop = False
|
||||||
|
if event.video_source.duration is not None and not event.video_source.pause_narration:
|
||||||
|
remaining = event.video_source.duration - skip
|
||||||
|
needs_loop = remaining < clip_duration - 0.1 # 0.1 s tolerance
|
||||||
|
|
||||||
|
# Audio presence from stored metadata (no ffprobe unless it's missing).
|
||||||
|
has_audio = _has_audio(event.video_source, video_path)
|
||||||
|
if has_audio:
|
||||||
|
video_events_with_audio.add(i)
|
||||||
|
|
||||||
|
if needs_loop:
|
||||||
|
cmd.extend(["-stream_loop", "-1"])
|
||||||
if skip > 0:
|
if skip > 0:
|
||||||
cmd.extend(["-ss", f"{skip:.3f}"])
|
cmd.extend(["-ss", f"{skip:.3f}"])
|
||||||
cmd.extend(["-analyzeduration", "0", "-probesize", "1000"])
|
# Use 1 MB probesize for files with audio so FFmpeg can locate the audio stream
|
||||||
# Use pre-probed duration to tell FFmpeg exactly how much to read,
|
# in the moov atom (large UHD files may have moov atoms > 1 KB). Keep 1000 for
|
||||||
# preventing scans of ghost audio tracks on empty MP4 audio streams.
|
# video-only files to skip ghost-track scanning without a performance cost.
|
||||||
if event.video_source.duration is not None:
|
probesize = "1000000" if has_audio else "1000"
|
||||||
|
cmd.extend(["-analyzeduration", "0", "-probesize", probesize])
|
||||||
|
# Use pre-probed duration (or loop-limited duration) to tell FFmpeg exactly
|
||||||
|
# how much to read, preventing scans of ghost audio tracks on empty streams.
|
||||||
|
if needs_loop:
|
||||||
|
cmd.extend(["-t", f"{clip_duration:.3f}"])
|
||||||
|
elif event.video_source.duration is not None:
|
||||||
remaining = event.video_source.duration - skip
|
remaining = event.video_source.duration - skip
|
||||||
if remaining > 0:
|
if remaining > 0:
|
||||||
cmd.extend(["-t", f"{remaining:.3f}"])
|
cmd.extend(["-t", f"{remaining:.3f}"])
|
||||||
cmd.extend(["-i", str(video_path)])
|
cmd.extend(["-i", str(video_path)])
|
||||||
video_inputs[i] = input_idx
|
video_inputs[i] = input_idx
|
||||||
input_idx += 1
|
input_idx += 1
|
||||||
has_audio = event.video_source.has_audio
|
|
||||||
if has_audio is None:
|
|
||||||
print(
|
|
||||||
f" Warning: no cached metadata for '{event.video_source.source_file}' — run 'gnommo import' to avoid slow probing"
|
|
||||||
)
|
|
||||||
has_audio = _has_audio_stream(video_path)
|
|
||||||
if has_audio:
|
|
||||||
video_events_with_audio.add(i)
|
|
||||||
|
|
||||||
# Input: outro videos (play after narration ends)
|
# Input: outro videos (play after narration ends)
|
||||||
outro_inputs: dict[int, int] = {} # event_index -> input_idx
|
outro_inputs: dict[int, int] = {} # event_index -> input_idx
|
||||||
@@ -433,9 +641,14 @@ def build_ffmpeg_command(plan: RenderPlan, output_path: Path) -> list[str]:
|
|||||||
videos_dir, event.video_source, shared_assets_dir, project_path
|
videos_dir, event.video_source, shared_assets_dir, project_path
|
||||||
)
|
)
|
||||||
skip = event.video_source.skip or 0.0
|
skip = event.video_source.skip or 0.0
|
||||||
|
has_audio = _has_audio(event.video_source, video_path)
|
||||||
|
if has_audio:
|
||||||
|
outro_events_with_audio.add(i)
|
||||||
|
|
||||||
if skip > 0:
|
if skip > 0:
|
||||||
cmd.extend(["-ss", f"{skip:.3f}"])
|
cmd.extend(["-ss", f"{skip:.3f}"])
|
||||||
cmd.extend(["-analyzeduration", "0", "-probesize", "1000"])
|
probesize = "1000000" if has_audio else "1000"
|
||||||
|
cmd.extend(["-analyzeduration", "0", "-probesize", probesize])
|
||||||
if event.video_source.duration is not None:
|
if event.video_source.duration is not None:
|
||||||
remaining = event.video_source.duration - skip
|
remaining = event.video_source.duration - skip
|
||||||
if remaining > 0:
|
if remaining > 0:
|
||||||
@@ -443,14 +656,6 @@ def build_ffmpeg_command(plan: RenderPlan, output_path: Path) -> list[str]:
|
|||||||
cmd.extend(["-i", str(video_path)])
|
cmd.extend(["-i", str(video_path)])
|
||||||
outro_inputs[i] = input_idx
|
outro_inputs[i] = input_idx
|
||||||
input_idx += 1
|
input_idx += 1
|
||||||
has_audio = event.video_source.has_audio
|
|
||||||
if has_audio is None:
|
|
||||||
print(
|
|
||||||
f" Warning: no cached metadata for '{event.video_source.source_file}' — run 'gnommo import' to avoid slow probing"
|
|
||||||
)
|
|
||||||
has_audio = _has_audio_stream(video_path)
|
|
||||||
if has_audio:
|
|
||||||
outro_events_with_audio.add(i)
|
|
||||||
|
|
||||||
# Track where audio inputs start
|
# Track where audio inputs start
|
||||||
num_inputs_before_audio = input_idx
|
num_inputs_before_audio = input_idx
|
||||||
@@ -463,7 +668,9 @@ def build_ffmpeg_command(plan: RenderPlan, output_path: Path) -> list[str]:
|
|||||||
for event in plan.audio_events:
|
for event in plan.audio_events:
|
||||||
if event.audio_id not in audio_inputs:
|
if event.audio_id not in audio_inputs:
|
||||||
if event.audio_def.is_shared and plan.shared_assets_dir:
|
if event.audio_def.is_shared and plan.shared_assets_dir:
|
||||||
audio_path = plan.shared_assets_dir / "media" / "audio" / event.audio_def.file
|
audio_path = (
|
||||||
|
plan.shared_assets_dir / "media" / "audio" / event.audio_def.file
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
audio_path = audio_dir / event.audio_def.file
|
audio_path = audio_dir / event.audio_def.file
|
||||||
audio_path, _ = resolve_with_cache(audio_path, project_path)
|
audio_path, _ = resolve_with_cache(audio_path, project_path)
|
||||||
@@ -502,6 +709,7 @@ def build_ffmpeg_command(plan: RenderPlan, output_path: Path) -> list[str]:
|
|||||||
video_events_with_audio,
|
video_events_with_audio,
|
||||||
outro_inputs,
|
outro_inputs,
|
||||||
outro_events_with_audio,
|
outro_events_with_audio,
|
||||||
|
narration_concat,
|
||||||
)
|
)
|
||||||
cmd.extend(["-filter_complex", filter_complex])
|
cmd.extend(["-filter_complex", filter_complex])
|
||||||
|
|
||||||
@@ -510,8 +718,9 @@ def build_ffmpeg_command(plan: RenderPlan, output_path: Path) -> list[str]:
|
|||||||
|
|
||||||
# Determine audio source
|
# Determine audio source
|
||||||
# Priority: [aout] from filter > triggered video > no audio
|
# Priority: [aout] from filter > triggered video > no audio
|
||||||
# Note: we always create [aout] when always_visible_inputs exists
|
# Note: [aout] is created whenever there's narration audio — either a single
|
||||||
if always_visible_inputs:
|
# always-visible input or the in-graph concatenated narration [narrsrc_a].
|
||||||
|
if always_visible_inputs or narration_concat:
|
||||||
cmd.extend(
|
cmd.extend(
|
||||||
["-map", "[aout]"]
|
["-map", "[aout]"]
|
||||||
) # Audio from filter (may be segmented or simple copy)
|
) # Audio from filter (may be segmented or simple copy)
|
||||||
@@ -552,8 +761,24 @@ def _calculate_cutout_position(
|
|||||||
) -> tuple[int, int, int, int]:
|
) -> tuple[int, int, int, int]:
|
||||||
"""Calculate pixel position, width, and height from cutout definition.
|
"""Calculate pixel position, width, and height from cutout definition.
|
||||||
|
|
||||||
Returns: (x, y, width, height)
|
Returns: (x, y, width, height) — top-left corner + size, in pixels.
|
||||||
"""
|
"""
|
||||||
|
# New center-based model: cx/cy are the center as fractions of frame W/H;
|
||||||
|
# width/height/margin are fractions of min(W,H) so a square stays square on any
|
||||||
|
# aspect ratio. margin insets each side.
|
||||||
|
if cutout.cx_percent is not None:
|
||||||
|
m = min(frame_width, frame_height)
|
||||||
|
margin_px = cutout.margin_percent * m
|
||||||
|
cut_width = int(round(cutout.width_percent * m - 2 * margin_px))
|
||||||
|
cut_height = int(round(cutout.height_percent * m - 2 * margin_px))
|
||||||
|
cut_width = max(0, cut_width)
|
||||||
|
cut_height = max(0, cut_height)
|
||||||
|
center_x = cutout.cx_percent * frame_width
|
||||||
|
center_y = cutout.cy_percent * frame_height
|
||||||
|
cut_x = int(round(center_x - cut_width / 2))
|
||||||
|
cut_y = int(round(center_y - cut_height / 2))
|
||||||
|
return cut_x, cut_y, cut_width, cut_height
|
||||||
|
|
||||||
# Calculate height
|
# Calculate height
|
||||||
if cutout.height >= 0:
|
if cutout.height >= 0:
|
||||||
cut_height = cutout.height
|
cut_height = cutout.height
|
||||||
@@ -808,6 +1033,7 @@ def build_filter_complex(
|
|||||||
video_events_with_audio: set[int] = None,
|
video_events_with_audio: set[int] = None,
|
||||||
outro_inputs: dict[int, int] = None, # outro event_index -> input_idx
|
outro_inputs: dict[int, int] = None, # outro event_index -> input_idx
|
||||||
outro_events_with_audio: set[int] = None,
|
outro_events_with_audio: set[int] = None,
|
||||||
|
narration_concat: tuple = None, # (video_label, audio_label, filter_lines) or None
|
||||||
) -> str:
|
) -> str:
|
||||||
"""
|
"""
|
||||||
Build the filter_complex string for FFmpeg.
|
Build the filter_complex string for FFmpeg.
|
||||||
@@ -827,6 +1053,14 @@ def build_filter_complex(
|
|||||||
width, height = plan.config.resolution
|
width, height = plan.config.resolution
|
||||||
filters: list[str] = []
|
filters: list[str] = []
|
||||||
|
|
||||||
|
# Concat mode: emit the narration segment normalize+concat prefilter up front.
|
||||||
|
# It produces [narrsrc_v]/[narrsrc_a], which the talking-head node and the
|
||||||
|
# main-audio path consume in place of a single narration input.
|
||||||
|
narr_v_label = narr_a_label = None
|
||||||
|
if narration_concat:
|
||||||
|
narr_v_label, narr_a_label, _concat_lines = narration_concat
|
||||||
|
filters.extend(_concat_lines)
|
||||||
|
|
||||||
# Create base layer (background)
|
# Create base layer (background)
|
||||||
if has_background:
|
if has_background:
|
||||||
if bg_is_image:
|
if bg_is_image:
|
||||||
@@ -878,12 +1112,115 @@ def build_filter_complex(
|
|||||||
enable_expr = f"between(t\\,{event.start_time:.3f}\\,{effective_end:.3f})"
|
enable_expr = f"between(t\\,{event.start_time:.3f}\\,{effective_end:.3f})"
|
||||||
filters.append(
|
filters.append(
|
||||||
f"[{current_label}][{video_label}]overlay="
|
f"[{current_label}][{video_label}]overlay="
|
||||||
f"x={cut_x}:y={cut_y}:enable={enable_expr}"
|
f"x={cut_x}:y={cut_y}:enable={enable_expr}:eof_action=pass"
|
||||||
f"[{next_label}]"
|
f"[{next_label}]"
|
||||||
)
|
)
|
||||||
current_label = next_label
|
current_label = next_label
|
||||||
|
|
||||||
# Layer 3: Slides (transparent in the talking-head cutout area)
|
# Layer 3: Talking head — above below-videos, but under slides so fullscreen slides cover it
|
||||||
|
for i, (video_id, video_source, cutout) in enumerate(plan.narration_videos):
|
||||||
|
# Concat mode feeds the talking head from the in-graph concatenated
|
||||||
|
# narration [narrsrc_v]; otherwise from the single narration input pad.
|
||||||
|
narr_src = f"[{narr_v_label}]" if narr_v_label else f"[{always_visible_inputs[i]}:v]"
|
||||||
|
cut_x, cut_y, cut_width, cut_height = _calculate_cutout_position(
|
||||||
|
cutout, width, height
|
||||||
|
)
|
||||||
|
|
||||||
|
zoom = video_source.zoom
|
||||||
|
zoomed_width = int(cut_width * zoom)
|
||||||
|
zoomed_height = int(cut_height * zoom)
|
||||||
|
|
||||||
|
if not plan.narration_pauses:
|
||||||
|
video_label = f"av{i}"
|
||||||
|
filters.append(
|
||||||
|
f"{narr_src}fps={plan.config.fps},setpts=PTS-STARTPTS,"
|
||||||
|
f"format=yuva444p10le,"
|
||||||
|
f"scale={zoomed_width}:{zoomed_height}:force_original_aspect_ratio=increase,"
|
||||||
|
f"crop={cut_width}:{cut_height}:(iw-{cut_width})/2:(ih-{cut_height})/2,"
|
||||||
|
f"format=rgba[{video_label}]"
|
||||||
|
)
|
||||||
|
|
||||||
|
next_label = f"avbase{i}"
|
||||||
|
filters.append(
|
||||||
|
f"[{current_label}][{video_label}]overlay=x={cut_x}:y={cut_y}[{next_label}]"
|
||||||
|
)
|
||||||
|
current_label = next_label
|
||||||
|
else:
|
||||||
|
# Freeze-splice: cut the narration video at each pause and re-time the
|
||||||
|
# remainder so it resumes exactly where it stopped (the cutscene owns
|
||||||
|
# the frame during the gap). narr_src may be the concat filter output
|
||||||
|
# [narrsrc_v], which is single-consumer, so split it into one branch
|
||||||
|
# per segment before trimming.
|
||||||
|
segments = _build_narration_segments(
|
||||||
|
plan.narration_pauses, plan.total_duration
|
||||||
|
)
|
||||||
|
split_labels = [f"nvsplit{i}_{k}" for k in range(len(segments))]
|
||||||
|
filters.append(
|
||||||
|
f"{narr_src}split={len(segments)}"
|
||||||
|
+ "".join(f"[{lbl}]" for lbl in split_labels)
|
||||||
|
)
|
||||||
|
|
||||||
|
for seg_idx, (src_start, src_end, out_start, out_end) in enumerate(
|
||||||
|
segments
|
||||||
|
):
|
||||||
|
seg_label = f"av{i}_seg{seg_idx}"
|
||||||
|
pts_offset = out_start
|
||||||
|
filters.append(
|
||||||
|
f"[{split_labels[seg_idx]}]trim={src_start:.3f}:{src_end:.3f},"
|
||||||
|
f"setpts=PTS-STARTPTS+{pts_offset:.3f}/TB,"
|
||||||
|
f"format=yuva444p10le,"
|
||||||
|
f"scale={zoomed_width}:{zoomed_height}:force_original_aspect_ratio=increase,"
|
||||||
|
f"crop={cut_width}:{cut_height}:(iw-{cut_width})/2:(ih-{cut_height})/2,"
|
||||||
|
f"format=rgba[{seg_label}]"
|
||||||
|
)
|
||||||
|
|
||||||
|
next_label = f"avbase{i}_seg{seg_idx}"
|
||||||
|
enable_expr = f"between(t\\,{out_start:.3f}\\,{out_end:.3f})"
|
||||||
|
filters.append(
|
||||||
|
f"[{current_label}][{seg_label}]overlay=x={cut_x}:y={cut_y}:"
|
||||||
|
f"enable={enable_expr}[{next_label}]"
|
||||||
|
)
|
||||||
|
current_label = next_label
|
||||||
|
|
||||||
|
# Layer 4: "mid" triggered videos (vfm/vsm) — above talking head, below slides
|
||||||
|
# Use case: content that should show through a slide's transparent "screen hole"
|
||||||
|
for i, event in enumerate(plan.video_events):
|
||||||
|
if event.layer != "mid":
|
||||||
|
continue
|
||||||
|
video_idx = video_inputs[i]
|
||||||
|
cut_x, cut_y, cut_width, cut_height = _calculate_cutout_position(
|
||||||
|
event.cutout, width, height
|
||||||
|
)
|
||||||
|
|
||||||
|
duration = event.end_time - event.start_time
|
||||||
|
if event.video_source.take is not None:
|
||||||
|
duration = min(duration, event.video_source.take)
|
||||||
|
effective_end = event.start_time + duration
|
||||||
|
|
||||||
|
zoom = event.video_source.zoom
|
||||||
|
zoomed_width = int(cut_width * zoom)
|
||||||
|
zoomed_height = int(cut_height * zoom)
|
||||||
|
|
||||||
|
video_label = f"tvm{i}"
|
||||||
|
start_pts = event.start_time
|
||||||
|
filters.append(
|
||||||
|
f"[{video_idx}:v]format=yuva444p10le,"
|
||||||
|
f"setpts=PTS-STARTPTS+{start_pts:.3f}/TB,"
|
||||||
|
f"scale={zoomed_width}:{zoomed_height}:force_original_aspect_ratio=increase,"
|
||||||
|
f"crop={cut_width}:{cut_height}:(iw-{cut_width})/2:(ih-{cut_height})/2,"
|
||||||
|
f"format=rgba[{video_label}]"
|
||||||
|
)
|
||||||
|
|
||||||
|
next_label = f"tvmbase{i}"
|
||||||
|
enable_expr = f"between(t\\,{event.start_time:.3f}\\,{effective_end:.3f})"
|
||||||
|
filters.append(
|
||||||
|
f"[{current_label}][{video_label}]overlay="
|
||||||
|
f"x={cut_x}:y={cut_y}:enable={enable_expr}:eof_action=pass"
|
||||||
|
f"[{next_label}]"
|
||||||
|
)
|
||||||
|
current_label = next_label
|
||||||
|
|
||||||
|
# Layer 5: Slides — on top of talking head so fullscreen slides cover the narrator
|
||||||
for i, event in enumerate(plan.slide_events):
|
for i, event in enumerate(plan.slide_events):
|
||||||
slide_idx = slide_inputs[event.slide_id]
|
slide_idx = slide_inputs[event.slide_id]
|
||||||
|
|
||||||
@@ -902,58 +1239,7 @@ def build_filter_complex(
|
|||||||
)
|
)
|
||||||
current_label = next_label
|
current_label = next_label
|
||||||
|
|
||||||
# Layer 4: Always-visible videos (talking head) — above slides, visible through cutout
|
# Layer 6: "above" triggered videos (vft/vf2t/vst) — topmost, covers slides and talking head
|
||||||
for i, (video_id, video_source, cutout) in enumerate(plan.narration_videos):
|
|
||||||
input_idx = always_visible_inputs[i]
|
|
||||||
cut_x, cut_y, cut_width, cut_height = _calculate_cutout_position(
|
|
||||||
cutout, width, height
|
|
||||||
)
|
|
||||||
|
|
||||||
zoom = video_source.zoom
|
|
||||||
zoomed_width = int(cut_width * zoom)
|
|
||||||
zoomed_height = int(cut_height * zoom)
|
|
||||||
|
|
||||||
if not plan.narration_pauses:
|
|
||||||
video_label = f"av{i}"
|
|
||||||
filters.append(
|
|
||||||
f"[{input_idx}:v]fps={plan.config.fps},setpts=PTS-STARTPTS,"
|
|
||||||
f"format=yuva444p10le,"
|
|
||||||
f"scale={zoomed_width}:{zoomed_height}:force_original_aspect_ratio=increase,"
|
|
||||||
f"crop={cut_width}:{cut_height}:(iw-{cut_width})/2:(ih-{cut_height})/2,"
|
|
||||||
f"format=rgba[{video_label}]"
|
|
||||||
)
|
|
||||||
|
|
||||||
next_label = f"avbase{i}"
|
|
||||||
filters.append(
|
|
||||||
f"[{current_label}][{video_label}]overlay=x={cut_x}:y={cut_y}[{next_label}]"
|
|
||||||
)
|
|
||||||
current_label = next_label
|
|
||||||
else:
|
|
||||||
segments = _build_narration_segments(
|
|
||||||
plan.narration_pauses, plan.total_duration
|
|
||||||
)
|
|
||||||
|
|
||||||
for seg_idx, (src_start, src_end, out_start, out_end) in enumerate(segments):
|
|
||||||
seg_label = f"av{i}_seg{seg_idx}"
|
|
||||||
pts_offset = out_start
|
|
||||||
filters.append(
|
|
||||||
f"[{input_idx}:v]trim={src_start:.3f}:{src_end:.3f},"
|
|
||||||
f"setpts=PTS-STARTPTS+{pts_offset:.3f}/TB,"
|
|
||||||
f"format=yuva444p10le,"
|
|
||||||
f"scale={zoomed_width}:{zoomed_height}:force_original_aspect_ratio=increase,"
|
|
||||||
f"crop={cut_width}:{cut_height}:(iw-{cut_width})/2:(ih-{cut_height})/2,"
|
|
||||||
f"format=rgba[{seg_label}]"
|
|
||||||
)
|
|
||||||
|
|
||||||
next_label = f"avbase{i}_seg{seg_idx}"
|
|
||||||
enable_expr = f"between(t\\,{out_start:.3f}\\,{out_end:.3f})"
|
|
||||||
filters.append(
|
|
||||||
f"[{current_label}][{seg_label}]overlay=x={cut_x}:y={cut_y}:"
|
|
||||||
f"enable={enable_expr}[{next_label}]"
|
|
||||||
)
|
|
||||||
current_label = next_label
|
|
||||||
|
|
||||||
# Layer 5: "above" triggered videos (vft/vf2t/vst) — topmost, covers slides and talking head
|
|
||||||
# Use case: fullscreen video that intentionally masks the narrator
|
# Use case: fullscreen video that intentionally masks the narrator
|
||||||
for i, event in enumerate(plan.video_events):
|
for i, event in enumerate(plan.video_events):
|
||||||
if event.layer != "above":
|
if event.layer != "above":
|
||||||
@@ -986,7 +1272,7 @@ def build_filter_complex(
|
|||||||
enable_expr = f"between(t\\,{event.start_time:.3f}\\,{effective_end:.3f})"
|
enable_expr = f"between(t\\,{event.start_time:.3f}\\,{effective_end:.3f})"
|
||||||
filters.append(
|
filters.append(
|
||||||
f"[{current_label}][{video_label}]overlay="
|
f"[{current_label}][{video_label}]overlay="
|
||||||
f"x={cut_x}:y={cut_y}:enable={enable_expr}:format=auto"
|
f"x={cut_x}:y={cut_y}:enable={enable_expr}:format=auto:eof_action=pass"
|
||||||
f"[{next_label}]"
|
f"[{next_label}]"
|
||||||
)
|
)
|
||||||
current_label = next_label
|
current_label = next_label
|
||||||
@@ -1073,8 +1359,10 @@ def build_filter_complex(
|
|||||||
filters.append(f"[{current_label}]copy[vout]")
|
filters.append(f"[{current_label}]copy[vout]")
|
||||||
|
|
||||||
# Audio mixing: combine main audio with sound effects
|
# Audio mixing: combine main audio with sound effects
|
||||||
if always_visible_inputs:
|
if always_visible_inputs or narration_concat:
|
||||||
main_audio_idx = always_visible_inputs[0]
|
# In concat mode the main narration audio is the in-graph [narrsrc_a];
|
||||||
|
# otherwise it's the first always-visible input's audio pad.
|
||||||
|
_main_aud = f"[{narr_a_label}]" if narr_a_label else f"[{always_visible_inputs[0]}:a]"
|
||||||
audio_labels_to_mix = []
|
audio_labels_to_mix = []
|
||||||
|
|
||||||
# Get audio channel setting and volume from first narration video
|
# Get audio channel setting and volume from first narration video
|
||||||
@@ -1084,8 +1372,18 @@ def build_filter_complex(
|
|||||||
_, first_video_source, _ = plan.narration_videos[0]
|
_, first_video_source, _ = plan.narration_videos[0]
|
||||||
use_channels = first_video_source.use_audio_channels
|
use_channels = first_video_source.use_audio_channels
|
||||||
if use_channels == "auto":
|
if use_channels == "auto":
|
||||||
|
# Concat mode = pre-processed segments: preprocess already resolved
|
||||||
|
# the channel and baked it into the audio, so play it as-is ("both").
|
||||||
|
# Re-probing here means a full-file volumedetect pass on the narration
|
||||||
|
# (tens of seconds on a long take) for no benefit — that was the render
|
||||||
|
# command taking ~30s to build.
|
||||||
|
if narration_concat and plan.narration_segments:
|
||||||
|
use_channels = "both"
|
||||||
|
else:
|
||||||
|
# Legacy single-input narration (not preprocessed): resolve once.
|
||||||
narration_path = _resolve_video_path(
|
narration_path = _resolve_video_path(
|
||||||
videos_dir, first_video_source, shared_assets_dir, project_path
|
plan.videos_dir, first_video_source,
|
||||||
|
plan.shared_assets_dir, plan.project_path,
|
||||||
)
|
)
|
||||||
use_channels = _resolve_auto_channel(narration_path)
|
use_channels = _resolve_auto_channel(narration_path)
|
||||||
channel_filter = _build_audio_channel_filter(use_channels)
|
channel_filter = _build_audio_channel_filter(use_channels)
|
||||||
@@ -1114,19 +1412,28 @@ def build_filter_complex(
|
|||||||
filter_parts.append(f"atrim=0:{audio_end_time:.3f}")
|
filter_parts.append(f"atrim=0:{audio_end_time:.3f}")
|
||||||
filter_parts.append("asetpts=PTS-STARTPTS")
|
filter_parts.append("asetpts=PTS-STARTPTS")
|
||||||
filters.append(
|
filters.append(
|
||||||
f"[{main_audio_idx}:a]{','.join(filter_parts)}[main_aud]"
|
f"{_main_aud}{','.join(filter_parts)}[main_aud]"
|
||||||
)
|
)
|
||||||
audio_labels_to_mix.append("[main_aud]")
|
audio_labels_to_mix.append("[main_aud]")
|
||||||
elif filter_parts:
|
elif filter_parts:
|
||||||
filters.append(
|
filters.append(
|
||||||
f"[{main_audio_idx}:a]{','.join(filter_parts)}[main_aud]"
|
f"{_main_aud}{','.join(filter_parts)}[main_aud]"
|
||||||
)
|
)
|
||||||
audio_labels_to_mix.append("[main_aud]")
|
audio_labels_to_mix.append("[main_aud]")
|
||||||
else:
|
else:
|
||||||
audio_labels_to_mix.append(f"[{main_audio_idx}:a]")
|
audio_labels_to_mix.append(f"{_main_aud}")
|
||||||
else:
|
else:
|
||||||
# Complex case: segment the narration audio for pauses
|
# Freeze-splice: cut the narration audio at each pause and delay the
|
||||||
|
# remainder so it resumes on the exact sample it stopped on (the
|
||||||
|
# cutscene's own audio fills the gap). _main_aud may be the concat
|
||||||
|
# output [narrsrc_a], which is single-consumer, so asplit it into one
|
||||||
|
# branch per segment first.
|
||||||
segments = _build_narration_segments(plan.narration_pauses, audio_end_time)
|
segments = _build_narration_segments(plan.narration_pauses, audio_end_time)
|
||||||
|
asplit_labels = [f"narr_aud_split{k}" for k in range(len(segments))]
|
||||||
|
filters.append(
|
||||||
|
f"{_main_aud}asplit={len(segments)}"
|
||||||
|
+ "".join(f"[{lbl}]" for lbl in asplit_labels)
|
||||||
|
)
|
||||||
for seg_idx, (src_start, src_end, out_start, out_end) in enumerate(
|
for seg_idx, (src_start, src_end, out_start, out_end) in enumerate(
|
||||||
segments
|
segments
|
||||||
):
|
):
|
||||||
@@ -1143,7 +1450,7 @@ def build_filter_complex(
|
|||||||
if volume_filter:
|
if volume_filter:
|
||||||
filter_parts.append(volume_filter)
|
filter_parts.append(volume_filter)
|
||||||
filters.append(
|
filters.append(
|
||||||
f"[{main_audio_idx}:a]{','.join(filter_parts)}[{seg_label}]"
|
f"[{asplit_labels[seg_idx]}]{','.join(filter_parts)}[{seg_label}]"
|
||||||
)
|
)
|
||||||
audio_labels_to_mix.append(f"[{seg_label}]")
|
audio_labels_to_mix.append(f"[{seg_label}]")
|
||||||
|
|
||||||
@@ -1166,7 +1473,9 @@ def build_filter_complex(
|
|||||||
for p in plan.narration_pauses
|
for p in plan.narration_pauses
|
||||||
if p.output_time > event.start_time
|
if p.output_time > event.start_time
|
||||||
]
|
]
|
||||||
src_pos = 0.0
|
# Chunking v2: start partway into the looped stream when the
|
||||||
|
# clip began in an earlier chunk (docs/chunking_v2.md).
|
||||||
|
src_pos = getattr(event, "src_offset", 0.0)
|
||||||
seg_start = event.start_time
|
seg_start = event.start_time
|
||||||
seg_count = 0
|
seg_count = 0
|
||||||
|
|
||||||
@@ -1223,21 +1532,26 @@ def build_filter_complex(
|
|||||||
)
|
)
|
||||||
filters.extend(crossfade_filters)
|
filters.extend(crossfade_filters)
|
||||||
else:
|
else:
|
||||||
# Standard loop without crossfade
|
# Standard loop without crossfade. Chunking v2: seek to
|
||||||
|
# the loop phase when the clip began in an earlier chunk.
|
||||||
|
_off = getattr(event, "src_offset", 0.0)
|
||||||
filters.append(
|
filters.append(
|
||||||
f"[{audio_idx}:a]aloop=loop=-1:size=2e+09,"
|
f"[{audio_idx}:a]aloop=loop=-1:size=2e+09,"
|
||||||
f"atrim=0:{remaining:.3f},"
|
f"atrim={_off:.3f}:{_off + remaining:.3f},"
|
||||||
f"asetpts=PTS-STARTPTS,"
|
f"asetpts=PTS-STARTPTS,"
|
||||||
f"adelay={delay_ms}|{delay_ms},"
|
f"adelay={delay_ms}|{delay_ms},"
|
||||||
f"volume={volume:.2f}[{label}]"
|
f"volume={volume:.2f}[{label}]"
|
||||||
)
|
)
|
||||||
audio_labels_to_mix.append(f"[{label}]")
|
audio_labels_to_mix.append(f"[{label}]")
|
||||||
else:
|
else:
|
||||||
# One-shot audio: delay to trigger time
|
# One-shot audio: delay to trigger time. Chunking v2: seek in if
|
||||||
|
# the clip began in an earlier chunk (docs/chunking_v2.md).
|
||||||
label = f"aud{i}"
|
label = f"aud{i}"
|
||||||
delay_ms = int(event.start_time * 1000)
|
delay_ms = int(event.start_time * 1000)
|
||||||
|
_off = getattr(event, "src_offset", 0.0)
|
||||||
|
_seek = f"atrim={_off:.3f},asetpts=PTS-STARTPTS," if _off > 0 else ""
|
||||||
filters.append(
|
filters.append(
|
||||||
f"[{audio_idx}:a]adelay={delay_ms}|{delay_ms},volume={volume:.2f}[{label}]"
|
f"[{audio_idx}:a]{_seek}adelay={delay_ms}|{delay_ms},volume={volume:.2f}[{label}]"
|
||||||
)
|
)
|
||||||
audio_labels_to_mix.append(f"[{label}]")
|
audio_labels_to_mix.append(f"[{label}]")
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,493 @@
|
|||||||
|
"""events.json + scaffold.json — the timing layer between alignment and render.
|
||||||
|
|
||||||
|
The render stage is split in two:
|
||||||
|
|
||||||
|
build aligns manuscript markers to the transcript, then writes
|
||||||
|
events.json — one editable entry per marker: {type, id, time, mapping}
|
||||||
|
scaffold.json — the compiled timeline (transcription + narration
|
||||||
|
schedule + merged events + total_duration)
|
||||||
|
render loads the scaffold's event times (honouring any hand-edits to
|
||||||
|
events.json) and renders WITHOUT re-aligning.
|
||||||
|
|
||||||
|
events.json is the authoritative, GUI-editable seam: a future gnommoweb
|
||||||
|
review-stage timeline can read it, let the user drag slide-marker / video-start
|
||||||
|
events around, and write it straight back. Because it stores absolute
|
||||||
|
final-timeline times with stable ids, edits always win over the fuzzy aligner.
|
||||||
|
|
||||||
|
mapping values:
|
||||||
|
exact a confident fuzzy match to the spoken words
|
||||||
|
interpolated placed by the aligner's repair/fallback path, or filled in
|
||||||
|
between neighbours because the phrase wasn't found at all
|
||||||
|
manual set by a human / the GUI; never recomputed by build
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
from .models import CAMERA_PRESETS
|
||||||
|
from .transformer import MarkerTiming, resolve_video_presentation
|
||||||
|
|
||||||
|
# Per-occurrence presentation fields materialized onto video events (atomic events.json,
|
||||||
|
# GUI-ready). Round-tripped as overrides so a stored value drives render over the default.
|
||||||
|
_PRESENTATION_KEYS = ("cutout", "layer", "end_on", "take")
|
||||||
|
|
||||||
|
EVENTS_FILE = "events.json"
|
||||||
|
SCAFFOLD_FILE = "scaffold.json"
|
||||||
|
TRANSCRIBED_FILE = "manuscript_transcribed.txt"
|
||||||
|
|
||||||
|
MAPPING_EXACT = "exact"
|
||||||
|
MAPPING_INTERPOLATED = "interpolated"
|
||||||
|
MAPPING_MANUAL = "manual"
|
||||||
|
|
||||||
|
# Confidence at/above this counts as an exact match (mirrors the aligner default).
|
||||||
|
_EXACT_THRESHOLD = 0.6
|
||||||
|
|
||||||
|
|
||||||
|
# ── marker classification ─────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
# Shorthand prefixes that denote a video/narration trigger (mirrors transformer).
|
||||||
|
_VIDEO_MARKER_PREFIXES = (
|
||||||
|
"video:", "narration:",
|
||||||
|
"vft:", "vfb:", "vfm:", "vf2t:", "vf2b:", "vf2m:",
|
||||||
|
"vst:", "vsb:", "vsm:",
|
||||||
|
"vftp:", "vfbp:", "vfmp:", "vf2tp:", "vf2bp:", "vf2mp:",
|
||||||
|
"vstp:", "vsbp:", "vsmp:",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _ci_contains(d: dict, key: str) -> bool:
|
||||||
|
"""Case-insensitive membership: the render lowercases video/audio handles
|
||||||
|
(e.g. the marker vst:UnconstrainedLight resolves to videos.json's
|
||||||
|
unconstrainedlight), so classification must match case-insensitively too."""
|
||||||
|
if not d:
|
||||||
|
return False
|
||||||
|
if key in d:
|
||||||
|
return True
|
||||||
|
lk = key.lower()
|
||||||
|
return any(k.lower() == lk for k in d)
|
||||||
|
|
||||||
|
|
||||||
|
def marker_type(marker_id: str, slides: dict, videos: dict, audio: dict) -> str:
|
||||||
|
"""Classify a marker id as slide / video / audio / camera / other.
|
||||||
|
|
||||||
|
Prefix-aware and case-insensitive. Video markers carry a shorthand prefix
|
||||||
|
(vst:, vfb:, video:, …) and their handle is stored lowercased in videos.json,
|
||||||
|
so a marker like `vst:UnconstrainedLight` is a video even though videos.json
|
||||||
|
only has `unconstrainedlight`.
|
||||||
|
"""
|
||||||
|
if _ci_contains(slides, marker_id):
|
||||||
|
return "slide"
|
||||||
|
if marker_id.startswith(_VIDEO_MARKER_PREFIXES) or _ci_contains(videos, marker_id):
|
||||||
|
return "video"
|
||||||
|
if marker_id.startswith("audio:") or _ci_contains(audio, marker_id):
|
||||||
|
return "audio"
|
||||||
|
if marker_id.startswith("A") and len(marker_id) > 1:
|
||||||
|
aid = marker_id[1:]
|
||||||
|
if aid.isdigit() or _ci_contains(audio, aid):
|
||||||
|
return "audio"
|
||||||
|
if _ci_contains(CAMERA_PRESETS, marker_id):
|
||||||
|
return "camera"
|
||||||
|
return "other"
|
||||||
|
|
||||||
|
|
||||||
|
# Pause-variant video marker prefixes (mirrors transformer). These freeze the
|
||||||
|
# narration, so they're what makes narration_time and final_time diverge.
|
||||||
|
_PAUSE_MARKER_PREFIXES = (
|
||||||
|
"vftp:", "vfbp:", "vfmp:", "vf2tp:", "vf2bp:", "vf2mp:",
|
||||||
|
"vstp:", "vsbp:", "vsmp:",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _lookup_video(marker_id: str, videos: dict):
|
||||||
|
"""Case-insensitive videos.json lookup for a video marker (prefix stripped)."""
|
||||||
|
if not videos:
|
||||||
|
return None
|
||||||
|
handle = marker_id.split(":", 1)[1].lower() if ":" in marker_id else marker_id.lower()
|
||||||
|
return videos.get(handle) or next(
|
||||||
|
(v for k, v in videos.items() if k.lower() == handle), None
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _pause_duration(marker_id: str, videos: dict) -> float:
|
||||||
|
"""Seconds a pause-variant video marker freezes the narration for, else 0."""
|
||||||
|
if not marker_id.startswith(_PAUSE_MARKER_PREFIXES):
|
||||||
|
return 0.0
|
||||||
|
handle = marker_id.split(":", 1)[1].lower() if ":" in marker_id else marker_id.lower()
|
||||||
|
vs = None
|
||||||
|
if videos:
|
||||||
|
vs = videos.get(handle) or next(
|
||||||
|
(v for k, v in videos.items() if k.lower() == handle), None
|
||||||
|
)
|
||||||
|
return float(getattr(vs, "pause_narration", 0.0) or 0.0) if vs else 0.0
|
||||||
|
|
||||||
|
|
||||||
|
# ── derive events from aligner output ─────────────────────────────────────────
|
||||||
|
|
||||||
|
def derive_events(
|
||||||
|
marker_timings: list[MarkerTiming],
|
||||||
|
slides: dict,
|
||||||
|
videos: dict,
|
||||||
|
audio: dict,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Turn ordered MarkerTiming objects into three-clock event dicts.
|
||||||
|
|
||||||
|
Each event carries two clocks plus a tweak:
|
||||||
|
narration_time — aligner-owned position on the narration/transcript timeline
|
||||||
|
(interpolated for markers Whisper couldn't place). Read-only.
|
||||||
|
adjustment — a human tweak in seconds layered on narration_time. 0 here;
|
||||||
|
preserved across rebuilds by merge_events. The ONLY editable knob.
|
||||||
|
final_time — filled by compute_final_times: (narration_time + adjustment)
|
||||||
|
shifted forward by every pause before it. What the viewer sees.
|
||||||
|
Pause-narration videos also carry pause_duration so the offset is self-describing.
|
||||||
|
"""
|
||||||
|
events: list[dict] = []
|
||||||
|
for t in marker_timings:
|
||||||
|
placed = t.timestamp is not None and t.timestamp >= 0
|
||||||
|
after_prev = (t.context or "").startswith("(after previous)")
|
||||||
|
mapping = (
|
||||||
|
MAPPING_EXACT
|
||||||
|
if (placed and t.confidence >= _EXACT_THRESHOLD and not after_prev)
|
||||||
|
else MAPPING_INTERPOLATED
|
||||||
|
)
|
||||||
|
etype = marker_type(t.marker_id, slides, videos, audio)
|
||||||
|
e = {
|
||||||
|
"type": etype,
|
||||||
|
"id": t.marker_id,
|
||||||
|
"narration_time": round(t.timestamp, 3) if placed else None,
|
||||||
|
"adjustment": 0.0,
|
||||||
|
"final_time": None,
|
||||||
|
"mapping": mapping,
|
||||||
|
"confidence": round(t.confidence, 3),
|
||||||
|
"context": (t.context or "")[:80],
|
||||||
|
}
|
||||||
|
# Materialize per-occurrence presentation onto video events so events.json is
|
||||||
|
# atomic (each occurrence self-contained, GUI-editable) rather than depending on
|
||||||
|
# the handle's videos.json entry. Resolved from the shorthand prefix + any prior
|
||||||
|
# override the marker carried.
|
||||||
|
if etype == "video":
|
||||||
|
vs = _lookup_video(t.marker_id, videos)
|
||||||
|
if vs is not None:
|
||||||
|
pres = resolve_video_presentation(
|
||||||
|
t.marker_id,
|
||||||
|
vs,
|
||||||
|
t.overrides,
|
||||||
|
default_end_on=(
|
||||||
|
None if t.marker_id.startswith("narration:") else "next_video"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
e["handle"] = pres["handle"]
|
||||||
|
e["cutout"] = pres["cutout"]
|
||||||
|
e["layer"] = pres["layer"]
|
||||||
|
e["end_on"] = pres["end_on"]
|
||||||
|
if pres["take"] is not None:
|
||||||
|
e["take"] = pres["take"]
|
||||||
|
pd = _pause_duration(t.marker_id, videos)
|
||||||
|
if pd:
|
||||||
|
e["pause_duration"] = pd
|
||||||
|
events.append(e)
|
||||||
|
|
||||||
|
_interpolate_narration(events)
|
||||||
|
return events
|
||||||
|
|
||||||
|
|
||||||
|
def _interpolate_narration(events: list[dict]) -> None:
|
||||||
|
"""Fill `narration_time: None` entries by linear interpolation between placed
|
||||||
|
neighbours. Head/tail runs spread at +1s steps from the nearest known time (or
|
||||||
|
0.0). Mutates events in place."""
|
||||||
|
n = len(events)
|
||||||
|
i = 0
|
||||||
|
while i < n:
|
||||||
|
if events[i]["narration_time"] is not None:
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
j = i
|
||||||
|
while j < n and events[j]["narration_time"] is None:
|
||||||
|
j += 1
|
||||||
|
prev_time = events[i - 1]["narration_time"] if i > 0 else None
|
||||||
|
next_time = events[j]["narration_time"] if j < n else None
|
||||||
|
gap = j - i
|
||||||
|
if prev_time is not None and next_time is not None:
|
||||||
|
step = (next_time - prev_time) / (gap + 1)
|
||||||
|
for k in range(gap):
|
||||||
|
events[i + k]["narration_time"] = round(prev_time + step * (k + 1), 3)
|
||||||
|
elif prev_time is not None: # trailing run
|
||||||
|
for k in range(gap):
|
||||||
|
events[i + k]["narration_time"] = round(prev_time + 1.0 * (k + 1), 3)
|
||||||
|
elif next_time is not None: # leading run
|
||||||
|
base = max(0.0, next_time - gap)
|
||||||
|
for k in range(gap):
|
||||||
|
events[i + k]["narration_time"] = round(base + 1.0 * k, 3)
|
||||||
|
else: # nothing placed at all
|
||||||
|
for k in range(gap):
|
||||||
|
events[i + k]["narration_time"] = round(1.0 * (i + k), 3)
|
||||||
|
i = j
|
||||||
|
|
||||||
|
|
||||||
|
# ── final_time = narration + adjustment + pause offsets ───────────────────────
|
||||||
|
|
||||||
|
def compute_final_times(events: list[dict]) -> None:
|
||||||
|
"""Fill each event's final_time from its (narration_time + adjustment), shifted
|
||||||
|
forward by every pause whose own (narration + adjustment) is at or before it.
|
||||||
|
|
||||||
|
This is the single, clean definition of the narration→final mapping; the render
|
||||||
|
is fed the same effective narration times so its output matches these values.
|
||||||
|
Mutates events in place.
|
||||||
|
"""
|
||||||
|
def eff(e):
|
||||||
|
n = e.get("narration_time")
|
||||||
|
return None if n is None else n + e.get("adjustment", 0.0)
|
||||||
|
|
||||||
|
pauses = [
|
||||||
|
(eff(e), e.get("pause_duration", 0.0), id(e))
|
||||||
|
for e in events
|
||||||
|
if e.get("pause_duration") and eff(e) is not None
|
||||||
|
]
|
||||||
|
for e in events:
|
||||||
|
ee = eff(e)
|
||||||
|
if ee is None:
|
||||||
|
e["final_time"] = None
|
||||||
|
continue
|
||||||
|
shift = sum(dur for (pn, dur, oid) in pauses if pn <= ee and oid != id(e))
|
||||||
|
e["final_time"] = round(ee + shift, 3)
|
||||||
|
|
||||||
|
|
||||||
|
# ── re-interpolation on rebuild ───────────────────────────────────────────────
|
||||||
|
|
||||||
|
def reinterpolate_events(events: list[dict]) -> None:
|
||||||
|
"""Recompute every `interpolated` event's narration_time between its anchors.
|
||||||
|
|
||||||
|
Anchors are the events the aligner placed (mapping `exact`). Interpolated events
|
||||||
|
carry no real spoken cue, so their narration_time is only ever a guess spread
|
||||||
|
between anchors — recomputed each build. Adjustments are untouched (they layer
|
||||||
|
on top). Mutates events in place; call compute_final_times afterwards.
|
||||||
|
"""
|
||||||
|
for e in events:
|
||||||
|
if e.get("mapping") == MAPPING_INTERPOLATED:
|
||||||
|
e["narration_time"] = None
|
||||||
|
_interpolate_narration(events)
|
||||||
|
|
||||||
|
|
||||||
|
# ── merge: human tweaks survive re-align ──────────────────────────────────────
|
||||||
|
|
||||||
|
def pair_events_by_ordinal(fresh: list[dict], existing: Optional[list[dict]]):
|
||||||
|
"""Yield (fresh_event, matching_existing_event_or_None) pairs.
|
||||||
|
|
||||||
|
Both lists are in deterministic manuscript order, so the Nth occurrence of a
|
||||||
|
given id in `fresh` corresponds to the Nth occurrence in `existing`. Keying by
|
||||||
|
id alone would collapse legitimately-repeated markers (e.g. the same video
|
||||||
|
reused at two points in the script) onto a single old entry and cross-
|
||||||
|
contaminate their timings/tweaks.
|
||||||
|
"""
|
||||||
|
from collections import defaultdict
|
||||||
|
|
||||||
|
buckets: dict = defaultdict(list)
|
||||||
|
for e in existing or []:
|
||||||
|
buckets[e.get("id")].append(e)
|
||||||
|
counters: dict = defaultdict(int)
|
||||||
|
for e in fresh:
|
||||||
|
mid = e.get("id")
|
||||||
|
idx = counters[mid]
|
||||||
|
counters[mid] += 1
|
||||||
|
bucket = buckets.get(mid, [])
|
||||||
|
yield e, (bucket[idx] if idx < len(bucket) else None)
|
||||||
|
|
||||||
|
|
||||||
|
def merge_events(fresh: list[dict], existing: Optional[list[dict]]) -> list[dict]:
|
||||||
|
"""Carry each id's `adjustment` (the human tweak) from the existing events.json
|
||||||
|
onto the freshly-derived events. narration_time and mapping come from `fresh`
|
||||||
|
(aligner-owned); the tweak is relative so it stays meaningful even when the
|
||||||
|
alignment shifts. New ids are added; ids gone from the manuscript are dropped.
|
||||||
|
|
||||||
|
Matching is by ordinal (Nth occurrence to Nth occurrence), not by id, so a
|
||||||
|
marker reused several times keeps each occurrence's own tweak.
|
||||||
|
"""
|
||||||
|
if not existing:
|
||||||
|
return fresh
|
||||||
|
for e, old in pair_events_by_ordinal(fresh, existing):
|
||||||
|
if old is not None and old.get("adjustment"):
|
||||||
|
e["adjustment"] = old["adjustment"]
|
||||||
|
return fresh
|
||||||
|
|
||||||
|
|
||||||
|
# ── events <-> marker timings ─────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def events_to_marker_timings(events: list[dict]) -> list[MarkerTiming]:
|
||||||
|
"""Reconstruct MarkerTiming objects for build_render_plan override.
|
||||||
|
|
||||||
|
Emits the EFFECTIVE narration time (narration_time + adjustment). The render's
|
||||||
|
pause pass then shifts these into final time, reproducing each event's stored
|
||||||
|
final_time exactly — so events.json is the single source of the render's timing.
|
||||||
|
"""
|
||||||
|
timings: list[MarkerTiming] = []
|
||||||
|
for e in events:
|
||||||
|
n = e.get("narration_time")
|
||||||
|
eff = (n + e.get("adjustment", 0.0)) if n is not None else -1.0
|
||||||
|
# Carry any stored presentation as overrides so render honors the atomic event
|
||||||
|
# (e.g. a GUI edit) over the shorthand/videos.json default. Absent on old events.
|
||||||
|
overrides = {k: e[k] for k in _PRESENTATION_KEYS if k in e} or None
|
||||||
|
timings.append(
|
||||||
|
MarkerTiming(
|
||||||
|
marker_id=e["id"],
|
||||||
|
timestamp=eff,
|
||||||
|
context=e.get("context", ""),
|
||||||
|
confidence=float(e.get("confidence", 1.0)),
|
||||||
|
overrides=overrides,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return timings
|
||||||
|
|
||||||
|
|
||||||
|
# ── serialization ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def read_events(project_path: Path) -> Optional[list[dict]]:
|
||||||
|
p = project_path / EVENTS_FILE
|
||||||
|
if not p.exists():
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return json.loads(p.read_text(encoding="utf-8"))
|
||||||
|
except (json.JSONDecodeError, OSError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def write_events(project_path: Path, events: list[dict]) -> Path:
|
||||||
|
p = project_path / EVENTS_FILE
|
||||||
|
p.write_text(json.dumps(events, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
|
||||||
|
return p
|
||||||
|
|
||||||
|
|
||||||
|
def read_scaffold(project_path: Path) -> Optional[dict]:
|
||||||
|
p = project_path / SCAFFOLD_FILE
|
||||||
|
if not p.exists():
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return json.loads(p.read_text(encoding="utf-8"))
|
||||||
|
except (json.JSONDecodeError, OSError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def write_scaffold(
|
||||||
|
project_path: Path,
|
||||||
|
events: list[dict],
|
||||||
|
transcription: list,
|
||||||
|
narration_schedule: list,
|
||||||
|
total_duration: float,
|
||||||
|
) -> Path:
|
||||||
|
"""Write the compiled timeline document render consumes.
|
||||||
|
|
||||||
|
Stores times + ids (not heavy slide/video objects — those are re-resolved
|
||||||
|
from disk at render). Includes the transcription so the future timeline GUI
|
||||||
|
can show the spoken words under the draggable markers.
|
||||||
|
"""
|
||||||
|
scaffold = {
|
||||||
|
"version": 1,
|
||||||
|
"total_duration": round(total_duration, 3),
|
||||||
|
"narration": [
|
||||||
|
{
|
||||||
|
"seg_id": s.seg_id,
|
||||||
|
"source_path": str(s.source_path),
|
||||||
|
"skip": round(s.skip, 3),
|
||||||
|
"take": (round(s.take, 3) if s.take is not None else None),
|
||||||
|
"duration": round(s.duration, 3),
|
||||||
|
"offset": round(s.offset, 3),
|
||||||
|
}
|
||||||
|
for s in (narration_schedule or [])
|
||||||
|
],
|
||||||
|
"events": events,
|
||||||
|
"transcription": [
|
||||||
|
{"word": w.word, "start": round(w.start, 3), "end": round(w.end, 3)}
|
||||||
|
for w in (transcription or [])
|
||||||
|
],
|
||||||
|
}
|
||||||
|
p = project_path / SCAFFOLD_FILE
|
||||||
|
p.write_text(json.dumps(scaffold, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
|
||||||
|
return p
|
||||||
|
|
||||||
|
|
||||||
|
# ── transcribed manuscript (for diffing against the script) ───────────────────
|
||||||
|
|
||||||
|
def _w_start(w) -> float:
|
||||||
|
return w["start"] if isinstance(w, dict) else w.start
|
||||||
|
|
||||||
|
|
||||||
|
def _w_word(w) -> str:
|
||||||
|
return w["word"] if isinstance(w, dict) else w.word
|
||||||
|
|
||||||
|
|
||||||
|
def _fmt_marker(e: dict) -> str:
|
||||||
|
"""Render a marker line, mirroring manuscript.txt so the two files diff cleanly."""
|
||||||
|
mid = e["id"]
|
||||||
|
t = e.get("type")
|
||||||
|
if t == "video":
|
||||||
|
return f"[video:{mid}]"
|
||||||
|
if t == "audio":
|
||||||
|
return f"[audio:{mid}]"
|
||||||
|
return f"[{mid}]" # slide / camera / other
|
||||||
|
|
||||||
|
|
||||||
|
def build_transcribed_manuscript(events: list[dict], transcription: list) -> str:
|
||||||
|
"""Interleave the aligned markers into the ACTUAL spoken transcript.
|
||||||
|
|
||||||
|
Walks the transcript along the timeline and drops each marker (at its aligned
|
||||||
|
time) on its own line followed by the words spoken until the next marker —
|
||||||
|
the same shape as manuscript.txt, but with what was really said. Diff the two
|
||||||
|
to see exactly where the recording drifted from the script.
|
||||||
|
"""
|
||||||
|
# The transcript is the NARRATION timeline, so place markers by their effective
|
||||||
|
# narration time (narration_time + adjustment), not final_time.
|
||||||
|
def _eff(e):
|
||||||
|
nt = e.get("narration_time")
|
||||||
|
return None if nt is None else nt + e.get("adjustment", 0.0)
|
||||||
|
|
||||||
|
evs = sorted(
|
||||||
|
[(_eff(e), e) for e in events if _eff(e) is not None],
|
||||||
|
key=lambda pair: pair[0],
|
||||||
|
)
|
||||||
|
words = transcription or []
|
||||||
|
n = len(words)
|
||||||
|
|
||||||
|
lines: list[str] = [
|
||||||
|
"# Auto-generated from the spoken transcript with aligned markers.",
|
||||||
|
"# Diff against manuscript.txt to see where the recording drifts from the script.",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
buf: list[str] = []
|
||||||
|
|
||||||
|
def _flush() -> None:
|
||||||
|
if buf:
|
||||||
|
lines.append(" ".join(buf))
|
||||||
|
buf.clear()
|
||||||
|
|
||||||
|
wi = 0
|
||||||
|
for t, e in evs:
|
||||||
|
while wi < n and _w_start(words[wi]) < t:
|
||||||
|
buf.append(_w_word(words[wi]))
|
||||||
|
wi += 1
|
||||||
|
_flush()
|
||||||
|
if lines and lines[-1] != "":
|
||||||
|
lines.append("")
|
||||||
|
lines.append(_fmt_marker(e))
|
||||||
|
while wi < n:
|
||||||
|
buf.append(_w_word(words[wi]))
|
||||||
|
wi += 1
|
||||||
|
_flush()
|
||||||
|
|
||||||
|
return "\n".join(lines).rstrip() + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def write_transcribed_manuscript(
|
||||||
|
project_path: Path, events: list[dict], transcription: list
|
||||||
|
) -> Path:
|
||||||
|
p = project_path / TRANSCRIBED_FILE
|
||||||
|
p.write_text(build_transcribed_manuscript(events, transcription), encoding="utf-8")
|
||||||
|
return p
|
||||||
|
|
||||||
|
|
||||||
|
def mapping_summary(events: list[dict]) -> dict:
|
||||||
|
"""Count events by mapping for a concise build/render summary."""
|
||||||
|
out = {MAPPING_EXACT: 0, MAPPING_INTERPOLATED: 0, MAPPING_MANUAL: 0}
|
||||||
|
for e in events:
|
||||||
|
out[e.get("mapping", MAPPING_INTERPOLATED)] = out.get(e.get("mapping", MAPPING_INTERPOLATED), 0) + 1
|
||||||
|
return out
|
||||||
+149
@@ -0,0 +1,149 @@
|
|||||||
|
"""Persistent per-stage completion tracking.
|
||||||
|
|
||||||
|
Each pipeline stage (preprocess, trim, render) records a fingerprint of
|
||||||
|
its inputs in ``.gnommo_state.json`` when it completes successfully. On the next
|
||||||
|
run a stage can ask whether its inputs are unchanged (and its output still
|
||||||
|
present) and skip the work — the same staleness intelligence that ``all``'s
|
||||||
|
in-memory cascade provides, but persisted so it also applies to stages run on
|
||||||
|
their own.
|
||||||
|
|
||||||
|
Fingerprinting is hybrid:
|
||||||
|
- small text manifests (narration.json, videos.json, manuscript.txt,
|
||||||
|
project.json, slides.json, audio.json, transcripts) are hashed (sha256) so a
|
||||||
|
``touch`` or a git checkout that only rewrites mtimes doesn't force a
|
||||||
|
needless rerun;
|
||||||
|
- large media (processed narration segments, source videos and
|
||||||
|
images) use mtime+size, which is cheap and good enough to detect real edits.
|
||||||
|
|
||||||
|
The state file is purely an optimization: any read/parse/write failure degrades
|
||||||
|
to "not current" (rerun) and never raises, so a corrupt or missing state file
|
||||||
|
can't break a build.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Iterable, Optional, Union
|
||||||
|
|
||||||
|
STATE_FILENAME = ".gnommo_state.json"
|
||||||
|
STATE_VERSION = 1
|
||||||
|
|
||||||
|
# Fingerprint modes
|
||||||
|
HASH = "hash" # sha256 of file contents — for small text manifests
|
||||||
|
META = "meta" # mtime_ns + size — for large media
|
||||||
|
|
||||||
|
# An input descriptor is a (label, path, mode) triple.
|
||||||
|
InputSpec = tuple[str, Path, str]
|
||||||
|
|
||||||
|
|
||||||
|
def _state_path(project_path: Path) -> Path:
|
||||||
|
return project_path / STATE_FILENAME
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_state() -> dict:
|
||||||
|
return {"version": STATE_VERSION, "stages": {}}
|
||||||
|
|
||||||
|
|
||||||
|
def load_state(project_path: Path) -> dict:
|
||||||
|
"""Load the state file, returning an empty skeleton on any problem."""
|
||||||
|
path = _state_path(project_path)
|
||||||
|
if not path.exists():
|
||||||
|
return _empty_state()
|
||||||
|
try:
|
||||||
|
data = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
except (OSError, json.JSONDecodeError):
|
||||||
|
return _empty_state()
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
return _empty_state()
|
||||||
|
data.setdefault("version", STATE_VERSION)
|
||||||
|
if not isinstance(data.get("stages"), dict):
|
||||||
|
data["stages"] = {}
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def save_state(project_path: Path, state: dict) -> None:
|
||||||
|
"""Write the state file. Never raises — state is best-effort."""
|
||||||
|
try:
|
||||||
|
_state_path(project_path).write_text(
|
||||||
|
json.dumps(state, indent=2) + "\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def fingerprint_path(path: Union[str, Path], mode: str) -> Optional[str]:
|
||||||
|
"""Return a fingerprint for a single file, or None if it can't be read."""
|
||||||
|
p = Path(path)
|
||||||
|
try:
|
||||||
|
if mode == HASH:
|
||||||
|
h = hashlib.sha256()
|
||||||
|
with open(p, "rb") as fh:
|
||||||
|
for chunk in iter(lambda: fh.read(1 << 20), b""):
|
||||||
|
h.update(chunk)
|
||||||
|
return f"sha256:{h.hexdigest()}"
|
||||||
|
st = p.stat()
|
||||||
|
return f"meta:{st.st_mtime_ns}:{st.st_size}"
|
||||||
|
except OSError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def compute(inputs: Iterable[InputSpec]) -> dict:
|
||||||
|
"""Build a {label: fingerprint} map from (label, path, mode) triples.
|
||||||
|
|
||||||
|
A missing file yields a null fingerprint, so a file appearing or disappearing
|
||||||
|
counts as a change.
|
||||||
|
"""
|
||||||
|
return {label: fingerprint_path(path, mode) for label, path, mode in inputs}
|
||||||
|
|
||||||
|
|
||||||
|
def get_stage(project_path: Path, stage_key: str) -> dict:
|
||||||
|
"""Return the recorded record for a stage (``{}`` if none)."""
|
||||||
|
return load_state(project_path).get("stages", {}).get(stage_key, {})
|
||||||
|
|
||||||
|
|
||||||
|
def get_items(project_path: Path, stage_key: str) -> dict:
|
||||||
|
"""Return the per-item fingerprint map recorded for a stage (``{}`` if none)."""
|
||||||
|
items = get_stage(project_path, stage_key).get("items")
|
||||||
|
return items if isinstance(items, dict) else {}
|
||||||
|
|
||||||
|
|
||||||
|
def is_current(
|
||||||
|
project_path: Path,
|
||||||
|
stage_key: str,
|
||||||
|
inputs: dict,
|
||||||
|
outputs: Iterable[Union[str, Path]] = (),
|
||||||
|
) -> bool:
|
||||||
|
"""True iff the recorded input fingerprint matches ``inputs`` exactly and
|
||||||
|
every output in ``outputs`` exists on disk."""
|
||||||
|
for out in outputs:
|
||||||
|
if not Path(out).exists():
|
||||||
|
return False
|
||||||
|
recorded = get_stage(project_path, stage_key).get("inputs")
|
||||||
|
return recorded == inputs
|
||||||
|
|
||||||
|
|
||||||
|
def record(project_path: Path, stage_key: str, inputs: dict) -> None:
|
||||||
|
"""Persist a stage-level input fingerprint, marking the stage complete."""
|
||||||
|
state = load_state(project_path)
|
||||||
|
state.setdefault("stages", {})[stage_key] = {
|
||||||
|
"completed_at": datetime.now(timezone.utc).isoformat(),
|
||||||
|
"inputs": inputs,
|
||||||
|
}
|
||||||
|
save_state(project_path, state)
|
||||||
|
|
||||||
|
|
||||||
|
def record_items(project_path: Path, stage_key: str, items: dict) -> None:
|
||||||
|
"""Merge per-item fingerprints into a stage's record (for multi-segment
|
||||||
|
stages like preprocess/trim). Existing items for other keys are preserved."""
|
||||||
|
state = load_state(project_path)
|
||||||
|
stage = state.setdefault("stages", {}).setdefault(stage_key, {})
|
||||||
|
stage["completed_at"] = datetime.now(timezone.utc).isoformat()
|
||||||
|
merged = stage.get("items")
|
||||||
|
if not isinstance(merged, dict):
|
||||||
|
merged = {}
|
||||||
|
merged.update(items)
|
||||||
|
stage["items"] = merged
|
||||||
|
save_state(project_path, state)
|
||||||
@@ -102,88 +102,3 @@ def load_transcript(
|
|||||||
return [
|
return [
|
||||||
TranscribedWord(word=w["word"], start=w["start"], end=w["end"]) for w in data
|
TranscribedWord(word=w["word"], start=w["start"], end=w["end"]) for w in data
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
def _format_srt_timestamp(seconds: float) -> str:
|
|
||||||
"""Format seconds as SRT timestamp: HH:MM:SS,mmm"""
|
|
||||||
hours = int(seconds // 3600)
|
|
||||||
minutes = int((seconds % 3600) // 60)
|
|
||||||
secs = int(seconds % 60)
|
|
||||||
millis = int((seconds % 1) * 1000)
|
|
||||||
return f"{hours:02d}:{minutes:02d}:{secs:02d},{millis:03d}"
|
|
||||||
|
|
||||||
|
|
||||||
def words_to_srt(
|
|
||||||
words: list[TranscribedWord],
|
|
||||||
max_words_per_line: int = 10,
|
|
||||||
max_duration: float = 5.0,
|
|
||||||
gap_threshold: float = 1.0,
|
|
||||||
) -> str:
|
|
||||||
"""
|
|
||||||
Convert word-level timestamps to SRT caption format.
|
|
||||||
|
|
||||||
Groups words into readable caption segments based on:
|
|
||||||
- Maximum words per line (default: 10)
|
|
||||||
- Maximum segment duration (default: 5 seconds)
|
|
||||||
- Natural gaps between words (default: 1 second pause triggers new segment)
|
|
||||||
|
|
||||||
Args:
|
|
||||||
words: List of TranscribedWord with timestamps
|
|
||||||
max_words_per_line: Maximum words before splitting to new segment
|
|
||||||
max_duration: Maximum duration of a single caption segment
|
|
||||||
gap_threshold: Pause duration that triggers a new segment
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
SRT formatted string ready for YouTube upload
|
|
||||||
"""
|
|
||||||
if not words:
|
|
||||||
return ""
|
|
||||||
|
|
||||||
segments: list[tuple[float, float, str]] = [] # (start, end, text)
|
|
||||||
current_words: list[str] = []
|
|
||||||
segment_start: float = words[0].start
|
|
||||||
segment_end: float = words[0].end
|
|
||||||
|
|
||||||
for i, word in enumerate(words):
|
|
||||||
# Check if we should start a new segment
|
|
||||||
start_new_segment = False
|
|
||||||
|
|
||||||
# Gap between words
|
|
||||||
if current_words and (word.start - segment_end) > gap_threshold:
|
|
||||||
start_new_segment = True
|
|
||||||
|
|
||||||
# Too many words
|
|
||||||
if len(current_words) >= max_words_per_line:
|
|
||||||
start_new_segment = True
|
|
||||||
|
|
||||||
# Segment too long
|
|
||||||
if current_words and (word.end - segment_start) > max_duration:
|
|
||||||
start_new_segment = True
|
|
||||||
|
|
||||||
if start_new_segment and current_words:
|
|
||||||
# Save current segment
|
|
||||||
text = " ".join(current_words)
|
|
||||||
segments.append((segment_start, segment_end, text))
|
|
||||||
# Start new segment
|
|
||||||
current_words = []
|
|
||||||
segment_start = word.start
|
|
||||||
|
|
||||||
current_words.append(word.word)
|
|
||||||
segment_end = word.end
|
|
||||||
|
|
||||||
# Don't forget the last segment
|
|
||||||
if current_words:
|
|
||||||
text = " ".join(current_words)
|
|
||||||
segments.append((segment_start, segment_end, text))
|
|
||||||
|
|
||||||
# Format as SRT
|
|
||||||
srt_lines = []
|
|
||||||
for idx, (start, end, text) in enumerate(segments, 1):
|
|
||||||
srt_lines.append(str(idx))
|
|
||||||
srt_lines.append(
|
|
||||||
f"{_format_srt_timestamp(start)} --> {_format_srt_timestamp(end)}"
|
|
||||||
)
|
|
||||||
srt_lines.append(text)
|
|
||||||
srt_lines.append("") # Blank line between entries
|
|
||||||
|
|
||||||
return "\n".join(srt_lines)
|
|
||||||
|
|||||||
@@ -0,0 +1,289 @@
|
|||||||
|
"""SSH-based file transfer for gnommo projects (commit / up / down).
|
||||||
|
|
||||||
|
Workflow:
|
||||||
|
gnommo -p video1 commit -m "re-recorded slides 7-15"
|
||||||
|
gnommo -p video1 up # push to rendering server
|
||||||
|
gnommo -p video1 down # pull from rendering server
|
||||||
|
|
||||||
|
Design:
|
||||||
|
- commit appends a timestamped entry to commits.log
|
||||||
|
- up checks server commits.log for newer entry (aborts if found), then
|
||||||
|
rsyncs the WHOLE project tree, excluding only the large/derived files
|
||||||
|
in _SYNC_EXCLUDES. Everything else — manuscript, slides, narration
|
||||||
|
raw_mov/, per-segment transcripts, events.json, audio, videos, keynote
|
||||||
|
— is carried automatically, so a new kind of input file can never be
|
||||||
|
silently left behind.
|
||||||
|
- down rsyncs the WHOLE project tree back, with the same _SYNC_EXCLUDES.
|
||||||
|
|
||||||
|
Sync model: move everything, exclude a small denylist. The exclusions are large
|
||||||
|
derived artifacts each side regenerates or ships on its own (rendered output,
|
||||||
|
preprocessed segments, downscales, chunk scratch), so they never travel over SSH.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
_COMMITS_LOG = "commits.log"
|
||||||
|
|
||||||
|
# Files/dirs NEVER transferred in either direction. Everything else syncs, so new
|
||||||
|
# artifacts (transcripts, manuscript_transcribed.txt, events.json, scaffold.json, …)
|
||||||
|
# are carried automatically without touching this list. These are large, derived,
|
||||||
|
# or machine-local outputs that each side regenerates or ships itself:
|
||||||
|
# - out/ rendered video (produced on the rig; `handoff` ships it for review)
|
||||||
|
# - processed/ heavy preprocessed *_processed.mov segments (rig regenerates)
|
||||||
|
# - intermediate/ ffmpeg work files
|
||||||
|
# - low/ proxy/ derived downscales
|
||||||
|
# - **/chunks/ per-chunk preprocess scratch
|
||||||
|
# Note: events.json / scaffold.json are deliberately NOT excluded — they must reach
|
||||||
|
# the rig so `down` + `render` is all that's needed there, and render never rewrites
|
||||||
|
# them, so they stay identical on both ends.
|
||||||
|
_SYNC_EXCLUDES = [
|
||||||
|
"out/",
|
||||||
|
"media/narration/processed/",
|
||||||
|
"media/narration/old/",
|
||||||
|
"media/narration/raw_mov/old/",
|
||||||
|
"media/narration/intermediate/",
|
||||||
|
"media/videos/intermediate/",
|
||||||
|
"media/narration/low/",
|
||||||
|
"media/videos/low/",
|
||||||
|
"media/narration/proxy/",
|
||||||
|
"media/videos/proxy/",
|
||||||
|
"**/chunks/",
|
||||||
|
"*.tmp",
|
||||||
|
".*", # rsync in-progress temp files (.filename.XXXXXX) and .DS_Store
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Shared assets
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _find_shared_assets_root(project_path: Path) -> Optional[Path]:
|
||||||
|
"""Return the local shared_assets directory, or None if not found."""
|
||||||
|
candidate = project_path.parent / "shared_assets"
|
||||||
|
if candidate.is_dir():
|
||||||
|
return candidate
|
||||||
|
try:
|
||||||
|
from .cache import load_assets_config
|
||||||
|
assets = load_assets_config()
|
||||||
|
if assets:
|
||||||
|
candidate = assets / "shared_assets"
|
||||||
|
if candidate.is_dir():
|
||||||
|
return candidate
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Commit log helpers
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _read_log_lines(path: Path) -> list[str]:
|
||||||
|
if not path.exists():
|
||||||
|
return []
|
||||||
|
return [l for l in path.read_text(encoding="utf-8").splitlines() if l.strip()]
|
||||||
|
|
||||||
|
|
||||||
|
def _last_timestamp(lines: list[str]) -> Optional[str]:
|
||||||
|
for line in reversed(lines):
|
||||||
|
ts = line.split(" | ")[0].strip()
|
||||||
|
if ts:
|
||||||
|
return ts
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Commands
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def cmd_commit(project_path: Path, message: str) -> int:
|
||||||
|
log_path = project_path / _COMMITS_LOG
|
||||||
|
ts = datetime.now().strftime("%Y-%m-%dT%H:%M:%S")
|
||||||
|
with open(log_path, "a", encoding="utf-8") as f:
|
||||||
|
f.write(f"{ts} | {message}\n")
|
||||||
|
print(f"[{ts}] {message}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_up(project_path: Path, verbose: bool, dry_run: bool) -> int:
|
||||||
|
from .cache import load_server_config
|
||||||
|
|
||||||
|
server = load_server_config()
|
||||||
|
if server is None:
|
||||||
|
_print_server_error()
|
||||||
|
return 1
|
||||||
|
|
||||||
|
# Require a local commit
|
||||||
|
log_path = project_path / _COMMITS_LOG
|
||||||
|
local_lines = _read_log_lines(log_path)
|
||||||
|
if not local_lines:
|
||||||
|
print(f"Error: No commits. Run: gnommo -p {project_path.name} commit -m 'message'")
|
||||||
|
return 1
|
||||||
|
local_last = _last_timestamp(local_lines)
|
||||||
|
|
||||||
|
# Fetch server's commits.log and compare
|
||||||
|
remote_project = f"{server['path']}/{project_path.name}"
|
||||||
|
ssh_cmd = ["ssh", "-p", server["port"], f"{server['user']}@{server['host']}"]
|
||||||
|
result = subprocess.run(
|
||||||
|
[*ssh_cmd, f"cat {remote_project}/{_COMMITS_LOG} 2>/dev/null"],
|
||||||
|
capture_output=True, text=True,
|
||||||
|
)
|
||||||
|
server_lines = [l for l in result.stdout.splitlines() if l.strip()]
|
||||||
|
server_last = _last_timestamp(server_lines)
|
||||||
|
|
||||||
|
if server_last and server_last > local_last:
|
||||||
|
print(f"Error: Server has a more recent commit — pull first.")
|
||||||
|
print(f" Local: {local_last}")
|
||||||
|
print(f" Server: {server_last}")
|
||||||
|
print(f" Run: gnommo -p {project_path.name} down")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
print(f"Pushing: {project_path.name} (whole tree minus excludes, last commit: {local_last})")
|
||||||
|
|
||||||
|
shared_root = _find_shared_assets_root(project_path)
|
||||||
|
remote_shared = f"{server['path']}/shared_assets"
|
||||||
|
|
||||||
|
# Pass 1: project files — whole tree, denylist excludes.
|
||||||
|
rsync_cmd = [
|
||||||
|
"rsync", "-av", "--progress",
|
||||||
|
"-e", f"ssh -p {server['port']}",
|
||||||
|
*[f"--exclude={p}" for p in _SYNC_EXCLUDES],
|
||||||
|
f"{project_path}/",
|
||||||
|
f"{server['user']}@{server['host']}:{remote_project}/",
|
||||||
|
]
|
||||||
|
# Pass 2: shared assets — whole tree, same excludes.
|
||||||
|
rsync_shared = [
|
||||||
|
"rsync", "-av", "--progress",
|
||||||
|
"-e", f"ssh -p {server['port']}",
|
||||||
|
*[f"--exclude={p}" for p in _SYNC_EXCLUDES],
|
||||||
|
f"{shared_root}/" if shared_root else "",
|
||||||
|
f"{server['user']}@{server['host']}:{remote_shared}/",
|
||||||
|
]
|
||||||
|
|
||||||
|
if dry_run:
|
||||||
|
dry_project = rsync_cmd[:1] + ["--dry-run"] + rsync_cmd[1:]
|
||||||
|
print("\n[DRY RUN] Would push project:")
|
||||||
|
print(f" {' '.join(dry_project)}")
|
||||||
|
if shared_root:
|
||||||
|
dry_shared = rsync_shared[:1] + ["--dry-run"] + rsync_shared[1:]
|
||||||
|
print("\n[DRY RUN] Would push shared assets:")
|
||||||
|
print(f" {' '.join(dry_shared)}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
if verbose:
|
||||||
|
print(f" {' '.join(rsync_cmd)}")
|
||||||
|
|
||||||
|
# Pass 1: project files
|
||||||
|
subprocess.run([*ssh_cmd, f"mkdir -p {remote_project}"], check=True)
|
||||||
|
result = subprocess.run(rsync_cmd)
|
||||||
|
if result.returncode != 0:
|
||||||
|
print(f"Error: rsync failed for project files (code {result.returncode})")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
# Pass 2: shared assets
|
||||||
|
if shared_root:
|
||||||
|
print("\nPushing shared assets...")
|
||||||
|
subprocess.run([*ssh_cmd, f"mkdir -p {remote_shared}"], check=True)
|
||||||
|
result = subprocess.run(rsync_shared)
|
||||||
|
if result.returncode != 0:
|
||||||
|
print(f"Error: rsync failed for shared assets (code {result.returncode})")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
print("\nDone.")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_down(project_path: Path, verbose: bool, dry_run: bool) -> int:
|
||||||
|
from .cache import load_server_config
|
||||||
|
|
||||||
|
server = load_server_config()
|
||||||
|
if server is None:
|
||||||
|
_print_server_error()
|
||||||
|
return 1
|
||||||
|
|
||||||
|
remote_project = f"{server['path']}/{project_path.name}"
|
||||||
|
|
||||||
|
# Verify the project exists on the server
|
||||||
|
ssh_cmd = ["ssh", "-p", server["port"], f"{server['user']}@{server['host']}"]
|
||||||
|
result = subprocess.run(
|
||||||
|
[*ssh_cmd, f"test -d {remote_project} && echo ok"],
|
||||||
|
capture_output=True, text=True,
|
||||||
|
)
|
||||||
|
if result.stdout.strip() != "ok":
|
||||||
|
print(f"Error: Project not found on server: {remote_project}")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
remote_log_result = subprocess.run(
|
||||||
|
[*ssh_cmd, f"cat {remote_project}/{_COMMITS_LOG} 2>/dev/null"],
|
||||||
|
capture_output=True, text=True,
|
||||||
|
)
|
||||||
|
server_lines = [l for l in remote_log_result.stdout.splitlines() if l.strip()]
|
||||||
|
server_last = _last_timestamp(server_lines)
|
||||||
|
|
||||||
|
print(f"Pulling: {project_path.name} (server last commit: {server_last or 'none'})")
|
||||||
|
|
||||||
|
project_path.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
rsync_cmd = [
|
||||||
|
"rsync", "-av", "--progress",
|
||||||
|
"-e", f"ssh -p {server['port']}",
|
||||||
|
*[f"--exclude={p}" for p in _SYNC_EXCLUDES],
|
||||||
|
f"{server['user']}@{server['host']}:{remote_project}/",
|
||||||
|
f"{project_path}/",
|
||||||
|
]
|
||||||
|
|
||||||
|
remote_shared = f"{server['path']}/shared_assets"
|
||||||
|
local_shared = project_path.parent / "shared_assets"
|
||||||
|
rsync_shared_cmd = [
|
||||||
|
"rsync", "-av", "--progress",
|
||||||
|
"-e", f"ssh -p {server['port']}",
|
||||||
|
*[f"--exclude={p}" for p in _SYNC_EXCLUDES],
|
||||||
|
f"{server['user']}@{server['host']}:{remote_shared}/",
|
||||||
|
f"{local_shared}/",
|
||||||
|
]
|
||||||
|
|
||||||
|
if dry_run:
|
||||||
|
rsync_cmd.insert(1, "--dry-run")
|
||||||
|
print("\n[DRY RUN] Would pull project:")
|
||||||
|
print(f" {' '.join(rsync_cmd)}")
|
||||||
|
print("\n[DRY RUN] Would pull shared assets:")
|
||||||
|
print(f" {' '.join(rsync_shared_cmd)}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
if verbose:
|
||||||
|
print(f" {' '.join(rsync_cmd)}")
|
||||||
|
|
||||||
|
# Pass 1: project files
|
||||||
|
result = subprocess.run(rsync_cmd)
|
||||||
|
if result.returncode != 0:
|
||||||
|
print(f"Error: rsync failed for project files (code {result.returncode})")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
# Pass 2: shared assets (only if server has any)
|
||||||
|
check = subprocess.run(
|
||||||
|
[*ssh_cmd, f"test -d {remote_shared} && echo ok"],
|
||||||
|
capture_output=True, text=True,
|
||||||
|
)
|
||||||
|
if check.stdout.strip() == "ok":
|
||||||
|
print("\nPulling shared assets...")
|
||||||
|
local_shared.mkdir(parents=True, exist_ok=True)
|
||||||
|
result = subprocess.run(rsync_shared_cmd)
|
||||||
|
if result.returncode != 0:
|
||||||
|
print(f"Error: rsync failed for shared assets (code {result.returncode})")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
print("\nDone.")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _print_server_error():
|
||||||
|
print("Error: Server not configured. Add to ~/.gnommo.conf:")
|
||||||
|
print(" [server]")
|
||||||
|
print(" host = 192.168.1.100")
|
||||||
|
print(" user = username")
|
||||||
|
print(" path = /gnommo/project")
|
||||||
|
print(" port = 22")
|
||||||
+484
-109
@@ -22,7 +22,7 @@ from .models import (
|
|||||||
VideoEvent,
|
VideoEvent,
|
||||||
VideoSource,
|
VideoSource,
|
||||||
)
|
)
|
||||||
from .parser import get_video_duration, resolve_missing_videos
|
from .parser import get_video_duration, resolve_missing_videos, parse_marker
|
||||||
from .transcriber import TranscribedWord
|
from .transcriber import TranscribedWord
|
||||||
|
|
||||||
# Audio trigger offset: play sound this many seconds before the marker
|
# Audio trigger offset: play sound this many seconds before the marker
|
||||||
@@ -36,18 +36,77 @@ AUDIO_OFFSET_SECONDS = 1.0
|
|||||||
_SHORTHAND_PREFIXES: dict[str, tuple] = {
|
_SHORTHAND_PREFIXES: dict[str, tuple] = {
|
||||||
"vft:": ("fullscreen", "above"),
|
"vft:": ("fullscreen", "above"),
|
||||||
"vfb:": ("fullscreen", "below"),
|
"vfb:": ("fullscreen", "below"),
|
||||||
|
"vfm:": ("fullscreen", "mid"),
|
||||||
"vf2t:": ("fullscreen2", "above"),
|
"vf2t:": ("fullscreen2", "above"),
|
||||||
"vf2b:": ("fullscreen2", "below"),
|
"vf2b:": ("fullscreen2", "below"),
|
||||||
|
"vf2m:": ("fullscreen2", "mid"),
|
||||||
"vst:": ("square", "above"),
|
"vst:": ("square", "above"),
|
||||||
"vsb:": ("square", "below"),
|
"vsb:": ("square", "below"),
|
||||||
|
"vsm:": ("square", "mid"),
|
||||||
"vftp:": ("fullscreen", "above"),
|
"vftp:": ("fullscreen", "above"),
|
||||||
"vfbp:": ("fullscreen", "below"),
|
"vfbp:": ("fullscreen", "below"),
|
||||||
|
"vfmp:": ("fullscreen", "mid"),
|
||||||
"vf2tp:": ("fullscreen2", "above"),
|
"vf2tp:": ("fullscreen2", "above"),
|
||||||
"vf2bp:": ("fullscreen2", "below"),
|
"vf2bp:": ("fullscreen2", "below"),
|
||||||
|
"vf2mp:": ("fullscreen2", "mid"),
|
||||||
"vstp:": ("square", "above"),
|
"vstp:": ("square", "above"),
|
||||||
"vsbp:": ("square", "below"),
|
"vsbp:": ("square", "below"),
|
||||||
|
"vsmp:": ("square", "mid"),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Cutout zone the narration talking-head defaults to when a segment/source has none.
|
||||||
|
# Matches the convention import uses (cli._import_narration_segments writes this).
|
||||||
|
_NARRATION_CUTOUT = "talkinghead"
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_video_presentation(
|
||||||
|
marker_id: str,
|
||||||
|
video_source,
|
||||||
|
overrides: Optional[dict] = None,
|
||||||
|
default_end_on: Optional[str] = "next_video",
|
||||||
|
) -> dict:
|
||||||
|
"""Resolve a video marker's per-occurrence presentation to an atomic dict.
|
||||||
|
|
||||||
|
Single source of truth for how a marker becomes an event, shared by the render
|
||||||
|
transformer (_extract_video_events) and the scaffold that materializes events.json.
|
||||||
|
Presentation is per-occurrence: the shorthand prefix (vst: → square/above) decides
|
||||||
|
cutout+layer for THIS marker, so one handle can appear as vst: and vsb: without a
|
||||||
|
videos.json collision.
|
||||||
|
|
||||||
|
Precedence per field: explicit event override > shorthand prefix > videos.json
|
||||||
|
default > built-in default. `default_end_on` is "next_video" for video triggers and
|
||||||
|
None for [narration:] (which runs to the end).
|
||||||
|
|
||||||
|
Returns {handle, cutout, layer, end_on, take, pause_narration}.
|
||||||
|
"""
|
||||||
|
overrides = overrides or {}
|
||||||
|
|
||||||
|
prefix = next((p for p in _SHORTHAND_PREFIXES if marker_id.startswith(p)), None)
|
||||||
|
if prefix is not None:
|
||||||
|
handle = marker_id[len(prefix):].lower()
|
||||||
|
impl_cutout, impl_layer = _SHORTHAND_PREFIXES[prefix]
|
||||||
|
else:
|
||||||
|
# Legacy [video:X] / [narration:X] — strip the generic prefix if present.
|
||||||
|
handle = (marker_id.split(":", 1)[1] if ":" in marker_id else marker_id).lower()
|
||||||
|
impl_cutout = impl_layer = None
|
||||||
|
|
||||||
|
cutout = overrides.get("cutout") or impl_cutout or video_source.cutout
|
||||||
|
layer = overrides.get("layer") or impl_layer or video_source.layer
|
||||||
|
end_on = overrides.get("end_on") or video_source.end_on or default_end_on
|
||||||
|
take = overrides["take"] if "take" in overrides else video_source.take
|
||||||
|
pause_narration = overrides.get(
|
||||||
|
"pause_narration", video_source.pause_narration or 0.0
|
||||||
|
)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"handle": handle,
|
||||||
|
"cutout": cutout,
|
||||||
|
"layer": layer,
|
||||||
|
"end_on": end_on,
|
||||||
|
"take": take,
|
||||||
|
"pause_narration": float(pause_narration or 0.0),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class MarkerTiming:
|
class MarkerTiming:
|
||||||
@@ -57,6 +116,9 @@ class MarkerTiming:
|
|||||||
timestamp: float # -1 if not found
|
timestamp: float # -1 if not found
|
||||||
context: str # the text following the marker
|
context: str # the text following the marker
|
||||||
confidence: float # 0-1, how confident the match is
|
confidence: float # 0-1, how confident the match is
|
||||||
|
# Per-occurrence presentation overrides carried from events.json (GUI edits) so
|
||||||
|
# render honors them over the shorthand/videos.json defaults. None on fresh align.
|
||||||
|
overrides: Optional[dict] = None
|
||||||
|
|
||||||
|
|
||||||
def _normalize_text(text: str) -> str:
|
def _normalize_text(text: str) -> str:
|
||||||
@@ -157,18 +219,12 @@ def _is_known_marker(
|
|||||||
_VIDEO_PREFIXES = (
|
_VIDEO_PREFIXES = (
|
||||||
"video:",
|
"video:",
|
||||||
"narration:",
|
"narration:",
|
||||||
"vft:",
|
"vft:", "vfb:", "vfm:",
|
||||||
"vfb:",
|
"vf2t:", "vf2b:", "vf2m:",
|
||||||
"vf2t:",
|
"vst:", "vsb:", "vsm:",
|
||||||
"vf2b:",
|
"vftp:", "vfbp:", "vfmp:",
|
||||||
"vst:",
|
"vf2tp:", "vf2bp:", "vf2mp:",
|
||||||
"vsb:",
|
"vstp:", "vsbp:", "vsmp:",
|
||||||
"vftp:",
|
|
||||||
"vfbp:",
|
|
||||||
"vf2tp:",
|
|
||||||
"vf2bp:",
|
|
||||||
"vstp:",
|
|
||||||
"vsbp:",
|
|
||||||
)
|
)
|
||||||
if any(marker_id.startswith(p) for p in _VIDEO_PREFIXES):
|
if any(marker_id.startswith(p) for p in _VIDEO_PREFIXES):
|
||||||
return True
|
return True
|
||||||
@@ -226,7 +282,10 @@ def _extract_marker_contexts(
|
|||||||
|
|
||||||
raw_contexts = []
|
raw_contexts = []
|
||||||
for i in range(1, len(parts), 2):
|
for i in range(1, len(parts), 2):
|
||||||
marker_id = parts[i]
|
# Split the bracket into a bare marker id and any inline overrides
|
||||||
|
# ([vsb:clip, end_on=next_video]); the id drives alignment, the overrides ride
|
||||||
|
# along to the event.
|
||||||
|
marker_id, overrides = parse_marker(parts[i])
|
||||||
|
|
||||||
if not _is_known_marker(marker_id, slides, videos, audio):
|
if not _is_known_marker(marker_id, slides, videos, audio):
|
||||||
continue
|
continue
|
||||||
@@ -240,7 +299,7 @@ def _extract_marker_contexts(
|
|||||||
j += 1
|
j += 1
|
||||||
if j >= len(parts):
|
if j >= len(parts):
|
||||||
break
|
break
|
||||||
if _is_known_marker(parts[j], slides, videos, audio):
|
if _is_known_marker(parse_marker(parts[j])[0], slides, videos, audio):
|
||||||
break
|
break
|
||||||
j += 1
|
j += 1
|
||||||
|
|
||||||
@@ -248,22 +307,22 @@ def _extract_marker_contexts(
|
|||||||
following_text = " ".join(following_text.split())
|
following_text = " ".join(following_text.split())
|
||||||
following_text = _strip_unknown_markers(following_text, slides, videos, audio)
|
following_text = _strip_unknown_markers(following_text, slides, videos, audio)
|
||||||
following_text = " ".join(following_text.split())
|
following_text = " ".join(following_text.split())
|
||||||
raw_contexts.append((marker_id, following_text))
|
raw_contexts.append((marker_id, following_text, overrides))
|
||||||
|
|
||||||
contexts = []
|
contexts = []
|
||||||
for i, (marker_id, following_text) in enumerate(raw_contexts):
|
for i, (marker_id, following_text, overrides) in enumerate(raw_contexts):
|
||||||
if following_text:
|
if following_text:
|
||||||
words = following_text.split()[:10]
|
words = following_text.split()[:10]
|
||||||
contexts.append((marker_id, " ".join(words), False, "before"))
|
contexts.append((marker_id, " ".join(words), False, "before", overrides))
|
||||||
else:
|
else:
|
||||||
borrowed = False
|
borrowed = False
|
||||||
for j in range(i + 1, len(raw_contexts)):
|
for j in range(i + 1, len(raw_contexts)):
|
||||||
next_marker_id, next_text = raw_contexts[j]
|
next_marker_id, next_text, _ = raw_contexts[j]
|
||||||
if next_text:
|
if next_text:
|
||||||
if next_marker_id in (slides or {}):
|
if next_marker_id in (slides or {}):
|
||||||
break
|
break
|
||||||
words = next_text.split()[:10]
|
words = next_text.split()[:10]
|
||||||
contexts.append((marker_id, " ".join(words), True, "before"))
|
contexts.append((marker_id, " ".join(words), True, "before", overrides))
|
||||||
borrowed = True
|
borrowed = True
|
||||||
break
|
break
|
||||||
if not borrowed:
|
if not borrowed:
|
||||||
@@ -278,9 +337,9 @@ def _extract_marker_contexts(
|
|||||||
if preceding_text:
|
if preceding_text:
|
||||||
words = preceding_text.split()
|
words = preceding_text.split()
|
||||||
tail = " ".join(words[-6:])
|
tail = " ".join(words[-6:])
|
||||||
contexts.append((marker_id, tail, False, "after"))
|
contexts.append((marker_id, tail, False, "after", overrides))
|
||||||
else:
|
else:
|
||||||
contexts.append((marker_id, "", False, "before"))
|
contexts.append((marker_id, "", False, "before", overrides))
|
||||||
|
|
||||||
return contexts
|
return contexts
|
||||||
|
|
||||||
@@ -315,7 +374,9 @@ def _fuzzy_match_ratio(
|
|||||||
|
|
||||||
words_to_check = min(len(phrase_words), window_size)
|
words_to_check = min(len(phrase_words), window_size)
|
||||||
# Window only needs to cover pre_filler + phrase words + inter_filler slack
|
# Window only needs to cover pre_filler + phrase words + inter_filler slack
|
||||||
transcript_end = min(start_idx + pre_filler + words_to_check + inter_filler, len(transcription))
|
transcript_end = min(
|
||||||
|
start_idx + pre_filler + words_to_check + inter_filler, len(transcription)
|
||||||
|
)
|
||||||
|
|
||||||
transcript_words = [
|
transcript_words = [
|
||||||
_normalize_token(transcription[j].word)
|
_normalize_token(transcription[j].word)
|
||||||
@@ -410,6 +471,40 @@ def _find_phrase_timestamp(
|
|||||||
return -1, -1.0, 0.0, -1
|
return -1, -1.0, 0.0, -1
|
||||||
|
|
||||||
|
|
||||||
|
# Pause-variant video marker prefixes (e.g. [vftp:logo]). These freeze the
|
||||||
|
# narration when they fire, so their start must land in the gap BETWEEN words,
|
||||||
|
# never mid-word — otherwise the narration cuts out half-way through a word and
|
||||||
|
# finishes it when the video ends.
|
||||||
|
_PAUSE_MARKER_PREFIXES = (
|
||||||
|
"vftp:", "vfbp:", "vfmp:",
|
||||||
|
"vf2tp:", "vf2bp:", "vf2mp:",
|
||||||
|
"vstp:", "vsbp:", "vsmp:",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _snap_to_word_gap(t: float, transcription: list) -> float:
|
||||||
|
"""Snap a time to the midpoint of the gap between transcript words.
|
||||||
|
|
||||||
|
- t inside a word → midpoint of the gap AFTER that word (the word finishes,
|
||||||
|
then the pause begins), or the word's end if it's the last word.
|
||||||
|
- t already in a gap → midpoint of that gap.
|
||||||
|
- t before the first / after the last word → unchanged.
|
||||||
|
"""
|
||||||
|
if not transcription:
|
||||||
|
return t
|
||||||
|
n = len(transcription)
|
||||||
|
for i, w in enumerate(transcription):
|
||||||
|
if w.start <= t <= w.end:
|
||||||
|
nxt = transcription[i + 1] if i + 1 < n else None
|
||||||
|
return round((w.end + nxt.start) / 2, 3) if nxt else round(w.end, 3)
|
||||||
|
if w.start > t:
|
||||||
|
if i == 0:
|
||||||
|
return t
|
||||||
|
prev = transcription[i - 1]
|
||||||
|
return round((prev.end + w.start) / 2, 3)
|
||||||
|
return t
|
||||||
|
|
||||||
|
|
||||||
def align_markers_to_transcription(
|
def align_markers_to_transcription(
|
||||||
manuscript_text: str,
|
manuscript_text: str,
|
||||||
transcription: list[TranscribedWord],
|
transcription: list[TranscribedWord],
|
||||||
@@ -448,7 +543,7 @@ def align_markers_to_transcription(
|
|||||||
last_idx = 0
|
last_idx = 0
|
||||||
last_end_time = 0.0
|
last_end_time = 0.0
|
||||||
|
|
||||||
for marker_id, anchor_text, is_borrowed, anchor_type in contexts:
|
for marker_id, anchor_text, is_borrowed, anchor_type, overrides in contexts:
|
||||||
if not anchor_text.strip():
|
if not anchor_text.strip():
|
||||||
marker_time = last_end_time + 1.0
|
marker_time = last_end_time + 1.0
|
||||||
timings.append(
|
timings.append(
|
||||||
@@ -457,6 +552,7 @@ def align_markers_to_transcription(
|
|||||||
timestamp=marker_time,
|
timestamp=marker_time,
|
||||||
context="(after previous)",
|
context="(after previous)",
|
||||||
confidence=1.0,
|
confidence=1.0,
|
||||||
|
overrides=overrides,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
last_end_time = marker_time
|
last_end_time = marker_time
|
||||||
@@ -481,6 +577,7 @@ def align_markers_to_transcription(
|
|||||||
timestamp=marker_time,
|
timestamp=marker_time,
|
||||||
context=f"(end of: {anchor_text[:40]})",
|
context=f"(end of: {anchor_text[:40]})",
|
||||||
confidence=confidence,
|
confidence=confidence,
|
||||||
|
overrides=overrides,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
last_idx = match_end_idx
|
last_idx = match_end_idx
|
||||||
@@ -493,6 +590,7 @@ def align_markers_to_transcription(
|
|||||||
timestamp=adjusted_time,
|
timestamp=adjusted_time,
|
||||||
context=anchor_text[:50],
|
context=anchor_text[:50],
|
||||||
confidence=confidence,
|
confidence=confidence,
|
||||||
|
overrides=overrides,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
if not is_borrowed:
|
if not is_borrowed:
|
||||||
@@ -508,9 +606,78 @@ def align_markers_to_transcription(
|
|||||||
timestamp=-1.0,
|
timestamp=-1.0,
|
||||||
context=anchor_text[:50],
|
context=anchor_text[:50],
|
||||||
confidence=0.0,
|
confidence=0.0,
|
||||||
|
overrides=overrides,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Repair pass: retry INTERPOLATED markers that the forward scan missed.
|
||||||
|
# Root cause of cascade failures: one bad match advances last_idx past
|
||||||
|
# the true positions of several subsequent markers. Fix: search in a
|
||||||
|
# bounded window [prev_marker_time - 1s, next_marker_time + 2s] so we
|
||||||
|
# avoid false early matches while still recovering from cascade failures.
|
||||||
|
if any(t.timestamp < 0 for t in timings):
|
||||||
|
for i, timing in enumerate(timings):
|
||||||
|
if timing.timestamp >= 0:
|
||||||
|
continue
|
||||||
|
|
||||||
|
marker_id, anchor_text, is_borrowed, anchor_type, overrides = contexts[i]
|
||||||
|
if not anchor_text.strip():
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Lower bound: previous matched marker's timestamp → word index.
|
||||||
|
# Repairs processed in order, so already-repaired markers count too.
|
||||||
|
prev_time = 0.0
|
||||||
|
for j in range(i - 1, -1, -1):
|
||||||
|
if timings[j].timestamp >= 0:
|
||||||
|
prev_time = max(0.0, timings[j].timestamp - 1.0)
|
||||||
|
break
|
||||||
|
win_start = next(
|
||||||
|
(j for j, w in enumerate(transcription) if w.start >= prev_time),
|
||||||
|
0,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Upper bound: next matched marker in the timings list (+2s padding)
|
||||||
|
next_time = float("inf")
|
||||||
|
for j in range(i + 1, len(timings)):
|
||||||
|
if timings[j].timestamp >= 0:
|
||||||
|
next_time = timings[j].timestamp + 2.0
|
||||||
|
break
|
||||||
|
|
||||||
|
win_end = (
|
||||||
|
next(
|
||||||
|
(j for j, w in enumerate(transcription) if w.start > next_time),
|
||||||
|
len(transcription),
|
||||||
|
)
|
||||||
|
if next_time < float("inf")
|
||||||
|
else len(transcription)
|
||||||
|
)
|
||||||
|
|
||||||
|
if win_end <= win_start:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Search in the bounded window with a relaxed threshold
|
||||||
|
sub = transcription[win_start:win_end]
|
||||||
|
idx, timestamp, confidence, match_end_idx = _find_phrase_timestamp(
|
||||||
|
anchor_text,
|
||||||
|
sub,
|
||||||
|
start_from=0,
|
||||||
|
fuzzy_threshold=max(0.4, fuzzy_threshold - 0.1),
|
||||||
|
)
|
||||||
|
|
||||||
|
if idx >= 0:
|
||||||
|
if anchor_type == "after" and match_end_idx > 0:
|
||||||
|
end_word = sub[min(match_end_idx - 1, len(sub) - 1)]
|
||||||
|
marker_time = end_word.end
|
||||||
|
else:
|
||||||
|
marker_time = max(0.0, timestamp - 0.5)
|
||||||
|
timings[i] = MarkerTiming(
|
||||||
|
marker_id=marker_id,
|
||||||
|
timestamp=marker_time,
|
||||||
|
context=f"(repaired: {anchor_text[:40]})",
|
||||||
|
confidence=confidence,
|
||||||
|
overrides=overrides,
|
||||||
|
)
|
||||||
|
|
||||||
# Deduplicate slide markers. The manuscript pattern [SN]\n\n[SN] text... is
|
# Deduplicate slide markers. The manuscript pattern [SN]\n\n[SN] text... is
|
||||||
# common: the first blank occurrence is a visual-transition cue and the second
|
# common: the first blank occurrence is a visual-transition cue and the second
|
||||||
# carries the narration text used for alignment. We keep the first entry in
|
# carries the narration text used for alignment. We keep the first entry in
|
||||||
@@ -529,7 +696,24 @@ def align_markers_to_transcription(
|
|||||||
else:
|
else:
|
||||||
prev_idx = seen[timing.marker_id]
|
prev_idx = seen[timing.marker_id]
|
||||||
prev = deduped[prev_idx]
|
prev = deduped[prev_idx]
|
||||||
if prev.context == "(after previous)" and timing.context != "(after previous)":
|
# Upgrade if: previous was a placeholder/interpolated and the new one is better.
|
||||||
|
# Also upgrade if previous used the backward-looking "after" anchor —
|
||||||
|
# that heuristic gives end-of-preceding-section timing, but a direct
|
||||||
|
# "before" match on the second occurrence (start-of-new-section − 0.5s)
|
||||||
|
# is more accurate for when the slide should appear.
|
||||||
|
should_upgrade = (
|
||||||
|
prev.context == "(after previous)"
|
||||||
|
and timing.context != "(after previous)"
|
||||||
|
) or (
|
||||||
|
prev.timestamp < 0
|
||||||
|
and timing.timestamp >= 0
|
||||||
|
) or (
|
||||||
|
prev.context.startswith("(end of:")
|
||||||
|
and timing.timestamp >= 0
|
||||||
|
and timing.context != "(after previous)"
|
||||||
|
and not timing.context.startswith("(end of:")
|
||||||
|
)
|
||||||
|
if should_upgrade:
|
||||||
deduped[prev_idx] = MarkerTiming(
|
deduped[prev_idx] = MarkerTiming(
|
||||||
marker_id=prev.marker_id,
|
marker_id=prev.marker_id,
|
||||||
timestamp=timing.timestamp,
|
timestamp=timing.timestamp,
|
||||||
@@ -537,6 +721,16 @@ def align_markers_to_transcription(
|
|||||||
confidence=timing.confidence,
|
confidence=timing.confidence,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Snap pause-narration video markers off any word they land inside, to the
|
||||||
|
# midpoint of the gap after it — so the freeze happens between words, not
|
||||||
|
# mid-word. Flows into both the render plan and events.json.
|
||||||
|
if transcription:
|
||||||
|
for timing in deduped:
|
||||||
|
if timing.timestamp >= 0 and timing.marker_id.startswith(
|
||||||
|
_PAUSE_MARKER_PREFIXES
|
||||||
|
):
|
||||||
|
timing.timestamp = _snap_to_word_gap(timing.timestamp, transcription)
|
||||||
|
|
||||||
return deduped
|
return deduped
|
||||||
|
|
||||||
|
|
||||||
@@ -551,6 +745,9 @@ def build_render_plan(
|
|||||||
audio: Optional[dict[str, AudioDefinition]] = None,
|
audio: Optional[dict[str, AudioDefinition]] = None,
|
||||||
audio_dir: Optional[Path] = None,
|
audio_dir: Optional[Path] = None,
|
||||||
slide_range: Optional[tuple[str, Optional[str]]] = None,
|
slide_range: Optional[tuple[str, Optional[str]]] = None,
|
||||||
|
narration_schedule: Optional[list] = None,
|
||||||
|
narration_source: Optional[VideoSource] = None,
|
||||||
|
marker_timings_override: Optional[list["MarkerTiming"]] = None,
|
||||||
) -> tuple[RenderPlan, list[MarkerTiming]]:
|
) -> tuple[RenderPlan, list[MarkerTiming]]:
|
||||||
"""
|
"""
|
||||||
Build a complete render plan from manuscript and transcription.
|
Build a complete render plan from manuscript and transcription.
|
||||||
@@ -562,6 +759,12 @@ def build_render_plan(
|
|||||||
manuscript_text: The manuscript.txt content (source of truth for markers)
|
manuscript_text: The manuscript.txt content (source of truth for markers)
|
||||||
transcription: Word-level timestamps from whisper transcription
|
transcription: Word-level timestamps from whisper transcription
|
||||||
slide_range: Optional tuple of (start_slide, end_slide) for partial rendering.
|
slide_range: Optional tuple of (start_slide, end_slide) for partial rendering.
|
||||||
|
marker_timings_override: When provided (e.g. loaded from events.json /
|
||||||
|
scaffold.json), these timings are used verbatim instead of aligning
|
||||||
|
against the transcript. Their timestamps are already final-timeline
|
||||||
|
values, so the narration-skip adjustment below is skipped for them.
|
||||||
|
This is the seam that lets `render` consume a hand-edited scaffold
|
||||||
|
without re-running (and re-breaking on) fuzzy alignment.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Tuple of (RenderPlan, list of MarkerTiming for display)
|
Tuple of (RenderPlan, list of MarkerTiming for display)
|
||||||
@@ -569,7 +772,38 @@ def build_render_plan(
|
|||||||
audio = audio or {}
|
audio = audio or {}
|
||||||
audio_dir = audio_dir or project_path
|
audio_dir = audio_dir or project_path
|
||||||
|
|
||||||
# Find the main narration video first (need skip value for timing adjustment)
|
# Align markers to transcription timestamps — unless caller supplied timings
|
||||||
|
# (from the scaffold/events layer), in which case those win verbatim.
|
||||||
|
if marker_timings_override is not None:
|
||||||
|
marker_timings = marker_timings_override
|
||||||
|
else:
|
||||||
|
marker_timings = align_markers_to_transcription(
|
||||||
|
manuscript_text, transcription, slides=slides, videos=videos, audio=audio
|
||||||
|
)
|
||||||
|
|
||||||
|
# Find shared_assets directory
|
||||||
|
shared_assets_dir = None
|
||||||
|
if (project_path / "shared_assets").exists():
|
||||||
|
shared_assets_dir = project_path / "shared_assets"
|
||||||
|
elif (project_path.parent / "shared_assets").exists():
|
||||||
|
shared_assets_dir = project_path.parent / "shared_assets"
|
||||||
|
|
||||||
|
# Track which files are loaded from external cache
|
||||||
|
cached_files: set[str] = set()
|
||||||
|
|
||||||
|
# --- Narration source ---
|
||||||
|
# Render-time concat: narration is the concatenation of the scheduled
|
||||||
|
# segments, so there is no single file to probe — the total duration is the
|
||||||
|
# sum of the segment durations and skip is already baked into each segment.
|
||||||
|
if narration_schedule:
|
||||||
|
narration_video_id = "narration"
|
||||||
|
# cutout left unset — resolved to the talkinghead zone below (single source).
|
||||||
|
narration_video = narration_source or VideoSource(
|
||||||
|
source_file="", always_visible=True
|
||||||
|
)
|
||||||
|
narration_skip = 0.0
|
||||||
|
full_duration = sum(seg.duration for seg in narration_schedule)
|
||||||
|
else:
|
||||||
narration_video_id = config.main_video
|
narration_video_id = config.main_video
|
||||||
if isinstance(narration_video_id, list):
|
if isinstance(narration_video_id, list):
|
||||||
narration_video_id = narration_video_id[0] if narration_video_id else None
|
narration_video_id = narration_video_id[0] if narration_video_id else None
|
||||||
@@ -579,16 +813,18 @@ def build_render_plan(
|
|||||||
f"Available: {list(videos.keys())}"
|
f"Available: {list(videos.keys())}"
|
||||||
)
|
)
|
||||||
narration_video = videos[narration_video_id]
|
narration_video = videos[narration_video_id]
|
||||||
|
|
||||||
# Align markers to transcription timestamps
|
|
||||||
marker_timings = align_markers_to_transcription(
|
|
||||||
manuscript_text, transcription, slides=slides, videos=videos, audio=audio
|
|
||||||
)
|
|
||||||
|
|
||||||
# Apply skip offset: if narration video has skip, subtract it from all timestamps
|
|
||||||
# This accounts for the fact that the video will start at skip seconds, not 0
|
|
||||||
narration_skip = narration_video.skip
|
narration_skip = narration_video.skip
|
||||||
if narration_skip > 0:
|
video_path, is_cached = _resolve_video_path(
|
||||||
|
videos_dir, narration_video, shared_assets_dir, project_path
|
||||||
|
)
|
||||||
|
if is_cached:
|
||||||
|
cached_files.add(narration_video_id)
|
||||||
|
full_duration = get_video_duration(video_path)
|
||||||
|
|
||||||
|
# Apply skip offset: if narration starts at `skip` seconds, subtract it from
|
||||||
|
# all marker timestamps so they line up with the trimmed timeline. Skipped for
|
||||||
|
# override timings, which are already expressed in the final timeline.
|
||||||
|
if narration_skip > 0 and marker_timings_override is None:
|
||||||
for timing in marker_timings:
|
for timing in marker_timings:
|
||||||
if timing.timestamp >= 0:
|
if timing.timestamp >= 0:
|
||||||
timing.timestamp = max(0.0, timing.timestamp - narration_skip)
|
timing.timestamp = max(0.0, timing.timestamp - narration_skip)
|
||||||
@@ -599,30 +835,25 @@ def build_render_plan(
|
|||||||
if timing.timestamp >= 0:
|
if timing.timestamp >= 0:
|
||||||
marker_times[timing.marker_id] = timing.timestamp
|
marker_times[timing.marker_id] = timing.timestamp
|
||||||
|
|
||||||
# Find shared_assets directory
|
# Narration talking-head cutout. Narration segments carry no cutout of their own,
|
||||||
shared_assets_dir = None
|
# so default to the dedicated "talkinghead" zone (the convention import uses when it
|
||||||
if (project_path / "shared_assets").exists():
|
# creates narration entries — see cli._import_narration_segments), falling back to
|
||||||
shared_assets_dir = project_path / "shared_assets"
|
# the project's default slide type only if that zone isn't defined. Guard the lookup
|
||||||
elif (project_path.parent / "shared_assets").exists():
|
# so a misconfigured cutout gives a clear error instead of a bare KeyError.
|
||||||
shared_assets_dir = project_path.parent / "shared_assets"
|
narration_cutout_name = narration_video.cutout or (
|
||||||
|
_NARRATION_CUTOUT if _NARRATION_CUTOUT in config.cutouts else config.default_slide_type
|
||||||
narration_video = videos[narration_video_id]
|
|
||||||
cutout = config.cutouts[narration_video.cutout]
|
|
||||||
|
|
||||||
# Track which files are loaded from external cache
|
|
||||||
cached_files: set[str] = set()
|
|
||||||
|
|
||||||
narration_videos: list[tuple[str, VideoSource, CutoutDefinition]] = []
|
|
||||||
video_path, is_cached = _resolve_video_path(
|
|
||||||
videos_dir, narration_video, shared_assets_dir, project_path
|
|
||||||
)
|
)
|
||||||
if is_cached:
|
if narration_cutout_name not in config.cutouts:
|
||||||
cached_files.add(narration_video_id)
|
raise ValueError(
|
||||||
full_duration = get_video_duration(video_path)
|
f"Narration cutout '{narration_cutout_name}' not found in project cutouts "
|
||||||
|
f"{list(config.cutouts)}"
|
||||||
|
)
|
||||||
|
cutout = config.cutouts[narration_cutout_name]
|
||||||
# Adjust duration for skip (content starts at skip, so effective duration is less)
|
# Adjust duration for skip (content starts at skip, so effective duration is less)
|
||||||
effective_duration = full_duration - narration_skip
|
effective_duration = full_duration - narration_skip
|
||||||
# Get total duration from first always_visible video
|
narration_videos: list[tuple[str, VideoSource, CutoutDefinition]] = [
|
||||||
narration_videos.append((narration_video_id, narration_video, cutout))
|
(narration_video_id, narration_video, cutout)
|
||||||
|
]
|
||||||
# Resolve slide range to time range
|
# Resolve slide range to time range
|
||||||
time_offset = 0.0
|
time_offset = 0.0
|
||||||
render_end_time = effective_duration
|
render_end_time = effective_duration
|
||||||
@@ -651,16 +882,22 @@ def build_render_plan(
|
|||||||
# Before extracting video events, resolve any referenced videos that are missing
|
# Before extracting video events, resolve any referenced videos that are missing
|
||||||
# from the project's videos.json by looking them up in shared_assets/videos.json.
|
# from the project's videos.json by looking them up in shared_assets/videos.json.
|
||||||
_VIDEO_MARKER_PREFIXES = (
|
_VIDEO_MARKER_PREFIXES = (
|
||||||
"video:", "narration:", "vft:", "vfb:", "vf2t:", "vf2b:", "vst:", "vsb:",
|
"video:",
|
||||||
"vftp:", "vfbp:", "vf2tp:", "vf2bp:", "vstp:", "vsbp:",
|
"narration:",
|
||||||
|
"vft:", "vfb:", "vfm:",
|
||||||
|
"vf2t:", "vf2b:", "vf2m:",
|
||||||
|
"vst:", "vsb:", "vsm:",
|
||||||
|
"vftp:", "vfbp:", "vfmp:",
|
||||||
|
"vf2tp:", "vf2bp:", "vf2mp:",
|
||||||
|
"vstp:", "vsbp:", "vsmp:",
|
||||||
)
|
)
|
||||||
missing_video_ids = [
|
missing_video_ids = [
|
||||||
timing.marker_id[len(prefix):]
|
timing.marker_id[len(prefix) :]
|
||||||
for timing in marker_timings
|
for timing in marker_timings
|
||||||
if timing.timestamp >= 0
|
if timing.timestamp >= 0
|
||||||
for prefix in _VIDEO_MARKER_PREFIXES
|
for prefix in _VIDEO_MARKER_PREFIXES
|
||||||
if timing.marker_id.startswith(prefix)
|
if timing.marker_id.startswith(prefix)
|
||||||
and timing.marker_id[len(prefix):] not in videos
|
and timing.marker_id[len(prefix) :] not in videos
|
||||||
]
|
]
|
||||||
if missing_video_ids:
|
if missing_video_ids:
|
||||||
found = resolve_missing_videos(missing_video_ids, project_path, config)
|
found = resolve_missing_videos(missing_video_ids, project_path, config)
|
||||||
@@ -676,6 +913,7 @@ def build_render_plan(
|
|||||||
)
|
)
|
||||||
if video_warnings:
|
if video_warnings:
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
print("\nWarnings:", file=sys.stderr)
|
print("\nWarnings:", file=sys.stderr)
|
||||||
for w in video_warnings:
|
for w in video_warnings:
|
||||||
print(f" ⚠ {w}", file=sys.stderr)
|
print(f" ⚠ {w}", file=sys.stderr)
|
||||||
@@ -738,15 +976,22 @@ def build_render_plan(
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
# Offset all events that come AFTER this pause
|
# Offset all events that come AFTER this pause.
|
||||||
|
# Use >= so a slide that transitions at exactly narration_time is
|
||||||
|
# pushed past the pause (matching the >= already used for video events).
|
||||||
|
# Also extend the end_time of the current slide so it stays visible
|
||||||
|
# as the background behind the pause-video overlay, avoiding a gap.
|
||||||
for slide_event in slide_events:
|
for slide_event in slide_events:
|
||||||
if slide_event.start_time > narration_time:
|
if slide_event.start_time >= narration_time:
|
||||||
slide_event.start_time += pause_duration
|
slide_event.start_time += pause_duration
|
||||||
if slide_event.end_time > narration_time:
|
if slide_event.end_time >= narration_time:
|
||||||
slide_event.end_time += pause_duration
|
slide_event.end_time += pause_duration
|
||||||
|
|
||||||
for vid_event in video_events:
|
for vid_event in video_events:
|
||||||
if vid_event.start_time > narration_time:
|
if vid_event is event:
|
||||||
|
# Don't shift the pause event by its own pause
|
||||||
|
continue
|
||||||
|
if vid_event.start_time >= narration_time:
|
||||||
vid_event.start_time += pause_duration
|
vid_event.start_time += pause_duration
|
||||||
if vid_event.end_time > narration_time:
|
if vid_event.end_time > narration_time:
|
||||||
vid_event.end_time += pause_duration
|
vid_event.end_time += pause_duration
|
||||||
@@ -780,7 +1025,10 @@ def build_render_plan(
|
|||||||
videos.update(found)
|
videos.update(found)
|
||||||
still_missing = [vid_id for vid_id in config.outro if vid_id not in videos]
|
still_missing = [vid_id for vid_id in config.outro if vid_id not in videos]
|
||||||
for vid_id in still_missing:
|
for vid_id in still_missing:
|
||||||
print(f" WARNING: outro video '{vid_id}' not found in videos.json or shared_assets — skipped", flush=True)
|
print(
|
||||||
|
f" WARNING: outro video '{vid_id}' not found in videos.json or shared_assets — skipped",
|
||||||
|
flush=True,
|
||||||
|
)
|
||||||
|
|
||||||
# Build outro events (plays after narration ends)
|
# Build outro events (plays after narration ends)
|
||||||
outro_events = _extract_outro_events(
|
outro_events = _extract_outro_events(
|
||||||
@@ -822,6 +1070,7 @@ def build_render_plan(
|
|||||||
input_seek_time=time_offset,
|
input_seek_time=time_offset,
|
||||||
shared_assets_dir=shared_assets_dir,
|
shared_assets_dir=shared_assets_dir,
|
||||||
narration_pauses=narration_pauses,
|
narration_pauses=narration_pauses,
|
||||||
|
narration_segments=narration_schedule or [],
|
||||||
outro_events=outro_events,
|
outro_events=outro_events,
|
||||||
narration_end_time=narration_end_time,
|
narration_end_time=narration_end_time,
|
||||||
cached_files=cached_files,
|
cached_files=cached_files,
|
||||||
@@ -871,6 +1120,49 @@ def _resolve_video_path(
|
|||||||
return source_path, False
|
return source_path, False
|
||||||
|
|
||||||
|
|
||||||
|
def _interpolate_slide_times(
|
||||||
|
marker_timings: list[MarkerTiming],
|
||||||
|
slides: dict,
|
||||||
|
total_duration: float,
|
||||||
|
) -> list[float]:
|
||||||
|
"""
|
||||||
|
Return sorted slide timestamps with unaligned slides (timestamp < 0)
|
||||||
|
interpolated evenly between their aligned neighbours. Used by both
|
||||||
|
_extract_slide_events and _extract_video_events so video end-times
|
||||||
|
never skip over a slide that Whisper failed to align.
|
||||||
|
"""
|
||||||
|
all_markers = [
|
||||||
|
(t.timestamp, t.marker_id)
|
||||||
|
for t in marker_timings
|
||||||
|
if t.marker_id in slides
|
||||||
|
]
|
||||||
|
if not all_markers:
|
||||||
|
return []
|
||||||
|
|
||||||
|
n = len(all_markers)
|
||||||
|
resolved = list(all_markers)
|
||||||
|
i = 0
|
||||||
|
while i < n:
|
||||||
|
if resolved[i][0] < 0:
|
||||||
|
run_start = i
|
||||||
|
while i < n and resolved[i][0] < 0:
|
||||||
|
i += 1
|
||||||
|
run_end = i
|
||||||
|
prev_time = resolved[run_start - 1][0] if run_start > 0 else 0.0
|
||||||
|
next_time = resolved[run_end][0] if run_end < n else total_duration
|
||||||
|
count = run_end - run_start
|
||||||
|
for j, idx in enumerate(range(run_start, run_end)):
|
||||||
|
frac = (j + 1) / (count + 1)
|
||||||
|
resolved[idx] = (
|
||||||
|
prev_time + (next_time - prev_time) * frac,
|
||||||
|
resolved[idx][1],
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
i += 1
|
||||||
|
|
||||||
|
return sorted(t for t, _ in resolved)
|
||||||
|
|
||||||
|
|
||||||
def _extract_slide_events(
|
def _extract_slide_events(
|
||||||
marker_timings: list[MarkerTiming],
|
marker_timings: list[MarkerTiming],
|
||||||
slides: dict[str, SlideDefinition],
|
slides: dict[str, SlideDefinition],
|
||||||
@@ -888,7 +1180,8 @@ def _extract_slide_events(
|
|||||||
"""
|
"""
|
||||||
range_start, range_end = time_range if time_range else (0.0, float("inf"))
|
range_start, range_end = time_range if time_range else (0.0, float("inf"))
|
||||||
|
|
||||||
# Get ALL slide markers in manuscript order (aligned and unaligned)
|
# Get ALL slide markers in manuscript order (aligned and unaligned),
|
||||||
|
# with unaligned ones interpolated via the shared helper.
|
||||||
all_slide_markers: list[tuple[float, str]] = []
|
all_slide_markers: list[tuple[float, str]] = []
|
||||||
for timing in marker_timings:
|
for timing in marker_timings:
|
||||||
if timing.marker_id in slides:
|
if timing.marker_id in slides:
|
||||||
@@ -897,9 +1190,8 @@ def _extract_slide_events(
|
|||||||
if not all_slide_markers:
|
if not all_slide_markers:
|
||||||
return []
|
return []
|
||||||
|
|
||||||
# Interpolate timestamps for unaligned slides (timestamp < 0).
|
# Re-derive interpolated times (same logic as _interpolate_slide_times but
|
||||||
# For each run of consecutive unaligned slides, spread them evenly between
|
# we need the (time, id) pairs here for event building).
|
||||||
# the nearest aligned slides before and after in manuscript order.
|
|
||||||
n = len(all_slide_markers)
|
n = len(all_slide_markers)
|
||||||
resolved: list[tuple[float, str]] = list(all_slide_markers)
|
resolved: list[tuple[float, str]] = list(all_slide_markers)
|
||||||
|
|
||||||
@@ -972,23 +1264,18 @@ def _extract_video_events(
|
|||||||
warnings: list[str] = []
|
warnings: list[str] = []
|
||||||
range_start, range_end = time_range if time_range else (0.0, float("inf"))
|
range_start, range_end = time_range if time_range else (0.0, float("inf"))
|
||||||
|
|
||||||
# Collect slide times for video: end time calculation
|
# Collect slide times for video end-time calculation.
|
||||||
slide_times: list[float] = sorted(
|
# Use the interpolated times (same as _extract_slide_events) so that a slide
|
||||||
[
|
# Whisper failed to align doesn't get skipped, causing the preceding video to
|
||||||
t.timestamp
|
# bleed through into the following slide.
|
||||||
for t in marker_timings
|
slide_times: list[float] = _interpolate_slide_times(
|
||||||
if t.marker_id in slides and t.timestamp >= 0
|
marker_timings, slides, total_duration
|
||||||
]
|
|
||||||
)
|
)
|
||||||
|
|
||||||
# Pause-variant prefixes — the only thing the render pass still needs from
|
# Collect video markers. Carry the full marker_id (so presentation resolves from
|
||||||
# shorthand markers at event-build time (pause_narration is per-event, not stored in videos.json).
|
# its shorthand prefix PER occurrence) and any per-event overrides from events.json.
|
||||||
_PAUSE_PREFIXES = {"vftp:", "vfbp:", "vf2tp:", "vf2bp:", "vstp:", "vsbp:"}
|
# video_markers: (timestamp, marker_id, handle, trigger_type, overrides)
|
||||||
|
video_markers: list[tuple[float, str, str, str, Optional[dict]]] = []
|
||||||
# Collect video markers: (time, video_id, event_type, pause_narration)
|
|
||||||
# video_markers: (timestamp, video_id, marker_type, pause_narration)
|
|
||||||
# cutout and layer are read from videos.json (projected there by _project_markers_to_videos)
|
|
||||||
video_markers: list[tuple[float, str, str, bool]] = []
|
|
||||||
|
|
||||||
for timing in marker_timings:
|
for timing in marker_timings:
|
||||||
if timing.timestamp < 0:
|
if timing.timestamp < 0:
|
||||||
@@ -997,9 +1284,11 @@ def _extract_video_events(
|
|||||||
mid = timing.marker_id
|
mid = timing.marker_id
|
||||||
|
|
||||||
# --- shorthand markers (vft:/vfb:/vst:/vsb: and pause variants) ---
|
# --- shorthand markers (vft:/vfb:/vst:/vsb: and pause variants) ---
|
||||||
shorthand_match = next((p for p in _SHORTHAND_PREFIXES if mid.startswith(p)), None)
|
shorthand_match = next(
|
||||||
|
(p for p in _SHORTHAND_PREFIXES if mid.startswith(p)), None
|
||||||
|
)
|
||||||
if shorthand_match:
|
if shorthand_match:
|
||||||
video_id = mid[len(shorthand_match):]
|
video_id = mid[len(shorthand_match) :].lower()
|
||||||
if video_id not in videos:
|
if video_id not in videos:
|
||||||
warnings.append(
|
warnings.append(
|
||||||
f"[{mid}] references unknown video '{video_id}' — skipped. "
|
f"[{mid}] references unknown video '{video_id}' — skipped. "
|
||||||
@@ -1014,13 +1303,14 @@ def _extract_video_events(
|
|||||||
f"run render once to project values, or set cutout manually."
|
f"run render once to project values, or set cutout manually."
|
||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
pause_narration = shorthand_match in _PAUSE_PREFIXES
|
video_markers.append(
|
||||||
video_markers.append((timing.timestamp, video_id, "video", pause_narration))
|
(timing.timestamp, mid, video_id, "video", timing.overrides)
|
||||||
|
)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
# --- legacy [video:xxx] ---
|
# --- legacy [video:xxx] ---
|
||||||
if mid.startswith("video:"):
|
if mid.startswith("video:"):
|
||||||
video_id = mid[6:]
|
video_id = mid[6:].lower()
|
||||||
if video_id not in videos:
|
if video_id not in videos:
|
||||||
warnings.append(
|
warnings.append(
|
||||||
f"[video:{video_id}] references unknown video '{video_id}' — skipped."
|
f"[video:{video_id}] references unknown video '{video_id}' — skipped."
|
||||||
@@ -1032,12 +1322,14 @@ def _extract_video_events(
|
|||||||
f"[video:{video_id}] has no valid cutout in videos.json — skipped."
|
f"[video:{video_id}] has no valid cutout in videos.json — skipped."
|
||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
video_markers.append((timing.timestamp, video_id, "video", False))
|
video_markers.append(
|
||||||
|
(timing.timestamp, mid, video_id, "video", timing.overrides)
|
||||||
|
)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
# --- [narration:xxx] ---
|
# --- [narration:xxx] ---
|
||||||
if mid.startswith("narration:"):
|
if mid.startswith("narration:"):
|
||||||
video_id = mid[10:]
|
video_id = mid[10:].lower()
|
||||||
if video_id not in videos:
|
if video_id not in videos:
|
||||||
warnings.append(
|
warnings.append(
|
||||||
f"[narration:{video_id}] references unknown video '{video_id}' — skipped."
|
f"[narration:{video_id}] references unknown video '{video_id}' — skipped."
|
||||||
@@ -1049,36 +1341,94 @@ def _extract_video_events(
|
|||||||
f"[narration:{video_id}] has no valid cutout in videos.json — skipped."
|
f"[narration:{video_id}] has no valid cutout in videos.json — skipped."
|
||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
video_markers.append((timing.timestamp, video_id, "narration", False))
|
video_markers.append(
|
||||||
|
(timing.timestamp, mid, video_id, "narration", timing.overrides)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Sorted start times of all video markers — used by end_on="next_video" to cap
|
||||||
|
# a clip when the next video begins, so videos never overlap.
|
||||||
|
video_start_times = sorted(t for t, *_ in video_markers)
|
||||||
|
|
||||||
events: list[VideoEvent] = []
|
events: list[VideoEvent] = []
|
||||||
for (start_time, video_id, marker_type, pause_narration) in video_markers:
|
for start_time, marker_id, video_id, trigger_type, overrides in video_markers:
|
||||||
video_source = videos[video_id]
|
video_source = videos[video_id]
|
||||||
|
|
||||||
# Read cutout and layer directly from videos.json (projected by ETL)
|
# Resolve presentation per-occurrence: shorthand prefix (and any events.json
|
||||||
cutout_name = video_source.cutout
|
# override) wins over the videos.json default, so one handle can render above
|
||||||
|
# in one place and below in another. [narration:] runs to the end by default.
|
||||||
|
pres = resolve_video_presentation(
|
||||||
|
marker_id,
|
||||||
|
video_source,
|
||||||
|
overrides,
|
||||||
|
default_end_on=(None if trigger_type == "narration" else "next_video"),
|
||||||
|
)
|
||||||
|
cutout_name = pres["cutout"]
|
||||||
cutout = cutouts[cutout_name]
|
cutout = cutouts[cutout_name]
|
||||||
layer = video_source.layer
|
layer = pres["layer"]
|
||||||
|
end_on = pres["end_on"]
|
||||||
|
take = pres["take"]
|
||||||
|
pause_narration = pres["pause_narration"]
|
||||||
|
|
||||||
end_on = video_source.end_on
|
if end_on == "take" and take is not None:
|
||||||
if end_on == "take" and video_source.take is not None:
|
end_time = start_time + take
|
||||||
end_time = start_time + video_source.take
|
|
||||||
elif end_on == "end":
|
elif end_on == "end":
|
||||||
|
# Play the clip once through its natural length, then stop — no looping.
|
||||||
|
# Natural length = explicit take, else the file's own duration past skip.
|
||||||
|
if take is not None:
|
||||||
|
natural = take
|
||||||
|
elif video_source.duration is not None:
|
||||||
|
natural = max(0.0, video_source.duration - (video_source.skip or 0.0))
|
||||||
|
else:
|
||||||
|
natural = None # unknown length — fall back to running to render end
|
||||||
|
end_time = (start_time + natural) if natural is not None else total_duration
|
||||||
|
elif end_on == "loop":
|
||||||
|
# Loop the clip to fill the rest of the render.
|
||||||
end_time = total_duration
|
end_time = total_duration
|
||||||
elif end_on == "next_slide" or (end_on is None and marker_type == "video"):
|
elif end_on in ("next_video", "video"):
|
||||||
# End at next slide marker
|
# End when the next video (any) starts, so clips never overlap. Lets a
|
||||||
|
# video span multiple slides yet still yield to the following video.
|
||||||
|
end_time = total_duration
|
||||||
|
for vt in video_start_times:
|
||||||
|
if vt > start_time:
|
||||||
|
end_time = vt
|
||||||
|
break
|
||||||
|
# A pause-narration video must stay for at least the pause it holds.
|
||||||
|
if pause_narration:
|
||||||
|
end_time = max(end_time, start_time + pause_narration)
|
||||||
|
elif end_on in ("next_slide", "slide"):
|
||||||
|
# End at next slide marker ("slide" is a recognised alias for "next_slide")
|
||||||
end_time = total_duration
|
end_time = total_duration
|
||||||
for slide_time in slide_times:
|
for slide_time in slide_times:
|
||||||
if slide_time > start_time:
|
if slide_time > start_time:
|
||||||
end_time = slide_time
|
end_time = slide_time
|
||||||
break
|
break
|
||||||
|
# pause_narration videos must stay visible for the full pause duration —
|
||||||
|
# the narration is held for that long, so the overlay should match.
|
||||||
|
if pause_narration:
|
||||||
|
end_time = max(end_time, start_time + pause_narration)
|
||||||
else:
|
else:
|
||||||
# end_on is None and marker_type == "narration": runs to end
|
# end_on None ([narration:] with no explicit end) — runs to end.
|
||||||
end_time = total_duration
|
end_time = total_duration
|
||||||
|
|
||||||
# Filter by time range
|
# Filter by time range — CHUNKING v2 (docs/chunking_v2.md).
|
||||||
if start_time < range_start or start_time >= range_end:
|
# Include any clip that OVERLAPS the window (not just those starting inside
|
||||||
|
# it), so a clip spanning a chunk boundary survives into the later chunk.
|
||||||
|
if end_time <= range_start or start_time >= range_end:
|
||||||
continue
|
continue
|
||||||
|
# A clip that began before this window is already mid-playback at the seam;
|
||||||
|
# seek into it so it resumes at the right frame instead of restarting.
|
||||||
|
skip_override = None
|
||||||
|
if start_time < range_start:
|
||||||
|
into = range_start - start_time # elapsed since the clip started
|
||||||
|
base = video_source.skip or 0.0
|
||||||
|
playable = (video_source.duration - base) if video_source.duration else None
|
||||||
|
if playable and playable > 0 and into >= playable:
|
||||||
|
# the clip has looped by the window start → resume at the loop phase
|
||||||
|
skip_override = base + (into % playable)
|
||||||
|
else:
|
||||||
|
# still within the first play-through (or unknown length) → linear seek
|
||||||
|
skip_override = base + into
|
||||||
|
start_time = range_start # -> 0 after time_offset subtraction
|
||||||
end_time = min(end_time, range_end)
|
end_time = min(end_time, range_end)
|
||||||
|
|
||||||
events.append(
|
events.append(
|
||||||
@@ -1090,6 +1440,7 @@ def _extract_video_events(
|
|||||||
cutout=cutout,
|
cutout=cutout,
|
||||||
cutout_name=cutout_name,
|
cutout_name=cutout_name,
|
||||||
layer=layer,
|
layer=layer,
|
||||||
|
skip_override=skip_override,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -1116,14 +1467,35 @@ def _extract_audio_events(
|
|||||||
elif marker_id.startswith("audio:"):
|
elif marker_id.startswith("audio:"):
|
||||||
audio_id = marker_id[6:]
|
audio_id = marker_id[6:]
|
||||||
if audio_id is not None and audio_id in audio:
|
if audio_id is not None and audio_id in audio:
|
||||||
if timing.timestamp < range_start or timing.timestamp >= range_end:
|
adef = audio[audio_id]
|
||||||
|
astart = max(0, timing.timestamp - AUDIO_OFFSET_SECONDS)
|
||||||
|
# Effective end of this clip on the output timeline.
|
||||||
|
if adef.loop:
|
||||||
|
aend = range_end # a loop fills to the window/render end
|
||||||
|
elif adef.duration is not None:
|
||||||
|
aend = astart + adef.duration
|
||||||
|
else:
|
||||||
|
aend = float("inf") # unknown one-shot length — assume it may span
|
||||||
|
# CHUNKING v2 (docs/chunking_v2.md): include if it OVERLAPS the window,
|
||||||
|
# and seek into clips that began earlier so they resume mid-track — the
|
||||||
|
# loop phase for looping music, a linear seek for one-shots. v1 dropped
|
||||||
|
# these, silencing looping background music in every chunk but the first.
|
||||||
|
if aend <= range_start or astart >= range_end:
|
||||||
continue
|
continue
|
||||||
start_time = max(0, timing.timestamp - AUDIO_OFFSET_SECONDS)
|
src_offset = 0.0
|
||||||
|
if astart < range_start:
|
||||||
|
into = range_start - astart
|
||||||
|
if adef.loop and adef.duration:
|
||||||
|
src_offset = into % adef.duration
|
||||||
|
else:
|
||||||
|
src_offset = into
|
||||||
|
astart = range_start
|
||||||
events.append(
|
events.append(
|
||||||
AudioEvent(
|
AudioEvent(
|
||||||
audio_id=audio_id,
|
audio_id=audio_id,
|
||||||
start_time=start_time,
|
start_time=astart,
|
||||||
audio_def=audio[audio_id],
|
audio_def=adef,
|
||||||
|
src_offset=src_offset,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -1244,7 +1616,10 @@ def _extract_outro_events(
|
|||||||
)
|
)
|
||||||
if is_cached and cached_files is not None:
|
if is_cached and cached_files is not None:
|
||||||
cached_files.add(video_id)
|
cached_files.add(video_id)
|
||||||
if video_path.exists():
|
# Prefer the import-time duration from videos.json; only probe when absent.
|
||||||
|
if video_source.duration is not None:
|
||||||
|
full_duration = video_source.duration
|
||||||
|
elif video_path.exists():
|
||||||
full_duration = get_video_duration(video_path)
|
full_duration = get_video_duration(video_path)
|
||||||
else:
|
else:
|
||||||
full_duration = 10.0 # Fallback
|
full_duration = 10.0 # Fallback
|
||||||
|
|||||||
+97
-19
@@ -4,7 +4,7 @@ from pathlib import Path
|
|||||||
|
|
||||||
from .cache import resolve_with_cache
|
from .cache import resolve_with_cache
|
||||||
from .errors import ValidationError, ValidationIssue
|
from .errors import ValidationError, ValidationIssue
|
||||||
from .parser import _read_json
|
from .parser import _read_json, resolve_missing_videos
|
||||||
from .models import (
|
from .models import (
|
||||||
ProjectConfig,
|
ProjectConfig,
|
||||||
SlideDefinition,
|
SlideDefinition,
|
||||||
@@ -22,6 +22,7 @@ def validate_project(
|
|||||||
videos: dict[str, VideoSource],
|
videos: dict[str, VideoSource],
|
||||||
videos_dir: Path,
|
videos_dir: Path,
|
||||||
malformed_markers: list[tuple[int, str]] = None,
|
malformed_markers: list[tuple[int, str]] = None,
|
||||||
|
audio: dict = None,
|
||||||
) -> list[ValidationIssue]:
|
) -> list[ValidationIssue]:
|
||||||
"""
|
"""
|
||||||
Validate all parsed project data. Raises ValidationError if any issues found.
|
Validate all parsed project data. Raises ValidationError if any issues found.
|
||||||
@@ -34,10 +35,33 @@ def validate_project(
|
|||||||
- Background video exists (if specified)
|
- Background video exists (if specified)
|
||||||
- Slide types are valid
|
- Slide types are valid
|
||||||
- No malformed markers in manuscript
|
- No malformed markers in manuscript
|
||||||
|
- All audio: markers in manuscript exist in audio.json
|
||||||
"""
|
"""
|
||||||
issues: list[ValidationIssue] = []
|
issues: list[ValidationIssue] = []
|
||||||
warnings: list[ValidationIssue] = []
|
warnings: list[ValidationIssue] = []
|
||||||
|
|
||||||
|
# Collect video IDs actually referenced in the manuscript (for file-existence checks)
|
||||||
|
_VIDEO_PREFIXES = {
|
||||||
|
"video:": 6,
|
||||||
|
"vft:": 4, "vfb:": 4, "vfm:": 4,
|
||||||
|
"vf2t:": 5, "vf2b:": 5, "vf2m:": 5,
|
||||||
|
"vst:": 4, "vsb:": 4, "vsm:": 4,
|
||||||
|
"vftp:": 5, "vfbp:": 5, "vfmp:": 5,
|
||||||
|
"vf2tp:": 6, "vf2bp:": 6, "vf2mp:": 6,
|
||||||
|
"vstp:": 5, "vsbp:": 5, "vsmp:": 5,
|
||||||
|
}
|
||||||
|
referenced_video_ids: set[str] = set()
|
||||||
|
for marker in manuscript_markers:
|
||||||
|
prefix = next((p for p in _VIDEO_PREFIXES if marker.startswith(p)), None)
|
||||||
|
if prefix is not None:
|
||||||
|
referenced_video_ids.add(marker[_VIDEO_PREFIXES[prefix]:].lower())
|
||||||
|
elif marker.startswith("narration:"):
|
||||||
|
referenced_video_ids.add(marker[10:].lower())
|
||||||
|
|
||||||
|
# (Key-reuse across cutout/layer is legal: presentation is resolved per-occurrence
|
||||||
|
# from the shorthand prefix now — see transformer.resolve_video_presentation — so
|
||||||
|
# one handle can appear as vst: (above) and vsb: (below) without colliding.)
|
||||||
|
|
||||||
# Check for malformed markers first (these are likely typos)
|
# Check for malformed markers first (these are likely typos)
|
||||||
if malformed_markers:
|
if malformed_markers:
|
||||||
for line_num, marker_text in malformed_markers:
|
for line_num, marker_text in malformed_markers:
|
||||||
@@ -62,26 +86,11 @@ def validate_project(
|
|||||||
continue
|
continue
|
||||||
# Validate video trigger markers — both legacy [video:xxx] and
|
# Validate video trigger markers — both legacy [video:xxx] and
|
||||||
# shorthand [vft:xxx] / [vfb:xxx] / [vst:xxx] / [vsb:xxx].
|
# shorthand [vft:xxx] / [vfb:xxx] / [vst:xxx] / [vsb:xxx].
|
||||||
_VIDEO_PREFIXES = {
|
|
||||||
"video:": 6,
|
|
||||||
"vft:": 4,
|
|
||||||
"vfb:": 4,
|
|
||||||
"vf2t:": 5,
|
|
||||||
"vf2b:": 5,
|
|
||||||
"vst:": 4,
|
|
||||||
"vsb:": 4,
|
|
||||||
"vftp:": 5,
|
|
||||||
"vfbp:": 5,
|
|
||||||
"vf2tp:": 6,
|
|
||||||
"vf2bp:": 6,
|
|
||||||
"vstp:": 5,
|
|
||||||
"vsbp:": 5,
|
|
||||||
}
|
|
||||||
matched_prefix = next(
|
matched_prefix = next(
|
||||||
(p for p in _VIDEO_PREFIXES if marker.startswith(p)), None
|
(p for p in _VIDEO_PREFIXES if marker.startswith(p)), None
|
||||||
)
|
)
|
||||||
if matched_prefix is not None:
|
if matched_prefix is not None:
|
||||||
video_id = marker[_VIDEO_PREFIXES[matched_prefix] :]
|
video_id = marker[_VIDEO_PREFIXES[matched_prefix] :].lower()
|
||||||
if video_id not in videos:
|
if video_id not in videos:
|
||||||
hint = ""
|
hint = ""
|
||||||
if "." in video_id:
|
if "." in video_id:
|
||||||
@@ -94,11 +103,21 @@ def validate_project(
|
|||||||
project_path / "manuscript.txt",
|
project_path / "manuscript.txt",
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
else:
|
||||||
|
vs = videos[video_id]
|
||||||
|
if not vs.cutout or vs.cutout not in config.cutouts:
|
||||||
|
warnings.append(
|
||||||
|
ValidationIssue(
|
||||||
|
f"[{marker}] video '{video_id}' has no valid cutout in videos.json — "
|
||||||
|
f"run 'gnommo import' to project values, or set cutout manually.",
|
||||||
|
project_path / "manuscript.txt",
|
||||||
|
)
|
||||||
|
)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
# Validate narration trigger markers (narration:xxx) - continuous videos
|
# Validate narration trigger markers (narration:xxx) - continuous videos
|
||||||
if marker.startswith("narration:"):
|
if marker.startswith("narration:"):
|
||||||
video_id = marker[10:] # Remove 'narration:' prefix
|
video_id = marker[10:].lower() # Remove 'narration:' prefix
|
||||||
if video_id not in videos:
|
if video_id not in videos:
|
||||||
warnings.append(
|
warnings.append(
|
||||||
ValidationIssue(
|
ValidationIssue(
|
||||||
@@ -106,12 +125,26 @@ def validate_project(
|
|||||||
project_path / "manuscript.txt",
|
project_path / "manuscript.txt",
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
else:
|
||||||
|
vs = videos[video_id]
|
||||||
|
if not vs.cutout or vs.cutout not in config.cutouts:
|
||||||
|
warnings.append(
|
||||||
|
ValidationIssue(
|
||||||
|
f"[{marker}] video '{video_id}' has no valid cutout in videos.json — "
|
||||||
|
f"run 'gnommo import' to project values, or set cutout manually.",
|
||||||
|
project_path / "manuscript.txt",
|
||||||
|
)
|
||||||
|
)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
# Segment markers are structural annotations, not slide references
|
# Segment markers are structural annotations, not slide references
|
||||||
if marker.startswith("segment:"):
|
if marker.startswith("segment:"):
|
||||||
continue
|
continue
|
||||||
|
|
||||||
|
# Bare narrator cues (teleprompter hints, not pipeline markers)
|
||||||
|
if marker in ("pause", "stop"):
|
||||||
|
continue
|
||||||
|
|
||||||
# Unknown namespaced markers (e.g. [background:xxx]) — not supported, ignore with warning
|
# Unknown namespaced markers (e.g. [background:xxx]) — not supported, ignore with warning
|
||||||
if ":" in marker:
|
if ":" in marker:
|
||||||
warnings.append(
|
warnings.append(
|
||||||
@@ -168,6 +201,10 @@ def validate_project(
|
|||||||
shared_assets_dir = project_path.parent / "shared_assets"
|
shared_assets_dir = project_path.parent / "shared_assets"
|
||||||
|
|
||||||
for video_id, video_source in videos.items():
|
for video_id, video_source in videos.items():
|
||||||
|
# Only check files for videos actually used in this manuscript
|
||||||
|
if video_id not in referenced_video_ids:
|
||||||
|
continue
|
||||||
|
|
||||||
# Determine base directory based on is_shared flag
|
# Determine base directory based on is_shared flag
|
||||||
if video_source.is_shared:
|
if video_source.is_shared:
|
||||||
if shared_assets_dir:
|
if shared_assets_dir:
|
||||||
@@ -186,9 +223,15 @@ def validate_project(
|
|||||||
video_path = base_dir / video_source.source_file
|
video_path = base_dir / video_source.source_file
|
||||||
video_path, _ = resolve_with_cache(video_path, project_path)
|
video_path, _ = resolve_with_cache(video_path, project_path)
|
||||||
if not video_path.exists():
|
if not video_path.exists():
|
||||||
|
sf = video_source.source_file
|
||||||
|
hint = (
|
||||||
|
" — run 'gnommo pexels' to download"
|
||||||
|
if sf.startswith("pexels/")
|
||||||
|
else " — falling back to PlaceholderVideo"
|
||||||
|
)
|
||||||
warnings.append(
|
warnings.append(
|
||||||
ValidationIssue(
|
ValidationIssue(
|
||||||
f"Video file not found: {video_source.source_file} — falling back to PlaceholderVideo",
|
f"Video file not found: {sf}{hint}",
|
||||||
videos_json_path,
|
videos_json_path,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
@@ -229,6 +272,7 @@ def validate_project(
|
|||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
bg_path = shared_assets_dir / bg_videos[bg_handle]["source_file"]
|
bg_path = shared_assets_dir / bg_videos[bg_handle]["source_file"]
|
||||||
|
bg_path, _ = resolve_with_cache(bg_path, project_path)
|
||||||
if not bg_path.exists():
|
if not bg_path.exists():
|
||||||
issues.append(
|
issues.append(
|
||||||
ValidationIssue(
|
ValidationIssue(
|
||||||
@@ -272,6 +316,40 @@ def validate_project(
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Check outro videos exist in videos.json or shared_assets
|
||||||
|
if config.outro:
|
||||||
|
missing_outro = [vid_id for vid_id in config.outro if vid_id not in videos]
|
||||||
|
if missing_outro:
|
||||||
|
found = resolve_missing_videos(missing_outro, project_path, config)
|
||||||
|
still_missing = [vid_id for vid_id in missing_outro if vid_id not in found]
|
||||||
|
for vid_id in still_missing:
|
||||||
|
warnings.append(
|
||||||
|
ValidationIssue(
|
||||||
|
f"Outro video '{vid_id}' not found in videos.json or shared_assets — will be skipped at render",
|
||||||
|
project_path / "project.json",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Check all audio: markers in manuscript exist in audio.json
|
||||||
|
if audio is not None:
|
||||||
|
seen_audio: set[str] = set()
|
||||||
|
manuscript_path = project_path / "manuscript.txt"
|
||||||
|
for marker in manuscript_markers:
|
||||||
|
audio_id = None
|
||||||
|
if marker.startswith("audio:"):
|
||||||
|
audio_id = marker[6:]
|
||||||
|
elif marker.startswith("A") and len(marker) > 1 and marker[1:].isalnum():
|
||||||
|
audio_id = marker[1:]
|
||||||
|
if audio_id and audio_id not in seen_audio:
|
||||||
|
seen_audio.add(audio_id)
|
||||||
|
if audio_id not in audio:
|
||||||
|
warnings.append(
|
||||||
|
ValidationIssue(
|
||||||
|
f"Audio marker [{marker}] referenced in manuscript but '{audio_id}' not defined in audio.json — will be silent at render",
|
||||||
|
manuscript_path,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
# If any issues, raise ValidationError
|
# If any issues, raise ValidationError
|
||||||
if issues:
|
if issues:
|
||||||
raise ValidationError(issues)
|
raise ValidationError(issues)
|
||||||
|
|||||||
Executable
+35
@@ -0,0 +1,35 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
|
||||||
|
./gnommo.sh -p video0 grade --stage key
|
||||||
|
./gnommo.sh -p video1 grade --stage key
|
||||||
|
./gnommo.sh -p video2 grade --stage key
|
||||||
|
./gnommo.sh -p video3 grade --stage key
|
||||||
|
./gnommo.sh -p video4 grade --stage key
|
||||||
|
./gnommo.sh -p video5 grade --stage key
|
||||||
|
./gnommo.sh -p video6 grade --stage key
|
||||||
|
|
||||||
|
|
||||||
|
./gnommo.sh -p video0 grade --pick key_5
|
||||||
|
./gnommo.sh -p video1 grade --pick key_5
|
||||||
|
./gnommo.sh -p video2 grade --pick key_5
|
||||||
|
./gnommo.sh -p video3 grade --pick key_5
|
||||||
|
./gnommo.sh -p video4 grade --pick key_5
|
||||||
|
./gnommo.sh -p video5 grade --pick key_5
|
||||||
|
./gnommo.sh -p video6 grade --pick key_5
|
||||||
|
|
||||||
|
./gnommo.sh -p video0 grade --stage grade
|
||||||
|
./gnommo.sh -p video1 grade --stage grade
|
||||||
|
./gnommo.sh -p video2 grade --stage grade
|
||||||
|
./gnommo.sh -p video3 grade --stage grade
|
||||||
|
./gnommo.sh -p video4 grade --stage grade
|
||||||
|
./gnommo.sh -p video5 grade --stage grade
|
||||||
|
./gnommo.sh -p video6 grade --stage grade
|
||||||
|
|
||||||
|
./gnommo.sh -p video0 grade --pick grade_5
|
||||||
|
./gnommo.sh -p video1 grade --pick grade_5
|
||||||
|
./gnommo.sh -p video2 grade --pick grade_5
|
||||||
|
./gnommo.sh -p video3 grade --pick grade_5
|
||||||
|
./gnommo.sh -p video4 grade --pick grade_5
|
||||||
|
./gnommo.sh -p video5 grade --pick grade_5
|
||||||
|
./gnommo.sh -p video6 grade --pick grade_5
|
||||||
Executable
+9
@@ -0,0 +1,9 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
|
||||||
|
./gnommo.sh -p video1 render --force --prod
|
||||||
|
./gnommo.sh -p video2 render --force --prod
|
||||||
|
./gnommo.sh -p video3 render --force --prod
|
||||||
|
./gnommo.sh -p video4 render --force --prod
|
||||||
|
./gnommo.sh -p video5 render --force --prod
|
||||||
|
#./gnommo.sh -p video6 render --force --prod
|
||||||
|
|
||||||
@@ -21,9 +21,9 @@
|
|||||||
"defaultSlideType": "fullscreen",
|
"defaultSlideType": "fullscreen",
|
||||||
"cutouts": {
|
"cutouts": {
|
||||||
"talkinghead": {
|
"talkinghead": {
|
||||||
"x": "-23%",
|
"x": "-10%",
|
||||||
"y": "10%",
|
"y": "40%",
|
||||||
"height": "90%"
|
"height": "80%"
|
||||||
},
|
},
|
||||||
"fullscreen": {
|
"fullscreen": {
|
||||||
"x": "0%",
|
"x": "0%",
|
||||||
|
|||||||
@@ -1,476 +0,0 @@
|
|||||||
# Gnommo Feature Development Roadmap
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
Features to standardize the Keynote-to-YouTube workflow, so that once the presentation is complete, only a standardized recording session stands between you and a finished video.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Video Description Generator
|
|
||||||
|
|
||||||
**Command:** `gnommo -p <project> description`
|
|
||||||
|
|
||||||
Generate a complete YouTube description with citations, attributions, and chapters.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 1.1 Manuscript Citations (`[cite:...]`)
|
|
||||||
|
|
||||||
Citations embedded in the manuscript represent sources, references, or links mentioned during narration. The text after `cite:` is the **literal reference** that should appear in the description.
|
|
||||||
|
|
||||||
**Format in manuscript.txt:**
|
|
||||||
```
|
|
||||||
[cite:Reference text exactly as it should appear]
|
|
||||||
```
|
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
```
|
|
||||||
[S3]
|
|
||||||
According to this study [cite:Smith et al. (2024) "Effects of AI on Productivity" - https://example.com/paper],
|
|
||||||
the effect is significant.
|
|
||||||
|
|
||||||
[S7]
|
|
||||||
I'm using [cite:Keynote by Apple - https://apple.com/keynote] for all my presentations.
|
|
||||||
|
|
||||||
[S12]
|
|
||||||
This technique was pioneered by [cite:Dr. Jane Doe, MIT Media Lab].
|
|
||||||
```
|
|
||||||
|
|
||||||
**Output in description:**
|
|
||||||
```
|
|
||||||
SOURCES & REFERENCES
|
|
||||||
━━━━━━━━━━━━━━━━━━━━
|
|
||||||
1:23 - Smith et al. (2024) "Effects of AI on Productivity" - https://example.com/paper
|
|
||||||
4:56 - Keynote by Apple - https://apple.com/keynote
|
|
||||||
8:30 - Dr. Jane Doe, MIT Media Lab
|
|
||||||
```
|
|
||||||
|
|
||||||
**Requirements:**
|
|
||||||
- Parse `[cite:...]` markers from manuscript.txt
|
|
||||||
- Extract the literal text after `cite:` as the reference
|
|
||||||
- Align citations to timestamps (same fuzzy matching as other markers)
|
|
||||||
- Group citations in order of appearance
|
|
||||||
- Citations are NOT aligned for rendering (ignored by renderer) but ARE timestamped for description
|
|
||||||
|
|
||||||
**Note:** `[cite:...]` markers should not affect video rendering or narration alignment - they are metadata-only markers for description generation.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 1.2 Pexels/Stock Footage Attribution
|
|
||||||
|
|
||||||
Attribution for Pexels content is **not legally required** but is appreciated and professional.
|
|
||||||
|
|
||||||
**Official Pexels attribution format:**
|
|
||||||
```
|
|
||||||
by [Contributor Name] via Pexels
|
|
||||||
```
|
|
||||||
|
|
||||||
**Implementation:**
|
|
||||||
- Extend `videos.json` to include attribution metadata:
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"beach_waves": {
|
|
||||||
"source_file": "pexels/beach.mp4",
|
|
||||||
"is_shared": true,
|
|
||||||
"attribution": {
|
|
||||||
"source": "pexels",
|
|
||||||
"creator": "John Doe",
|
|
||||||
"url": "https://pexels.com/video/12345"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
- Auto-detect Pexels videos from `shared_assets/pexels/` folder
|
|
||||||
- Support Pexels metadata JSON files (if downloaded with video)
|
|
||||||
- Generate attribution section for video description:
|
|
||||||
```
|
|
||||||
STOCK FOOTAGE
|
|
||||||
━━━━━━━━━━━━━
|
|
||||||
Beach waves by John Doe via Pexels: https://pexels.com/video/12345
|
|
||||||
City timelapse by Jane Smith via Pexels: https://pexels.com/video/67890
|
|
||||||
```
|
|
||||||
|
|
||||||
**Pexels License Notes** (from pexels.com/license):
|
|
||||||
- Free for personal and commercial use
|
|
||||||
- Attribution not required but appreciated
|
|
||||||
- Cannot sell unaltered copies
|
|
||||||
- Cannot redistribute on other stock platforms
|
|
||||||
|
|
||||||
### 1.3 Complete Description Output
|
|
||||||
|
|
||||||
**Output file:** `out/description_youtube.txt`
|
|
||||||
|
|
||||||
Combine all elements into a ready-to-paste YouTube description.
|
|
||||||
|
|
||||||
**Structure:**
|
|
||||||
```
|
|
||||||
[Video description from project.json "description" field]
|
|
||||||
|
|
||||||
CHAPTERS
|
|
||||||
━━━━━━━━
|
|
||||||
0:00 Introduction
|
|
||||||
1:23 Topic One
|
|
||||||
3:45 Topic Two
|
|
||||||
...
|
|
||||||
|
|
||||||
REFERENCES
|
|
||||||
━━━━━━━━━━
|
|
||||||
1:23 - Smith et al. (2024) "AI Study" - https://example.com
|
|
||||||
4:56 - Keynote by Apple - https://apple.com/keynote
|
|
||||||
...
|
|
||||||
|
|
||||||
STOCK FOOTAGE
|
|
||||||
━━━━━━━━━━━━━
|
|
||||||
Beach waves by John Doe via Pexels: https://pexels.com/video/12345
|
|
||||||
...
|
|
||||||
|
|
||||||
[Optional footer from project.json "footer" field - social links, subscribe CTA, etc.]
|
|
||||||
```
|
|
||||||
|
|
||||||
**project.json additions:**
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"description": "In this video, I walk through the complete Gnommo workflow for creating YouTube videos from Keynote presentations.",
|
|
||||||
"footer": "Subscribe for more tutorials: https://youtube.com/@channel\nTwitter: https://twitter.com/handle"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Requirements:**
|
|
||||||
- Pull video description from `project.json` "description" field
|
|
||||||
- Generate chapters from slide markers (see Section 2)
|
|
||||||
- Collect all `[cite:...]` references with timestamps
|
|
||||||
- Collect all Pexels/stock attributions from `videos.json`
|
|
||||||
- Append optional footer from `project.json` "footer" field
|
|
||||||
- Output to `out/description_youtube.txt`
|
|
||||||
- Sections with no content are omitted (e.g., no STOCK FOOTAGE section if none used)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. YouTube Chapter Markers
|
|
||||||
|
|
||||||
**Command:** `gnommo -p <project> chapters`
|
|
||||||
|
|
||||||
Auto-generate chapter timestamps from slide markers.
|
|
||||||
|
|
||||||
**Requirements:**
|
|
||||||
- Extract chapter titles from:
|
|
||||||
- Keynote slide titles (via presenter notes import)
|
|
||||||
- First sentence after each `[SN]` marker
|
|
||||||
- Optional `[chapter:Title]` markers for explicit chapter names
|
|
||||||
- Calculate timestamps from aligned marker timings
|
|
||||||
- Output copy-paste ready format:
|
|
||||||
```
|
|
||||||
CHAPTERS
|
|
||||||
━━━━━━━━
|
|
||||||
0:00 Introduction
|
|
||||||
1:23 What is Gnommo?
|
|
||||||
3:45 Setting Up Your Project
|
|
||||||
7:12 Recording Tips
|
|
||||||
10:30 Rendering Your Video
|
|
||||||
12:45 Outro
|
|
||||||
```
|
|
||||||
- Option to merge small chapters (minimum duration threshold)
|
|
||||||
- Support for nested chapters (main topics + subtopics)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Subtitle/Caption Export
|
|
||||||
|
|
||||||
**Command:** `gnommo -p <project> subtitles`
|
|
||||||
|
|
||||||
Generate subtitle files from Whisper transcription.
|
|
||||||
|
|
||||||
**Requirements:**
|
|
||||||
- Export formats: SRT, VTT, TXT
|
|
||||||
- Use existing word-level timestamps from transcription
|
|
||||||
- Smart line breaking (max characters per line, break at punctuation)
|
|
||||||
- Speaker diarization support (future: multiple speakers)
|
|
||||||
- Options:
|
|
||||||
- `--format srt|vtt|txt`
|
|
||||||
- `--max-chars 42` (characters per line)
|
|
||||||
- `--max-duration 5` (seconds per subtitle block)
|
|
||||||
|
|
||||||
**Example output (SRT):**
|
|
||||||
```
|
|
||||||
1
|
|
||||||
00:00:01,500 --> 00:00:04,200
|
|
||||||
Hello and welcome to this tutorial
|
|
||||||
on video editing with Gnommo.
|
|
||||||
|
|
||||||
2
|
|
||||||
00:00:04,500 --> 00:00:07,800
|
|
||||||
Today we're going to cover
|
|
||||||
the complete workflow.
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Thumbnail Generation
|
|
||||||
|
|
||||||
**Command:** `gnommo -p <project> thumbnail`
|
|
||||||
|
|
||||||
Auto-generate thumbnail candidates from slides.
|
|
||||||
|
|
||||||
**Requirements:**
|
|
||||||
- Designate thumbnail slides with `[thumbnail]` marker
|
|
||||||
- If no marker, use slide 1 or title slide
|
|
||||||
- Apply text overlays from config:
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"thumbnail": {
|
|
||||||
"title_text": "Episode ${episode_number}",
|
|
||||||
"subtitle_text": "${title}",
|
|
||||||
"font": "Impact",
|
|
||||||
"text_color": "#FFFFFF",
|
|
||||||
"outline_color": "#000000",
|
|
||||||
"position": "bottom-left"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
- Generate multiple variants:
|
|
||||||
- With/without text overlay
|
|
||||||
- Different zoom levels
|
|
||||||
- Different color treatments (saturated, high contrast)
|
|
||||||
- Output to `out/thumbnails/` folder
|
|
||||||
- Resolution: 1280x720 (YouTube standard)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. Intro/Outro Templates
|
|
||||||
|
|
||||||
**Configuration in project.json:**
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"intro": {
|
|
||||||
"template": "templates/intro_v2.mp4",
|
|
||||||
"duration": 3.5,
|
|
||||||
"transition": "fade",
|
|
||||||
"variables": {
|
|
||||||
"episode_number": "12",
|
|
||||||
"title": "Getting Started with Gnommo"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"outro": {
|
|
||||||
"template": "templates/outro_subscribe.mp4",
|
|
||||||
"duration": 8.0,
|
|
||||||
"transition": "fade"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Requirements:**
|
|
||||||
- Define intro/outro templates in `shared_assets/templates/`
|
|
||||||
- Auto-prepend intro before first slide
|
|
||||||
- Auto-append outro after last slide
|
|
||||||
- Support variable substitution in templates (episode number, title)
|
|
||||||
- Configurable transition types (fade, cut, wipe)
|
|
||||||
- End screen safe zone support (last 20 seconds)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6. Multi-Platform Format Presets
|
|
||||||
|
|
||||||
**Command:** `gnommo -p <project> render --format <preset>`
|
|
||||||
|
|
||||||
**Presets:**
|
|
||||||
| Preset | Aspect | Resolution | Notes |
|
|
||||||
|--------|--------|------------|-------|
|
|
||||||
| `youtube` | 16:9 | 1920x1080 | Default, standard horizontal |
|
|
||||||
| `youtube-4k` | 16:9 | 3840x2160 | 4K export |
|
|
||||||
| `shorts` | 9:16 | 1080x1920 | Vertical, auto-reframe slides |
|
|
||||||
| `podcast` | - | Audio only | MP3/M4A export for podcast feeds |
|
|
||||||
| `square` | 1:1 | 1080x1080 | Instagram/LinkedIn |
|
|
||||||
|
|
||||||
**Requirements:**
|
|
||||||
- Auto-adjust cutout positions per format
|
|
||||||
- Smart slide reframing for vertical (zoom to content area)
|
|
||||||
- Separate output folders per format
|
|
||||||
- Batch export to multiple formats: `--format youtube,shorts,podcast`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7. Teleprompter Script Generation
|
|
||||||
|
|
||||||
**Command:** `gnommo -p <project> teleprompter`
|
|
||||||
|
|
||||||
Extract clean narration text for teleprompter display.
|
|
||||||
|
|
||||||
**Requirements:**
|
|
||||||
- Strip all markers from manuscript
|
|
||||||
- Keep only spoken text
|
|
||||||
- Output formats:
|
|
||||||
- `--format txt` - Plain text
|
|
||||||
- `--format html` - Scrollable HTML page with large font
|
|
||||||
- `--format json` - For teleprompter apps
|
|
||||||
- Optional: Include slide thumbnails as visual cues
|
|
||||||
- Configurable font size and scroll speed hints
|
|
||||||
|
|
||||||
**Example HTML output:**
|
|
||||||
```html
|
|
||||||
<div class="teleprompter">
|
|
||||||
<p class="cue">[SLIDE: Introduction]</p>
|
|
||||||
<p>Hello and welcome to this tutorial on video editing with Gnommo.</p>
|
|
||||||
<p class="cue">[SLIDE: What is Gnommo?]</p>
|
|
||||||
<p>Gnommo is a code-first video editing pipeline...</p>
|
|
||||||
</div>
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 8. Recording Checklist Generator
|
|
||||||
|
|
||||||
**Command:** `gnommo -p <project> checklist`
|
|
||||||
|
|
||||||
Generate a pre-recording checklist based on project configuration.
|
|
||||||
|
|
||||||
**Output includes:**
|
|
||||||
- [ ] Camera settings (resolution, fps from project.json)
|
|
||||||
- [ ] Lighting setup (if green screen detected in videos.json)
|
|
||||||
- [ ] Audio check (microphone levels)
|
|
||||||
- [ ] Props/demos needed (parsed from `[video:...]` markers)
|
|
||||||
- [ ] Slide count and estimated duration
|
|
||||||
- [ ] Teleprompter ready
|
|
||||||
- [ ] Recording space clear
|
|
||||||
|
|
||||||
**Customizable via `checklist_template.md` in project folder.**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 9. Audio Normalization
|
|
||||||
|
|
||||||
**Automatic during render or standalone command:**
|
|
||||||
`gnommo -p <project> normalize`
|
|
||||||
|
|
||||||
**Requirements:**
|
|
||||||
- Target: -14 LUFS (YouTube standard)
|
|
||||||
- Apply loudness normalization to narration track
|
|
||||||
- Preserve dynamic range (avoid over-compression)
|
|
||||||
- Normalize intro/outro audio to match
|
|
||||||
- Option: `--target-lufs -14`
|
|
||||||
|
|
||||||
**Implementation:**
|
|
||||||
- Use FFmpeg `loudnorm` filter
|
|
||||||
- Two-pass normalization for accurate results
|
|
||||||
- Report before/after levels
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 10. Project Templates
|
|
||||||
|
|
||||||
**Command:** `gnommo init <project-name> --template <template>`
|
|
||||||
|
|
||||||
**Built-in templates:**
|
|
||||||
| Template | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| `tutorial` | Talking head + slides, square slide layout |
|
|
||||||
| `explainer` | Full-screen slides, minimal presenter |
|
|
||||||
| `review` | Product review format, multiple camera angles |
|
|
||||||
| `talking-head` | Full-screen presenter, no slides |
|
|
||||||
| `screencast` | Screen recording with small presenter PIP |
|
|
||||||
|
|
||||||
**Requirements:**
|
|
||||||
- Templates stored in `~/.gnommo/templates/` or `shared_assets/templates/`
|
|
||||||
- Each template includes:
|
|
||||||
- `project.json` with preset cutouts and settings
|
|
||||||
- `manuscript.txt` skeleton with example markers
|
|
||||||
- Sample `videos.json` structure
|
|
||||||
- User can create custom templates: `gnommo template save <name>`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 11. Batch Processing
|
|
||||||
|
|
||||||
**Command:** `gnommo batch render project1 project2 project3`
|
|
||||||
|
|
||||||
**Requirements:**
|
|
||||||
- Process multiple projects in sequence
|
|
||||||
- Continue on failure (don't stop batch for one failed project)
|
|
||||||
- Summary report at end:
|
|
||||||
```
|
|
||||||
BATCH COMPLETE
|
|
||||||
━━━━━━━━━━━━━━
|
|
||||||
✓ project1 - rendered in 5:23
|
|
||||||
✓ project2 - rendered in 4:17
|
|
||||||
✗ project3 - failed (missing slide S12)
|
|
||||||
```
|
|
||||||
- Options:
|
|
||||||
- `--parallel 2` - Run N renders in parallel
|
|
||||||
- `--skip-existing` - Skip if `out/final.mp4` exists
|
|
||||||
- `--format youtube,shorts` - Render all formats for each project
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 12. Progress Dashboard
|
|
||||||
|
|
||||||
**Command:** `gnommo status` or `gnommo -p <project> status`
|
|
||||||
|
|
||||||
Display pipeline status for all projects or specific project.
|
|
||||||
|
|
||||||
**Output:**
|
|
||||||
```
|
|
||||||
PROJECT STATUS
|
|
||||||
━━━━━━━━━━━━━━
|
|
||||||
Project Import Preprocess Transcribe Render Output
|
|
||||||
─────────────────────────────────────────────────────────────
|
|
||||||
video1 ✓ ✓ ✓ ✓ final.mp4 (12:34)
|
|
||||||
video2 ✓ ✓ ✓ ✗ -
|
|
||||||
video3 ✓ ✗ - - -
|
|
||||||
video4 ✗ - - - -
|
|
||||||
```
|
|
||||||
|
|
||||||
**Requirements:**
|
|
||||||
- Scan all project directories
|
|
||||||
- Check for existence of intermediate files
|
|
||||||
- Show file timestamps and durations
|
|
||||||
- Highlight what needs to be done next
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 13. Recording Session Mode (Future)
|
|
||||||
|
|
||||||
**Command:** `gnommo -p <project> session`
|
|
||||||
|
|
||||||
Live recording assistant mode.
|
|
||||||
|
|
||||||
**Features:**
|
|
||||||
- Display current slide on secondary monitor
|
|
||||||
- Show teleprompter text overlay
|
|
||||||
- Keyboard shortcuts to advance slides
|
|
||||||
- Real-time recording with proper settings
|
|
||||||
- Auto-stop at end of manuscript
|
|
||||||
- Voice command support: "next slide", "pause"
|
|
||||||
|
|
||||||
**Note:** This is a stretch goal requiring significant UI work.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Implementation Priority
|
|
||||||
|
|
||||||
### Phase 1 - Core YouTube Workflow (High Impact)
|
|
||||||
1. **Video Description Generator** (citations + Pexels attribution)
|
|
||||||
2. **YouTube Chapter Markers**
|
|
||||||
3. **Subtitle/Caption Export**
|
|
||||||
4. **Audio Normalization**
|
|
||||||
|
|
||||||
### Phase 2 - Content Creation Efficiency
|
|
||||||
5. **Thumbnail Generation**
|
|
||||||
6. **Intro/Outro Templates**
|
|
||||||
7. **Teleprompter Script Generation**
|
|
||||||
8. **Recording Checklist Generator**
|
|
||||||
|
|
||||||
### Phase 3 - Scale & Automation
|
|
||||||
9. **Project Templates**
|
|
||||||
10. **Multi-Platform Format Presets**
|
|
||||||
11. **Batch Processing**
|
|
||||||
12. **Progress Dashboard**
|
|
||||||
|
|
||||||
### Phase 4 - Advanced
|
|
||||||
13. **Recording Session Mode**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- All new commands should follow existing CLI pattern: `gnommo -p <project> <command>`
|
|
||||||
- Output files go to `out/` subdirectory by default
|
|
||||||
- All features should support `--dry-run` where applicable
|
|
||||||
- Verbose mode (`-v`) should show detailed progress
|
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Plan-level validation for chunking v2 (docs/chunking_v2.md).
|
||||||
|
|
||||||
|
Verifies the transformer now INCLUDES clips that span a chunk boundary and seeks
|
||||||
|
into them (skip_override / src_offset), instead of the v1 behaviour that dropped
|
||||||
|
them. This is a pure plan-level check — the ffmpeg concat-seam (frame alignment via
|
||||||
|
-c copy) still needs a real render on the rig to confirm.
|
||||||
|
|
||||||
|
Run: ./venv/bin/python tests/test_chunking_v2.py
|
||||||
|
"""
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||||
|
|
||||||
|
from gnommo.transformer import (
|
||||||
|
MarkerTiming,
|
||||||
|
_extract_audio_events,
|
||||||
|
_extract_video_events,
|
||||||
|
AUDIO_OFFSET_SECONDS,
|
||||||
|
)
|
||||||
|
from gnommo.models import AudioDefinition, VideoSource, CutoutDefinition, SlideDefinition
|
||||||
|
|
||||||
|
_fails = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, cond, detail=""):
|
||||||
|
print(f" {'PASS' if cond else 'FAIL'} {name}" + (f" — {detail}" if detail and not cond else ""))
|
||||||
|
if not cond:
|
||||||
|
_fails.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
# ── audio ────────────────────────────────────────────────────────────────────
|
||||||
|
def test_audio():
|
||||||
|
print("audio events:")
|
||||||
|
audio = {
|
||||||
|
"music": AudioDefinition(file="music.mp3", loop=True, duration=90.0),
|
||||||
|
"sfx": AudioDefinition(file="sfx.wav", loop=False, duration=500.0),
|
||||||
|
"blip": AudioDefinition(file="blip.wav", loop=False, duration=50.0),
|
||||||
|
}
|
||||||
|
# music triggers at t=0, sfx at t=10, blip at t=10
|
||||||
|
markers = [
|
||||||
|
MarkerTiming(marker_id="Amusic", timestamp=0.0, context="", confidence=1.0),
|
||||||
|
MarkerTiming(marker_id="Asfx", timestamp=10.0, context="", confidence=1.0),
|
||||||
|
MarkerTiming(marker_id="Ablip", timestamp=10.0, context="", confidence=1.0),
|
||||||
|
]
|
||||||
|
|
||||||
|
# Chunk window [300, 600): all three started earlier.
|
||||||
|
evs = {e.audio_id: e for e in _extract_audio_events(markers, audio, time_range=(300.0, 600.0))}
|
||||||
|
|
||||||
|
# Looping music: included, resumes at loop phase. astart = max(0, 0-1)=0; into=300;
|
||||||
|
# phase = 300 % 90 = 30.
|
||||||
|
check("looping music spanning boundary is INCLUDED (v1 dropped it)", "music" in evs)
|
||||||
|
if "music" in evs:
|
||||||
|
m = evs["music"]
|
||||||
|
check("music clamped to window start", abs(m.start_time - 300.0) < 1e-6, f"start={m.start_time}")
|
||||||
|
check("music seeks to loop phase 30.0", abs(m.src_offset - 30.0) < 1e-6, f"src_offset={m.src_offset}")
|
||||||
|
|
||||||
|
# One-shot still playing at the window: included, linear seek.
|
||||||
|
# astart = max(0,10-1)=9; aend=9+500=509 > 300 → spans. into=300-9=291.
|
||||||
|
check("one-shot still playing is INCLUDED", "sfx" in evs)
|
||||||
|
if "sfx" in evs:
|
||||||
|
s = evs["sfx"]
|
||||||
|
check("sfx linear seek 291.0", abs(s.src_offset - 291.0) < 1e-6, f"src_offset={s.src_offset}")
|
||||||
|
|
||||||
|
# One-shot that ended before the window: excluded (aend=9+50=59 < 300).
|
||||||
|
check("one-shot ended before window is EXCLUDED", "blip" not in evs)
|
||||||
|
|
||||||
|
# Full render (no range): everything from the start, no seek.
|
||||||
|
full = {e.audio_id: e for e in _extract_audio_events(markers, audio, time_range=None)}
|
||||||
|
check("full render includes music with no seek", "music" in full and full["music"].src_offset == 0.0)
|
||||||
|
|
||||||
|
|
||||||
|
# ── video ────────────────────────────────────────────────────────────────────
|
||||||
|
def test_video():
|
||||||
|
print("video events:")
|
||||||
|
cutouts = {"fullscreen": CutoutDefinition(x=0, y=0, height=1080, width=1920)}
|
||||||
|
videos = {
|
||||||
|
"bg": VideoSource(
|
||||||
|
source_file="bg.mp4", cutout="fullscreen", layer="below",
|
||||||
|
duration=90.0, skip=0.0, end_on="next_video",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
slides = {f"S{i}": SlideDefinition(image=f"S{i}.png", type="slide") for i in range(1, 11)}
|
||||||
|
markers = [MarkerTiming(marker_id=f"S{i}", timestamp=(i - 1) * 60.0, context="", confidence=1.0)
|
||||||
|
for i in range(1, 11)]
|
||||||
|
# Background overlay starts at slide-7 time (360) and, as the only video with
|
||||||
|
# end_on next_video, runs to total_duration (600).
|
||||||
|
markers.append(MarkerTiming(marker_id="vfm:bg", timestamp=360.0, context="", confidence=1.0))
|
||||||
|
|
||||||
|
total = 600.0
|
||||||
|
# Chunk window that STARTS AFTER the video began: [420, 600) (slides 8-10).
|
||||||
|
evs, _ = _extract_video_events(markers, videos, cutouts, slides, total, time_range=(420.0, 600.0))
|
||||||
|
bg = next((e for e in evs if e.video_id == "bg"), None)
|
||||||
|
|
||||||
|
check("spanning background video is INCLUDED in the later chunk (v1 dropped it)", bg is not None)
|
||||||
|
if bg is not None:
|
||||||
|
check("bg clamped to window start", abs(bg.start_time - 420.0) < 1e-6, f"start={bg.start_time}")
|
||||||
|
# into = 420-360 = 60; playable = 90-0 = 90; 60 < 90 → linear seek 60.
|
||||||
|
check("bg linear seek 60.0 (first play-through)", abs((bg.skip_override or 0) - 60.0) < 1e-6,
|
||||||
|
f"skip_override={bg.skip_override}")
|
||||||
|
|
||||||
|
# Window starting deep enough that the 90s clip has looped once: [480, 600).
|
||||||
|
# into = 480-360 = 120; 120 >= 90 → phase = 120 % 90 = 30.
|
||||||
|
evs2, _ = _extract_video_events(markers, videos, cutouts, slides, total, time_range=(480.0, 600.0))
|
||||||
|
bg2 = next((e for e in evs2 if e.video_id == "bg"), None)
|
||||||
|
check("looped background resumes at phase 30.0", bg2 is not None and abs((bg2.skip_override or 0) - 30.0) < 1e-6,
|
||||||
|
f"skip_override={getattr(bg2, 'skip_override', None)}")
|
||||||
|
|
||||||
|
# Full render: no seek.
|
||||||
|
evs3, _ = _extract_video_events(markers, videos, cutouts, slides, total, time_range=None)
|
||||||
|
bg3 = next((e for e in evs3 if e.video_id == "bg"), None)
|
||||||
|
check("full render includes bg with no seek", bg3 is not None and bg3.skip_override is None)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
test_audio()
|
||||||
|
test_video()
|
||||||
|
print()
|
||||||
|
if _fails:
|
||||||
|
print(f"FAILED: {len(_fails)} check(s): {', '.join(_fails)}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("All chunking-v2 plan-level checks passed.")
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
|
||||||
|
./gnommo.sh -p video0 import
|
||||||
|
./gnommo.sh -p video1 import
|
||||||
|
./gnommo.sh -p video2 import
|
||||||
|
./gnommo.sh -p video3 import
|
||||||
|
./gnommo.sh -p video4 import
|
||||||
|
./gnommo.sh -p video5 import
|
||||||
|
./gnommo.sh -p video6 import
|
||||||
|
|
||||||
|
./gnommo.sh -p video0 prune
|
||||||
|
./gnommo.sh -p video1 prune
|
||||||
|
./gnommo.sh -p video2 prune
|
||||||
|
./gnommo.sh -p video3 prune
|
||||||
|
./gnommo.sh -p video4 prune
|
||||||
|
./gnommo.sh -p video5 prune
|
||||||
|
./gnommo.sh -p video6 prune
|
||||||
|
|
||||||
|
|
||||||
|
./gnommo.sh -p video0 up
|
||||||
|
./gnommo.sh -p video1 up
|
||||||
|
./gnommo.sh -p video2 up
|
||||||
|
./gnommo.sh -p video3 up
|
||||||
|
./gnommo.sh -p video4 up
|
||||||
|
./gnommo.sh -p video5 up
|
||||||
|
./gnommo.sh -p video6 up
|
||||||
|
|
||||||
Reference in New Issue
Block a user