492 lines
16 KiB
Markdown
492 lines
16 KiB
Markdown
# Gnommo
|
||
|
||
Gnommo is ADHD friendly video-editor for coders.
|
||
|
||
1. Design the presentation in keynote
|
||
2. Set up the greenscreen and audio settings once
|
||
3. Automatically times slides and videos to your voice.
|
||
4. Limited options means you waste less time on stuff that isn't important.
|
||
|
||
A code-first video editing pipeline for creating narrated presentations with slides, video overlays, and synchronized audio.
|
||
|
||
## Quick Start
|
||
|
||
```bash
|
||
# Create a project
|
||
gnommo -p myproject init
|
||
|
||
# Import slides and presenter notes from Keynote file
|
||
gnommo -p myproject import
|
||
|
||
# Process the narration videos with video and audio filters
|
||
gnommo -p myproject pre
|
||
|
||
# Stitch together the narration segments to one full length narration.
|
||
gnommo -p myproject stitch
|
||
|
||
# Transcribe the actual narrated content
|
||
gnommo -p myproject transcribe
|
||
|
||
# Generate the final video
|
||
gnommo -p myproject render
|
||
|
||
# Generate the final youtube assets. Manuscript file, description
|
||
gnommo -p myproject youtubeready
|
||
|
||
# Free up disk space locally by saving your project to an external drive
|
||
gnommo -p myproject archive
|
||
```
|
||
|
||
## Resolution modes
|
||
|
||
All commands accept `--res` to trade quality for speed during iteration:
|
||
|
||
| Flag | Resolution | Use case |
|
||
|---|---|---|
|
||
| `--res full` | Project resolution (default) | Final output |
|
||
| `--res low` | 490×270 | Fast preview render |
|
||
| `--res tiny` | 320×180 | Ultrafast iteration (preprocess, stitch, render) |
|
||
|
||
`--res tiny` and `--res low` create downscaled copies of source files in subdirectories (`proxy/` and `low/` respectively) and work from those. The originals are never modified.
|
||
|
||
```bash
|
||
gnommo -p myproject pre --res tiny # fast preprocess
|
||
gnommo -p myproject stitch --res tiny # fast stitch
|
||
gnommo -p myproject render --res tiny # fast preview render
|
||
gnommo -p myproject render --res low # medium preview render
|
||
```
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
myproject/
|
||
├── project.json # Project configuration
|
||
├── manuscript.txt # Narration script with [markers]
|
||
├── media/
|
||
│ ├── slides/
|
||
│ │ ├── slides.json # Slide definitions
|
||
│ │ └── *.png # Slide images
|
||
│ ├── videos/
|
||
│ │ ├── videos.json # Video source definitions
|
||
│ │ └── *.mov # Video files
|
||
│ ├── narration/
|
||
│ │ ├── narration.json # Narration segment definitions
|
||
│ │ └── *.mov # Raw narration recordings
|
||
│ └── audio/
|
||
│ ├── audio.json # Audio effect definitions
|
||
│ └── *.mp3 # Sound effects
|
||
└── output/
|
||
└── final.mp4 # Rendered output
|
||
└── preview.mp4 # Preview (lower resolution, faster render)
|
||
```
|
||
|
||
## The Five Stages
|
||
|
||
Gnommo uses a five-stage pipeline for processing video projects:
|
||
|
||
### Stage 1: Init
|
||
|
||
Creates a folder and a default project.json file inside it.
|
||
```bash
|
||
gnommo -p myproject init
|
||
```
|
||
|
||
### Stage 2: Import
|
||
First : Place the myproject.key Keynote presentation in the myproject folder.
|
||
Place videos, audio and narration you want to use in their respective folders in side myproject/media
|
||
Then : This command media scans directories and generates JSON definition files.
|
||
|
||
```bash
|
||
gnommo -p myproject import
|
||
```
|
||
|
||
**What it does:**
|
||
- Opens the keynote presentation and exports all slides a PNG images into media/slides/
|
||
- Scans `media/slides/` for images → generates `slides.json`
|
||
- Scans `media/videos/` for video files → generates `videos.json`
|
||
- Scans `media/narration/` for recordings → generates `narration.json`
|
||
- Scans `media/audio/` for sound effects → generates `audio.json`
|
||
|
||
**When to use:** After adding new media files to populate the JSON definitions with the actual files in the folders
|
||
|
||
---
|
||
|
||
### Stage 3: Grading
|
||
|
||
Iterate on keying/grading without running a full preprocess. It seeks a few
|
||
seconds into a raw clip, runs it through the `talkinghead` filter chain, and
|
||
writes `grade_preview.mov` (ProRes 4444 with alpha) to the project root.
|
||
|
||
# 1. KEY — auto, objective. Generates a candidate + preview, applies on pick.
|
||
gnommo -p video3 grade --stage key
|
||
open video3/grade_sweep/key_1.png # sanity-check the matte
|
||
gnommo -p video3 grade --pick key_1 # writes it to project.json
|
||
|
||
# 2. DESPILL — sweep, YOU pick. 7 stills, spill_suppress 0.7–1.5.
|
||
gnommo -p video3 grade --stage despill
|
||
open video3/grade_sweep/ # eyeball despill_1..7.png
|
||
gnommo -p video3 grade --pick despill_5 # apply whichever looks clean
|
||
|
||
# 3. GRADE — sweep, YOU pick. 6 stills, paleness 0.0–1.0.
|
||
gnommo -p video3 grade --stage grade
|
||
open video3/grade_sweep/ # eyeball grade_1..6.png
|
||
gnommo -p video3 grade --pick grade_3 # apply the paleness you like
|
||
|
||
|
||
|
||
### Stage 3: Preprocess
|
||
|
||
Applies video filters (chroma key, scaling, etc.) to narration segments.
|
||
|
||
```bash
|
||
gnommo -p myproject pre
|
||
```
|
||
|
||
**What it does:**
|
||
- Reads filter definitions from `project.json` and `narration.json`
|
||
- Processes each narration segment with its configured filters
|
||
- Outputs processed files (e.g., `segment1_processed.mov`)
|
||
|
||
**When to use:** After recording narration that needs background removal, sound normalization or other processing.
|
||
|
||
#### Green screen keying (`gnommokey`)
|
||
|
||
The `talkinghead` preset in `project.json` chains a `gnommokey` keyer (a
|
||
Keylight-style color-difference keyer) with a `color_grade` step. Key fields:
|
||
|
||
| Field | What it does |
|
||
|-------|--------------|
|
||
| `screen_color` | RGB of your green screen (sampled from the footage). |
|
||
| `screen_gain` | Key extraction strength. Higher = more aggressive matte. |
|
||
| `screen_balance` | Mixes luminance into the key (helps darker/lighter greens). |
|
||
| `clip_black` / `clip_white` | Compress the matte range (crush transparent / opaque). |
|
||
| `despill_bias` | RGB the keyed *edges* shift toward. A light neutral/skin tone reads better than cool blue. |
|
||
| `despill_strength` | How hard the edge/dominant-green despill pulls toward `despill_bias`. |
|
||
| `spill_suppress` | **Interior green-limiter — see below.** |
|
||
| `yellow_protect` | Shields saturated yellows/warm fabrics from `spill_suppress` (0–1). |
|
||
| `edge_erode` | Shrinks the alpha edge by N passes to kill green fringe (0–5). |
|
||
|
||
**`spill_suppress` — the bald-head knob.** The regular despill only acts where
|
||
green is the *dominant* channel (`g > max(r,b)`). On skin — especially a bald
|
||
head catching green bounce — red stays dominant, so that despill never touches
|
||
the interior and a green cast survives. `spill_suppress` caps green across the
|
||
**whole frame** at a reference extrapolated through the other two channels:
|
||
|
||
```
|
||
0.0 → cap at max(r, b) (only strong spill; effectively off)
|
||
1.0 → cap at avg(r, b) (removes visible cast)
|
||
2.0 → cap at min(r, b) (green can never exceed the smallest channel — nuclear)
|
||
```
|
||
|
||
Green is only ever *reduced*, never boosted, so clean pixels are untouched.
|
||
Start around `0.6`; go past `1.0` for heavy close-up spill. If skin tips
|
||
magenta/pink, ease back down. Range is `0.0`–`2.0`.
|
||
|
||
**`yellow_protect` — keep yellows yellow.** Because `spill_suppress` caps green
|
||
*everywhere*, it also drains green from things that are legitimately yellow (a
|
||
yellow jumpsuit, warm props), turning them orange. Blue is the tell: skin keeps
|
||
some blue, saturated yellow fabric reflects almost none. `yellow_protect`
|
||
(0.0–1.0) gates the green-limiter back down where `min(r,g) − b` is high
|
||
(yellow) while leaving skin/scalp spill fully suppressed. Bump it toward `1.0`
|
||
if warm colours go orange; leave at `0` if you have no strong yellows.
|
||
|
||
|
||
|
||
```bash
|
||
gnommo -p myproject grade # first raw_mov clip, 3s from 5s in
|
||
gnommo -p myproject grade --ss 12 --dur 4 # seek 12s in, make a 4s preview
|
||
gnommo -p myproject grade --file media/narration/raw_mov/clipA.mov
|
||
```
|
||
|
||
Tweak the `gnommokey` / `color_grade` values in `project.json`, re-run `grade`,
|
||
re-open the `.mov`, repeat until it looks right.
|
||
|
||
---
|
||
|
||
### Stage 4: stitch
|
||
First : Go through the source videos, and add trim settings to `begin` and `end` parameters in `narration.json`
|
||
|
||
Then : Run command to sticth the usable parts of narration segments into a single continuous video
|
||
|
||
```bash
|
||
gnommo -p myproject stitch
|
||
```
|
||
|
||
**What it does:**
|
||
- Reads segments from `narration.json`
|
||
- Concatenates them in order, respecting `begin`/`end` trim points
|
||
- Outputs `narration_combined.mov` in `media/videos/`
|
||
- Adds `narration_combined` entry to `videos.json` with volume settings
|
||
- Generates word-level timestamps from the narration using Whisper speech recognition.
|
||
|
||
**When to use:** After preprocessing, or adjusting trim settings, to create the main narration scaffolding.
|
||
|
||
### Stage 5: Render
|
||
|
||
Composites all elements into the final video.
|
||
|
||
```bash
|
||
gnommo -p myproject render
|
||
```
|
||
|
||
**What it does:**
|
||
- Parses `manuscript.txt` for slide/video markers
|
||
- Aligns markers to transcription timestamps
|
||
- Composites background, narration, slides, and video overlays
|
||
- Outputs `final.mp4`
|
||
|
||
**Options:**
|
||
```bash
|
||
gnommo -p myproject render --dry-run # Show FFmpeg command without running
|
||
gnommo -p myproject render --slides S1:S10 # Render only slides S1 through S10
|
||
gnommo -p myproject render --res low # Fast preview at 490x270
|
||
gnommo -p myproject render --res tiny # Ultrafast preview at 320x180
|
||
```
|
||
|
||
A partial `--slides S1:S10` render writes a range-suffixed file (e.g.
|
||
`PHIL_COSM_102_S1_S10.mp4`) so sections don't overwrite each other or the full render.
|
||
|
||
**Render log:** every render writes `<project>/<project>.log` (e.g. `video2.log`) with
|
||
the platform/ffmpeg/memory header, the exact ffmpeg command, and a `[mem …]` memory
|
||
sample every 3s. If a render crashes, check the tail of this log first.
|
||
|
||
---
|
||
|
||
### Render rig configuration (memory / performance)
|
||
|
||
The compositing ffmpeg graph holds a lot at once (RGBA layer buffers, many inputs, the
|
||
final mux), so peak RAM is high. Two knobs keep it bounded — both matter on a render rig.
|
||
|
||
**1. FFmpeg thread cap — `~/.gnommo.conf`** (on the render machine)
|
||
|
||
Fewer filter threads = far less peak memory (each parallel `format=rgba`/swscaler stage
|
||
holds its own full-frame buffers). The render honours `[performance] cpu_limit`, a
|
||
fraction of logical CPUs. **Unset defaults to 1 thread (safest).** On a memory-tight box
|
||
keep it low:
|
||
|
||
```ini
|
||
[performance]
|
||
cpu_limit = 0.25
|
||
```
|
||
|
||
> Note: the render uses `-filter_complex`, capped by `-filter_complex_threads` (not
|
||
> `-filter_threads`, which only applies to simple `-vf` graphs). This is why an
|
||
> uncapped render graph could OOM even when the preprocessor was fine.
|
||
|
||
**2. WSL2 memory/swap — `C:\Users\<you>\.wslconfig`** (Windows host, for an Ubuntu-on-WSL rig)
|
||
|
||
A WSL2 VM only gets a *slice* of host RAM (default ~50%, or 8 GB on older builds). If the
|
||
render exceeds that slice, **Windows OOM-kills the whole VM** — it surfaces as
|
||
`Wsl/Service/E_UNEXPECTED` / "Catastrophic failure", not a normal out-of-memory error, and
|
||
the Windows host still shows plenty of RAM free. Raise the cap and give it swap headroom:
|
||
|
||
```ini
|
||
[wsl2]
|
||
memory=24GB # give the VM more of the host RAM
|
||
swap=16GB # headroom so it pages instead of dying catastrophically
|
||
processors=8
|
||
```
|
||
|
||
Then, from PowerShell: `wsl --shutdown`, and restart the session. Confirm the VM's cap in
|
||
the render log header — its `memory: … total` is the VM slice, not the host RAM.
|
||
|
||
---
|
||
|
||
## Shortcut: All Stages
|
||
|
||
Run all stages 2-5 and render in one command:
|
||
|
||
```bash
|
||
gnommo -p myproject all
|
||
```
|
||
|
||
---
|
||
|
||
## Manuscript Format
|
||
|
||
The manuscript is plain text with embedded markers:
|
||
|
||
```
|
||
[S1] Welcome to this presentation.
|
||
|
||
[S2] Let me show you how this works.
|
||
|
||
[video:demo] Here's a quick demonstration.
|
||
|
||
[Zoom1] Notice this important detail.
|
||
|
||
[Reset] And that concludes our overview.
|
||
```
|
||
|
||
**Marker types:**
|
||
- `[S1]`, `[S2]` - Slide markers (reference slides.json)
|
||
- `[video:id]` - Triggered video overlay
|
||
- `[narration:id]` - Start continuous narration video
|
||
- `[Zoom1]`, `[Reset]` - Camera presets
|
||
- `[Awoosh]` - Audio effect trigger
|
||
|
||
---
|
||
|
||
## External Storage (GnommoCache)
|
||
|
||
For large projects, gnommo supports transparent external storage fallback.
|
||
|
||
**Setup:** Create `~/.gnommo.conf`:
|
||
```ini
|
||
[cache]
|
||
path = /Volumes/ExternalDrive/gnommo
|
||
```
|
||
|
||
**How it works:**
|
||
- Files are first looked up locally in the project directory
|
||
- If not found, gnommo checks `{cache_path}/{project_name}/...`
|
||
- The 📁 indicator shows files loaded from external storage
|
||
|
||
**Archive to external storage:**
|
||
```bash
|
||
gnommo -p myproject archive # Sync project to cache
|
||
gnommo -p myproject archive --dry-run # Preview what would sync
|
||
```
|
||
|
||
This allows you to move large preprocessed files to external storage while keeping the project functional.
|
||
|
||
---
|
||
|
||
## Common Workflows
|
||
|
||
### New Project Setup
|
||
```bash
|
||
# 1. Create project structure and add media files
|
||
mkdir -p myproject/media/{slides,videos,narration,audio}
|
||
|
||
# 2. Create project.json with basic config
|
||
|
||
# 3. Import media to generate JSON definitions
|
||
gnommo -p myproject import
|
||
|
||
# 4. Edit JSON files to configure filters, trim points, etc.
|
||
|
||
# 5. Run full pipeline
|
||
gnommo -p myproject all
|
||
```
|
||
|
||
### Re-render After Editing Manuscript
|
||
```bash
|
||
gnommo -p myproject render
|
||
```
|
||
|
||
### Re-process After Recording New Narration
|
||
```bash
|
||
gnommo -p myproject pre
|
||
gnommo -p myproject stitch
|
||
gnommo -p myproject transcribe
|
||
gnommo -p myproject render
|
||
```
|
||
|
||
---
|
||
|
||
## Additional Commands
|
||
|
||
```bash
|
||
gnommo -p myproject validate # Check for errors without rendering
|
||
gnommo -p myproject grade # Preview the keyer/grade on a few seconds of raw footage
|
||
gnommo -p myproject description # Generate YouTube description with chapters
|
||
gnommo -p myproject transcribe --final # Transcribe final.mp4 for subtitles
|
||
```
|
||
|
||
---
|
||
|
||
## Glitch University — Server Sync
|
||
|
||
Gnommo can push project metadata and short scripts to a gnommoweb server,
|
||
and pull changes back. This keeps the platform database in sync with your
|
||
local project files without manual copy-paste.
|
||
|
||
**Setup** — add to `gnommo/.env`:
|
||
```ini
|
||
GNOMMOWEB_URL=http://localhost:3001
|
||
GNOMMOWEB_API_KEY=your_content_api_key
|
||
```
|
||
|
||
### Push
|
||
|
||
Registers the project on the server and syncs all defined shorts (including
|
||
their scripts). Creates a filming task for each new short.
|
||
|
||
```bash
|
||
gnommo -p myproject push # push local → server
|
||
gnommo -p myproject push --force # overwrite server even if it has newer changes
|
||
```
|
||
|
||
On the first push, gnommo creates:
|
||
- A stub video record in the platform database
|
||
- One short record per entry in `project.json["shorts"]`
|
||
- One task per new short ("Film short: …")
|
||
|
||
Re-running push is safe — existing records are updated, no duplicate tasks.
|
||
Scripts are only overwritten on the server if the local file has changed;
|
||
edits made in the staff UI are preserved.
|
||
|
||
### Pull
|
||
|
||
Fetches the current project state from the server and merges the `shorts`
|
||
array back into `project.json`. Useful after editing short titles or hooks
|
||
in the web interface.
|
||
|
||
```bash
|
||
gnommo -p myproject pull # pull server → local
|
||
gnommo -p myproject pull --force # overwrite local even if it has unsaved changes
|
||
```
|
||
|
||
Pull preserves local `script` file paths — it won't overwrite your `.md`
|
||
script files.
|
||
|
||
### Conflict guards
|
||
|
||
Both commands check for conflicts before writing:
|
||
|
||
| Situation | Push behaviour | Pull behaviour |
|
||
|---|---|---|
|
||
| Server has changes you haven't pulled | Blocked — pull first | Proceeds (that's the point) |
|
||
| Local has changes you haven't pushed | Proceeds (that's the point) | Blocked — push first |
|
||
| `--force` flag | Overrides | Overrides |
|
||
|
||
Sync state is stored in `<project>/.gnommo_sync.json` (tracked by git,
|
||
so collaborators share the same reference point).
|
||
|
||
### Defining shorts in `project.json`
|
||
|
||
Add a `shorts` array to your project:
|
||
|
||
```json
|
||
"shorts": [
|
||
{
|
||
"id": "short_pixelated_universe",
|
||
"title": "Is the universe pixelated?",
|
||
"hook": "What if space is made of tiny blocks?",
|
||
"script": "shorts/short_pixelated_universe.md",
|
||
"platform_targets": ["youtube"]
|
||
}
|
||
]
|
||
```
|
||
|
||
- `id` — unique slug within the project, used as the upsert key
|
||
- `script` — relative path to a markdown file with the full short narration
|
||
- `hook` — opening line / thumbnail caption
|
||
- `platform_targets` — list of platforms (currently `["youtube"]`)
|
||
|
||
Scripts are plain markdown with the same `[SLIDE: name]` markers and
|
||
`{word}` whisper timestamp tags used elsewhere in gnommo.
|
||
|
||
---
|
||
|
||
## Requirements
|
||
|
||
- Python 3.10+
|
||
- FFmpeg
|
||
- OpenAI Whisper (for transcription)
|
||
|
||
```bash
|
||
pip install openai-whisper
|
||
```
|