gnommo/README.md

Gnommo

Gnommo is ADHD friendly video-editor for coders.

1. Design the presentation in keynote
2. Set up the greenscreen and audio settings once
3. Automatically times slides and videos to your voice.
4. Limited options means you waste less time on stuff that isn't important.

A code-first video editing pipeline for creating narrated presentations with slides, video overlays, and synchronized audio.

Quick Start

# Create a project
gnommo -p myproject init

# Import slides and presenter notes from Keynote file
gnommo -p myproject import

# Process the narration videos with video and audio filters
gnommo -p myproject pre

# Stitch together the narration segments to one full length narration. 
gnommo -p myproject stitch

# Transcribe the actual narrated content
gnommo -p myproject transcribe

# Generate the final video
gnommo -p myproject render

# Generate the final youtube assets. Manuscript file, description
gnommo -p myproject youtubeready

# Free up disk space locally by saving your project to an external drive
gnommo -p myproject archive

Resolution modes

All commands accept --res to trade quality for speed during iteration:

Flag	Resolution	Use case
--res full	Project resolution (default)	Final output
--res low	490×270	Fast preview render
--res tiny	320×180	Ultrafast iteration (preprocess, stitch, render)

--res tiny and --res low create downscaled copies of source files in subdirectories (proxy/ and low/ respectively) and work from those. The originals are never modified.

gnommo -p myproject pre --res tiny       # fast preprocess
gnommo -p myproject stitch --res tiny    # fast stitch
gnommo -p myproject render --res tiny    # fast preview render
gnommo -p myproject render --res low     # medium preview render

Project Structure

myproject/
├── project.json          # Project configuration
├── manuscript.txt        # Narration script with [markers]
├── media/
│   ├── slides/
│   │   ├── slides.json   # Slide definitions
│   │   └── *.png         # Slide images
│   ├── videos/
│   │   ├── videos.json   # Video source definitions
│   │   └── *.mov         # Video files
│   ├── narration/
│   │   ├── narration.json # Narration segment definitions
│   │   └── *.mov         # Raw narration recordings
│   └── audio/
│       ├── audio.json    # Audio effect definitions
│       └── *.mp3         # Sound effects
└── output/
    └── final.mp4         # Rendered output
    └── preview.mp4       # Preview (lower resolution, faster render)

The Five Stages

Gnommo uses a five-stage pipeline for processing video projects:

Stage 1: Init

Creates a folder and a default project.json file inside it.

gnommo -p myproject init

Stage 2: Import

First : Place the myproject.key Keynote presentation in the myproject folder. Place videos, audio and narration you want to use in their respective folders in side myproject/media Then : This command media scans directories and generates JSON definition files.

gnommo -p myproject import

What it does:

• Opens the keynote presentation and exports all slides a PNG images into media/slides/
• Scans media/slides/ for images → generates slides.json
• Scans media/videos/ for video files → generates videos.json
• Scans media/narration/ for recordings → generates narration.json
• Scans media/audio/ for sound effects → generates audio.json

When to use: After adding new media files to populate the JSON definitions with the actual files in the folders

---

Stage 3: Grading

Iterate on keying/grading without running a full preprocess. It seeks a few seconds into a raw clip, runs it through the talkinghead filter chain, and writes grade_preview.mov (ProRes 4444 with alpha) to the project root.

1. KEY — auto, objective. Generates a candidate + preview, applies on pick.

gnommo -p video3 grade --stage key open video3/grade_sweep/key_1.png # sanity-check the matte gnommo -p video3 grade --pick key_1 # writes it to project.json

2. DESPILL — sweep, YOU pick. 7 stills, spill_suppress 0.7–1.5.

gnommo -p video3 grade --stage despill open video3/grade_sweep/ # eyeball despill_1..7.png gnommo -p video3 grade --pick despill_5 # apply whichever looks clean

3. GRADE — sweep, YOU pick. 6 stills, paleness 0.0–1.0.

gnommo -p video3 grade --stage grade open video3/grade_sweep/ # eyeball grade_1..6.png gnommo -p video3 grade --pick grade_3 # apply the paleness you like

Stage 3: Preprocess

Applies video filters (chroma key, scaling, etc.) to narration segments.

gnommo -p myproject pre

What it does:

• Reads filter definitions from project.json and narration.json
• Processes each narration segment with its configured filters
• Outputs processed files (e.g., segment1_processed.mov)

When to use: After recording narration that needs background removal, sound normalization or other processing.

Green screen keying (gnommokey)

The talkinghead preset in project.json chains a gnommokey keyer (a Keylight-style color-difference keyer) with a color_grade step. Key fields:

Field	What it does
screen_color	RGB of your green screen (sampled from the footage).
screen_gain	Key extraction strength. Higher = more aggressive matte.
screen_balance	Mixes luminance into the key (helps darker/lighter greens).
clip_black / clip_white	Compress the matte range (crush transparent / opaque).
despill_bias	RGB the keyed edges shift toward. A light neutral/skin tone reads better than cool blue.
despill_strength	How hard the edge/dominant-green despill pulls toward despill_bias.
spill_suppress	Interior green-limiter — see below.
yellow_protect	Shields saturated yellows/warm fabrics from spill_suppress (0–1).
edge_erode	Shrinks the alpha edge by N passes to kill green fringe (0–5).

spill_suppress — the bald-head knob. The regular despill only acts where green is the dominant channel (g > max(r,b)). On skin — especially a bald head catching green bounce — red stays dominant, so that despill never touches the interior and a green cast survives. spill_suppress caps green across the whole frame at a reference extrapolated through the other two channels:

0.0  → cap at max(r, b)   (only strong spill; effectively off)
1.0  → cap at avg(r, b)   (removes visible cast)
2.0  → cap at min(r, b)   (green can never exceed the smallest channel — nuclear)

Green is only ever reduced, never boosted, so clean pixels are untouched. Start around 0.6; go past 1.0 for heavy close-up spill. If skin tips magenta/pink, ease back down. Range is 0.0–2.0.

yellow_protect — keep yellows yellow. Because spill_suppress caps green everywhere, it also drains green from things that are legitimately yellow (a yellow jumpsuit, warm props), turning them orange. Blue is the tell: skin keeps some blue, saturated yellow fabric reflects almost none. yellow_protect (0.0–1.0) gates the green-limiter back down where min(r,g) − b is high (yellow) while leaving skin/scalp spill fully suppressed. Bump it toward 1.0 if warm colours go orange; leave at 0 if you have no strong yellows.

gnommo -p myproject grade                      # first raw_mov clip, 3s from 5s in
gnommo -p myproject grade --ss 12 --dur 4      # seek 12s in, make a 4s preview
gnommo -p myproject grade --file media/narration/raw_mov/clipA.mov

Tweak the gnommokey / color_grade values in project.json, re-run grade, re-open the .mov, repeat until it looks right.

---

Stage 4: stitch

First : Go through the source videos, and add trim settings to begin and end parameters in narration.json

Then : Run command to sticth the usable parts of narration segments into a single continuous video

gnommo -p myproject stitch

What it does:

• Reads segments from narration.json
• Concatenates them in order, respecting begin/end trim points
• Outputs narration_combined.mov in media/videos/
• Adds narration_combined entry to videos.json with volume settings
• Generates word-level timestamps from the narration using Whisper speech recognition.

When to use: After preprocessing, or adjusting trim settings, to create the main narration scaffolding.

Stage 5: Render

Composites all elements into the final video.

gnommo -p myproject render

What it does:

• Parses manuscript.txt for slide/video markers
• Aligns markers to transcription timestamps
• Composites background, narration, slides, and video overlays
• Outputs final.mp4

Options:

gnommo -p myproject render --dry-run        # Show FFmpeg command without running
gnommo -p myproject render --slides S1:S10  # Render only slides S1 through S10
gnommo -p myproject render --res low        # Fast preview at 490x270
gnommo -p myproject render --res tiny       # Ultrafast preview at 320x180

A partial --slides S1:S10 render writes a range-suffixed file (e.g. PHIL_COSM_102_S1_S10.mp4) so sections don't overwrite each other or the full render.

Render log: every render writes <project>/<project>.log (e.g. video2.log) with the platform/ffmpeg/memory header, the exact ffmpeg command, and a [mem …] memory sample every 3s. If a render crashes, check the tail of this log first.

---

Render rig configuration (memory / performance)

The compositing ffmpeg graph holds a lot at once (RGBA layer buffers, many inputs, the final mux), so peak RAM is high. Two knobs keep it bounded — both matter on a render rig.

1. FFmpeg thread cap — ~/.gnommo.conf (on the render machine)

Fewer filter threads = far less peak memory (each parallel format=rgba/swscaler stage holds its own full-frame buffers). The render honours [performance] cpu_limit, a fraction of logical CPUs. Unset defaults to 1 thread (safest). On a memory-tight box keep it low:

[performance]
cpu_limit = 0.25

Note: the render uses -filter_complex, capped by -filter_complex_threads (not -filter_threads, which only applies to simple -vf graphs). This is why an uncapped render graph could OOM even when the preprocessor was fine.

2. WSL2 memory/swap — C:\Users\<you>\.wslconfig (Windows host, for an Ubuntu-on-WSL rig)

A WSL2 VM only gets a slice of host RAM (default ~50%, or 8 GB on older builds). If the render exceeds that slice, Windows OOM-kills the whole VM — it surfaces as Wsl/Service/E_UNEXPECTED / "Catastrophic failure", not a normal out-of-memory error, and the Windows host still shows plenty of RAM free. Raise the cap and give it swap headroom:

[wsl2]
memory=24GB      # give the VM more of the host RAM
swap=16GB        # headroom so it pages instead of dying catastrophically
processors=8

Then, from PowerShell: wsl --shutdown, and restart the session. Confirm the VM's cap in the render log header — its memory: … total is the VM slice, not the host RAM.

---

Shortcut: All Stages

Run all stages 2-5 and render in one command:

gnommo -p myproject all

---

Manuscript Format

The manuscript is plain text with embedded markers:

[S1] Welcome to this presentation.

[S2] Let me show you how this works.

[video:demo] Here's a quick demonstration.

[Zoom1] Notice this important detail.

[Reset] And that concludes our overview.

Marker types:

• [S1], [S2] - Slide markers (reference slides.json)
• [video:id] - Triggered video overlay
• [narration:id] - Start continuous narration video
• [Zoom1], [Reset] - Camera presets
• [Awoosh] - Audio effect trigger

---

External Storage (GnommoCache)

For large projects, gnommo supports transparent external storage fallback.

Setup: Create ~/.gnommo.conf:

[cache]
path = /Volumes/ExternalDrive/gnommo

How it works:

• Files are first looked up locally in the project directory
• If not found, gnommo checks {cache_path}/{project_name}/...
• The 📁 indicator shows files loaded from external storage

Archive to external storage:

gnommo -p myproject archive           # Sync project to cache
gnommo -p myproject archive --dry-run # Preview what would sync

This allows you to move large preprocessed files to external storage while keeping the project functional.

---

Common Workflows

New Project Setup

# 1. Create project structure and add media files
mkdir -p myproject/media/{slides,videos,narration,audio}

# 2. Create project.json with basic config

# 3. Import media to generate JSON definitions
gnommo -p myproject import

# 4. Edit JSON files to configure filters, trim points, etc.

# 5. Run full pipeline
gnommo -p myproject all

Re-render After Editing Manuscript

gnommo -p myproject render

Re-process After Recording New Narration

gnommo -p myproject pre
gnommo -p myproject stitch
gnommo -p myproject transcribe
gnommo -p myproject render

---

Additional Commands

gnommo -p myproject validate      # Check for errors without rendering
gnommo -p myproject grade         # Preview the keyer/grade on a few seconds of raw footage
gnommo -p myproject description   # Generate YouTube description with chapters
gnommo -p myproject transcribe --final  # Transcribe final.mp4 for subtitles

---

Glitch University — Server Sync

Gnommo can push project metadata and short scripts to a gnommoweb server, and pull changes back. This keeps the platform database in sync with your local project files without manual copy-paste.

Setup — add to gnommo/.env:

GNOMMOWEB_URL=http://localhost:3001
GNOMMOWEB_API_KEY=your_content_api_key

Push

Registers the project on the server and syncs all defined shorts (including their scripts). Creates a filming task for each new short.

gnommo -p myproject push           # push local → server
gnommo -p myproject push --force   # overwrite server even if it has newer changes

On the first push, gnommo creates:

• A stub video record in the platform database
• One short record per entry in project.json["shorts"]
• One task per new short ("Film short: …")

Re-running push is safe — existing records are updated, no duplicate tasks. Scripts are only overwritten on the server if the local file has changed; edits made in the staff UI are preserved.

Pull

Fetches the current project state from the server and merges the shorts array back into project.json. Useful after editing short titles or hooks in the web interface.

gnommo -p myproject pull           # pull server → local
gnommo -p myproject pull --force   # overwrite local even if it has unsaved changes

Pull preserves local script file paths — it won't overwrite your .md script files.

Conflict guards

Both commands check for conflicts before writing:

Situation	Push behaviour	Pull behaviour
Server has changes you haven't pulled	Blocked — pull first	Proceeds (that's the point)
Local has changes you haven't pushed	Proceeds (that's the point)	Blocked — push first
--force flag	Overrides	Overrides

Sync state is stored in <project>/.gnommo_sync.json (tracked by git, so collaborators share the same reference point).

Defining shorts in project.json

Add a shorts array to your project:

"shorts": [
  {
    "id": "short_pixelated_universe",
    "title": "Is the universe pixelated?",
    "hook": "What if space is made of tiny blocks?",
    "script": "shorts/short_pixelated_universe.md",
    "platform_targets": ["youtube"]
  }
]

• id — unique slug within the project, used as the upsert key
• script — relative path to a markdown file with the full short narration
• hook — opening line / thumbnail caption
• platform_targets — list of platforms (currently ["youtube"])

Scripts are plain markdown with the same [SLIDE: name] markers and {word} whisper timestamp tags used elsewhere in gnommo.

---

Requirements

• Python 3.10+
• FFmpeg
• OpenAI Whisper (for transcription)

pip install openai-whisper