Shared skills now for all agent and fixed the settings for Gunnar and Hermes
This commit is contained in:
@@ -0,0 +1,355 @@
|
||||
---
|
||||
name: touchdesigner-mcp
|
||||
description: "Control a running TouchDesigner instance via twozero MCP — create operators, set parameters, wire connections, execute Python, build real-time visuals. 36 native tools."
|
||||
version: 1.1.0
|
||||
author: kshitijk4poor
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [TouchDesigner, MCP, twozero, creative-coding, real-time-visuals, generative-art, audio-reactive, VJ, installation, GLSL]
|
||||
related_skills: [native-mcp, ascii-video, manim-video, hermes-video]
|
||||
|
||||
---
|
||||
|
||||
# TouchDesigner Integration (twozero MCP)
|
||||
|
||||
## CRITICAL RULES
|
||||
|
||||
1. **NEVER guess parameter names.** Call `td_get_par_info` for the op type FIRST. Your training data is wrong for TD 2025.32.
|
||||
2. **If `tdAttributeError` fires, STOP.** Call `td_get_operator_info` on the failing node before continuing.
|
||||
3. **NEVER hardcode absolute paths** in script callbacks. Use `me.parent()` / `scriptOp.parent()`.
|
||||
4. **Prefer native MCP tools over td_execute_python.** Use `td_create_operator`, `td_set_operator_pars`, `td_get_errors` etc. Only fall back to `td_execute_python` for complex multi-step logic.
|
||||
5. **Call `td_get_hints` before building.** It returns patterns specific to the op type you're working with.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Hermes Agent -> MCP (Streamable HTTP) -> twozero.tox (port 40404) -> TD Python
|
||||
```
|
||||
|
||||
36 native tools. Free plugin (no payment/license — confirmed April 2026).
|
||||
Context-aware (knows selected OP, current network).
|
||||
Hub health check: `GET http://localhost:40404/mcp` returns JSON with instance PID, project name, TD version.
|
||||
|
||||
## Setup (Automated)
|
||||
|
||||
Run the setup script to handle everything:
|
||||
|
||||
```bash
|
||||
bash "${HERMES_HOME:-$HOME/.hermes}/skills/creative/touchdesigner-mcp/scripts/setup.sh"
|
||||
```
|
||||
|
||||
The script will:
|
||||
1. Check if TD is running
|
||||
2. Download twozero.tox if not already cached
|
||||
3. Add `twozero_td` MCP server to Hermes config (if missing)
|
||||
4. Test the MCP connection on port 40404
|
||||
5. Report what manual steps remain (drag .tox into TD, enable MCP toggle)
|
||||
|
||||
### Manual steps (one-time, cannot be automated)
|
||||
|
||||
1. **Drag `~/Downloads/twozero.tox` into the TD network editor** → click Install
|
||||
2. **Enable MCP:** click twozero icon → Settings → mcp → "auto start MCP" → Yes
|
||||
3. **Restart Hermes session** to pick up the new MCP server
|
||||
|
||||
After setup, verify:
|
||||
```bash
|
||||
nc -z 127.0.0.1 40404 && echo "twozero MCP: READY"
|
||||
```
|
||||
|
||||
## Environment Notes
|
||||
|
||||
- **Non-Commercial TD** caps resolution at 1280×1280. Use `outputresolution = 'custom'` and set width/height explicitly.
|
||||
- **Codecs:** `prores` (preferred on macOS) or `mjpa` as fallback. H.264/H.265/AV1 require a Commercial license.
|
||||
- Always call `td_get_par_info` before setting params — names vary by TD version (see CRITICAL RULES #1).
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 0: Discover (before building anything)
|
||||
|
||||
```
|
||||
Call td_get_par_info with op_type for each type you plan to use.
|
||||
Call td_get_hints with the topic you're building (e.g. "glsl", "audio reactive", "feedback").
|
||||
Call td_get_focus to see where the user is and what's selected.
|
||||
Call td_get_network to see what already exists.
|
||||
```
|
||||
|
||||
No temp nodes, no cleanup. This replaces the old discovery dance entirely.
|
||||
|
||||
### Step 1: Clean + Build
|
||||
|
||||
**IMPORTANT: Split cleanup and creation into SEPARATE MCP calls.** Destroying and recreating same-named nodes in one `td_execute_python` script causes "Invalid OP object" errors. See pitfalls #11b.
|
||||
|
||||
Use `td_create_operator` for each node (handles viewport positioning automatically):
|
||||
|
||||
```
|
||||
td_create_operator(type="noiseTOP", parent="/project1", name="bg", parameters={"resolutionw": 1280, "resolutionh": 720})
|
||||
td_create_operator(type="levelTOP", parent="/project1", name="brightness")
|
||||
td_create_operator(type="nullTOP", parent="/project1", name="out")
|
||||
```
|
||||
|
||||
For bulk creation or wiring, use `td_execute_python`:
|
||||
|
||||
```python
|
||||
# td_execute_python script:
|
||||
root = op('/project1')
|
||||
nodes = []
|
||||
for name, optype in [('bg', noiseTOP), ('fx', levelTOP), ('out', nullTOP)]:
|
||||
n = root.create(optype, name)
|
||||
nodes.append(n.path)
|
||||
# Wire chain
|
||||
for i in range(len(nodes)-1):
|
||||
op(nodes[i]).outputConnectors[0].connect(op(nodes[i+1]).inputConnectors[0])
|
||||
result = {'created': nodes}
|
||||
```
|
||||
|
||||
### Step 2: Set Parameters
|
||||
|
||||
Prefer the native tool (validates params, won't crash):
|
||||
|
||||
```
|
||||
td_set_operator_pars(path="/project1/bg", parameters={"roughness": 0.6, "monochrome": true})
|
||||
```
|
||||
|
||||
For expressions or modes, use `td_execute_python`:
|
||||
|
||||
```python
|
||||
op('/project1/time_driver').par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
```
|
||||
|
||||
### Step 3: Wire
|
||||
|
||||
Use `td_execute_python` — no native wire tool exists:
|
||||
|
||||
```python
|
||||
op('/project1/bg').outputConnectors[0].connect(op('/project1/fx').inputConnectors[0])
|
||||
```
|
||||
|
||||
### Step 4: Verify
|
||||
|
||||
```
|
||||
td_get_errors(path="/project1", recursive=true)
|
||||
td_get_perf()
|
||||
td_get_operator_info(path="/project1/out", detail="full")
|
||||
```
|
||||
|
||||
### Step 5: Display / Capture
|
||||
|
||||
```
|
||||
td_get_screenshot(path="/project1/out")
|
||||
```
|
||||
|
||||
Or open a window via script:
|
||||
|
||||
```python
|
||||
win = op('/project1').create(windowCOMP, 'display')
|
||||
win.par.winop = op('/project1/out').path
|
||||
win.par.winw = 1280; win.par.winh = 720
|
||||
win.par.winopen.pulse()
|
||||
```
|
||||
|
||||
## MCP Tool Quick Reference
|
||||
|
||||
**Core (use these most):**
|
||||
| Tool | What |
|
||||
|------|------|
|
||||
| `td_execute_python` | Run arbitrary Python in TD. Full API access. |
|
||||
| `td_create_operator` | Create node with params + auto-positioning |
|
||||
| `td_set_operator_pars` | Set params safely (validates, won't crash) |
|
||||
| `td_get_operator_info` | Inspect one node: connections, params, errors |
|
||||
| `td_get_operators_info` | Inspect multiple nodes in one call |
|
||||
| `td_get_network` | See network structure at a path |
|
||||
| `td_get_errors` | Find errors/warnings recursively |
|
||||
| `td_get_par_info` | Get param names for an OP type (replaces discovery) |
|
||||
| `td_get_hints` | Get patterns/tips before building |
|
||||
| `td_get_focus` | What network is open, what's selected |
|
||||
|
||||
**Read/Write:**
|
||||
| Tool | What |
|
||||
|------|------|
|
||||
| `td_read_dat` | Read DAT text content |
|
||||
| `td_write_dat` | Write/patch DAT content |
|
||||
| `td_read_chop` | Read CHOP channel values |
|
||||
| `td_read_textport` | Read TD console output |
|
||||
|
||||
**Visual:**
|
||||
| Tool | What |
|
||||
|------|------|
|
||||
| `td_get_screenshot` | Capture one OP viewer to file |
|
||||
| `td_get_screenshots` | Capture multiple OPs at once |
|
||||
| `td_get_screen_screenshot` | Capture actual screen via TD |
|
||||
| `td_navigate_to` | Jump network editor to an OP |
|
||||
|
||||
**Search:**
|
||||
| Tool | What |
|
||||
|------|------|
|
||||
| `td_find_op` | Find ops by name/type across project |
|
||||
| `td_search` | Search code, expressions, string params |
|
||||
|
||||
**System:**
|
||||
| Tool | What |
|
||||
|------|------|
|
||||
| `td_get_perf` | Performance profiling (FPS, slow ops) |
|
||||
| `td_list_instances` | List all running TD instances |
|
||||
| `td_get_docs` | In-depth docs on a TD topic |
|
||||
| `td_agents_md` | Read/write per-COMP markdown docs |
|
||||
| `td_reinit_extension` | Reload extension after code edit |
|
||||
| `td_clear_textport` | Clear console before debug session |
|
||||
|
||||
**Input Automation:**
|
||||
| Tool | What |
|
||||
|------|------|
|
||||
| `td_input_execute` | Send mouse/keyboard to TD |
|
||||
| `td_input_status` | Poll input queue status |
|
||||
| `td_input_clear` | Stop input automation |
|
||||
| `td_op_screen_rect` | Get screen coords of a node |
|
||||
| `td_click_screen_point` | Click a point in a screenshot |
|
||||
| `td_screen_point_to_global` | Convert screenshot pixel to absolute screen coords |
|
||||
|
||||
The table above covers the 32 tools used in typical creative workflows. The remaining 4 tools (`td_project_quit`, `td_test_session`, `td_dev_log`, `td_clear_dev_log`) are admin/dev-mode utilities — see `references/mcp-tools.md` for the full 36-tool reference with complete parameter schemas.
|
||||
|
||||
## Key Implementation Rules
|
||||
|
||||
**GLSL time:** No `uTDCurrentTime` in GLSL TOP. Use the Values page:
|
||||
```python
|
||||
# Call td_get_par_info(op_type="glslTOP") first to confirm param names
|
||||
td_set_operator_pars(path="/project1/shader", parameters={"value0name": "uTime"})
|
||||
# Then set expression via script:
|
||||
# op('/project1/shader').par.value0.expr = "absTime.seconds"
|
||||
# In GLSL: uniform float uTime;
|
||||
```
|
||||
|
||||
Fallback: Constant TOP in `rgba32float` format (8-bit clamps to 0-1, freezing the shader).
|
||||
|
||||
**Feedback TOP:** Use `top` parameter reference, not direct input wire. "Not enough sources" resolves after first cook. "Cook dependency loop" warning is expected.
|
||||
|
||||
**Resolution:** Non-Commercial caps at 1280×1280. Use `outputresolution = 'custom'`.
|
||||
|
||||
**Large shaders:** Write GLSL to `/tmp/file.glsl`, then use `td_write_dat` or `td_execute_python` to load.
|
||||
|
||||
**Vertex/Point access (TD 2025.32):** `point.P[0]`, `point.P[1]`, `point.P[2]` — NOT `.x`, `.y`, `.z`.
|
||||
|
||||
**Extensions:** `ext0object` format is `"op('./datName').module.ClassName(me)"` in CONSTANT mode. After editing extension code with `td_write_dat`, call `td_reinit_extension`.
|
||||
|
||||
**Script callbacks:** ALWAYS use relative paths via `me.parent()` / `scriptOp.parent()`.
|
||||
|
||||
**Cleaning nodes:** Always `list(root.children)` before iterating + `child.valid` check.
|
||||
|
||||
## Recording / Exporting Video
|
||||
|
||||
```python
|
||||
# via td_execute_python:
|
||||
root = op('/project1')
|
||||
rec = root.create(moviefileoutTOP, 'recorder')
|
||||
op('/project1/out').outputConnectors[0].connect(rec.inputConnectors[0])
|
||||
rec.par.type = 'movie'
|
||||
rec.par.file = '/tmp/output.mov'
|
||||
rec.par.videocodec = 'prores' # Apple ProRes — NOT license-restricted on macOS
|
||||
rec.par.record = True # start
|
||||
# rec.par.record = False # stop (call separately later)
|
||||
```
|
||||
|
||||
H.264/H.265/AV1 need Commercial license. Use `prores` on macOS or `mjpa` as fallback.
|
||||
Extract frames: `ffmpeg -i /tmp/output.mov -vframes 120 /tmp/frames/frame_%06d.png`
|
||||
|
||||
**TOP.save() is useless for animation** — captures same GPU texture every time. Always use MovieFileOut.
|
||||
|
||||
### Before Recording: Checklist
|
||||
|
||||
1. **Verify FPS > 0** via `td_get_perf`. If FPS=0 the recording will be empty. See pitfalls #38-39.
|
||||
2. **Verify shader output is not black** via `td_get_screenshot`. Black output = shader error or missing input. See pitfalls #8, #40.
|
||||
3. **If recording with audio:** cue audio to start first, then delay recording by 3 frames. See pitfalls #19.
|
||||
4. **Set output path before starting record** — setting both in the same script can race.
|
||||
|
||||
## Audio-Reactive GLSL (Proven Recipe)
|
||||
|
||||
### Correct signal chain (tested April 2026)
|
||||
|
||||
```
|
||||
AudioFileIn CHOP (playmode=sequential)
|
||||
→ AudioSpectrum CHOP (FFT=512, outputmenu=setmanually, outlength=256, timeslice=ON)
|
||||
→ Math CHOP (gain=10)
|
||||
→ CHOP to TOP (dataformat=r, layout=rowscropped)
|
||||
→ GLSL TOP input 1 (spectrum texture, 256x2)
|
||||
|
||||
Constant TOP (rgba32float, time) → GLSL TOP input 0
|
||||
GLSL TOP → Null TOP → MovieFileOut
|
||||
```
|
||||
|
||||
### Critical audio-reactive rules (empirically verified)
|
||||
|
||||
1. **TimeSlice must stay ON** for AudioSpectrum. OFF = processes entire audio file → 24000+ samples → CHOP to TOP overflow.
|
||||
2. **Set Output Length manually** to 256 via `outputmenu='setmanually'` and `outlength=256`. Default outputs 22050 samples.
|
||||
3. **DO NOT use Lag CHOP for spectrum smoothing.** Lag CHOP operates in timeslice mode and expands 256 samples to 2400+, averaging all values to near-zero (~1e-06). The shader receives no usable data. This was the #1 audio sync failure in testing.
|
||||
4. **DO NOT use Filter CHOP either** — same timeslice expansion problem with spectrum data.
|
||||
5. **Smoothing belongs in the GLSL shader** if needed, via temporal lerp with a feedback texture: `mix(prevValue, newValue, 0.3)`. This gives frame-perfect sync with zero pipeline latency.
|
||||
6. **CHOP to TOP dataformat = 'r'**, layout = 'rowscropped'. Spectrum output is 256x2 (stereo). Sample at y=0.25 for first channel.
|
||||
7. **Math gain = 10** (not 5). Raw spectrum values are ~0.19 in bass range. Gain of 10 gives usable ~5.0 for the shader.
|
||||
8. **No Resample CHOP needed.** Control output size via AudioSpectrum's `outlength` param directly.
|
||||
|
||||
### GLSL spectrum sampling
|
||||
|
||||
```glsl
|
||||
// Input 0 = time (1x1 rgba32float), Input 1 = spectrum (256x2)
|
||||
float iTime = texture(sTD2DInputs[0], vec2(0.5)).r;
|
||||
|
||||
// Sample multiple points per band and average for stability:
|
||||
// NOTE: y=0.25 for first channel (stereo texture is 256x2, first row center is 0.25)
|
||||
float bass = (texture(sTD2DInputs[1], vec2(0.02, 0.25)).r +
|
||||
texture(sTD2DInputs[1], vec2(0.05, 0.25)).r) / 2.0;
|
||||
float mid = (texture(sTD2DInputs[1], vec2(0.2, 0.25)).r +
|
||||
texture(sTD2DInputs[1], vec2(0.35, 0.25)).r) / 2.0;
|
||||
float hi = (texture(sTD2DInputs[1], vec2(0.6, 0.25)).r +
|
||||
texture(sTD2DInputs[1], vec2(0.8, 0.25)).r) / 2.0;
|
||||
```
|
||||
|
||||
See `references/network-patterns.md` for complete build scripts + shader code.
|
||||
|
||||
## Operator Quick Reference
|
||||
|
||||
| Family | Color | Python class / MCP type | Suffix |
|
||||
|--------|-------|-------------|--------|
|
||||
| TOP | Purple | noiseTOP, glslTOP, compositeTOP, levelTop, blurTOP, textTOP, nullTOP | TOP |
|
||||
| CHOP | Green | audiofileinCHOP, audiospectrumCHOP, mathCHOP, lfoCHOP, constantCHOP | CHOP |
|
||||
| SOP | Blue | gridSOP, sphereSOP, transformSOP, noiseSOP | SOP |
|
||||
| DAT | White | textDAT, tableDAT, scriptDAT, webserverDAT | DAT |
|
||||
| MAT | Yellow | phongMAT, pbrMAT, glslMAT, constMAT | MAT |
|
||||
| COMP | Gray | geometryCOMP, containerCOMP, cameraCOMP, lightCOMP, windowCOMP | COMP |
|
||||
|
||||
## Security Notes
|
||||
|
||||
- MCP runs on localhost only (port 40404). No authentication — any local process can send commands.
|
||||
- `td_execute_python` has unrestricted access to the TD Python environment and filesystem as the TD process user.
|
||||
- `setup.sh` downloads twozero.tox from the official 404zero.com URL. Verify the download if concerned.
|
||||
- The skill never sends data outside localhost. All MCP communication is local.
|
||||
|
||||
## References
|
||||
|
||||
| File | What |
|
||||
|------|------|
|
||||
| `references/pitfalls.md` | Hard-won lessons from real sessions |
|
||||
| `references/operators.md` | All operator families with params and use cases |
|
||||
| `references/network-patterns.md` | Recipes: audio-reactive, generative, GLSL, instancing |
|
||||
| `references/mcp-tools.md` | Full twozero MCP tool parameter schemas |
|
||||
| `references/python-api.md` | TD Python: op(), scripting, extensions |
|
||||
| `references/troubleshooting.md` | Connection diagnostics, debugging |
|
||||
| `references/glsl.md` | GLSL uniforms, built-in functions, shader templates |
|
||||
| `references/postfx.md` | Post-FX: bloom, CRT, chromatic aberration, feedback glow |
|
||||
| `references/layout-compositor.md` | HUD layout patterns, panel grids, BSP-style layouts |
|
||||
| `references/operator-tips.md` | Wireframe rendering, feedback TOP setup |
|
||||
| `references/geometry-comp.md` | Geometry COMP: instancing, POP vs SOP, morphing |
|
||||
| `references/audio-reactive.md` | Audio band extraction, beat detection, envelope following |
|
||||
| `references/animation.md` | LFOs, timers, keyframes, easing, expression-driven motion |
|
||||
| `references/midi-osc.md` | MIDI/OSC controllers, TouchOSC, multi-machine sync |
|
||||
| `references/particles.md` | POPs and legacy particleSOP — emission, forces, collisions |
|
||||
| `references/projection-mapping.md` | Multi-window output, corner pin, mesh warp, edge blending |
|
||||
| `references/external-data.md` | HTTP, WebSocket, MQTT, Serial, TCP, webserverDAT |
|
||||
| `references/panel-ui.md` | Custom params, panel COMPs, button/slider/field, panelExecuteDAT |
|
||||
| `references/replicator.md` | replicatorCOMP — data-driven cloning, layouts, callbacks |
|
||||
| `references/dat-scripting.md` | Execute DAT family — chop/dat/parameter/panel/op/executeDAT |
|
||||
| `references/3d-scene.md` | Lighting rigs, shadows, IBL/cubemaps, multi-camera, PBR |
|
||||
| `scripts/setup.sh` | Automated setup script |
|
||||
|
||||
---
|
||||
|
||||
> You're not writing code. You're conducting light.
|
||||
@@ -0,0 +1,275 @@
|
||||
# 3D Scene Reference
|
||||
|
||||
Lighting rigs, shadows, IBL/cubemaps, multi-camera, and PBR materials. For wireframe rendering and feedback TOPs see `operator-tips.md`. For instancing geometry see `geometry-comp.md`. For shader code see `glsl.md`.
|
||||
|
||||
---
|
||||
|
||||
## Anatomy of a 3D Scene
|
||||
|
||||
```
|
||||
[Geometry COMP] ← contains SOPs (the shapes)
|
||||
[Material] ← Phong/PBR/GLSL/Constant MAT
|
||||
[Light COMPs] ← point/directional/spot/area/environment
|
||||
[Camera COMP] ← view position, FOV
|
||||
│
|
||||
▼
|
||||
[Render TOP] ← combines geo + lights + camera into a 2D image
|
||||
│
|
||||
▼
|
||||
[post-FX chain] ← bloomTOP, glsl shaders, etc.
|
||||
│
|
||||
▼
|
||||
[windowCOMP] ← actual display
|
||||
```
|
||||
|
||||
Render TOP is the heart. It takes an explicit `geometry` path, an explicit `camera` path, and lights via the lights table or an envlight reference.
|
||||
|
||||
---
|
||||
|
||||
## Minimal Scene
|
||||
|
||||
```python
|
||||
# Geometry
|
||||
geo = root.create(geometryCOMP, 'scene_geo')
|
||||
sphere = geo.create(sphereSOP, 'shape')
|
||||
sphere.par.rad = 1.0; sphere.par.rows = 64; sphere.par.cols = 64
|
||||
|
||||
# Material — start with PBR
|
||||
mat = root.create(pbrMAT, 'mat')
|
||||
mat.par.basecolorr = 0.7; mat.par.basecolorg = 0.7; mat.par.basecolorb = 0.7
|
||||
mat.par.metallic = 0.0
|
||||
mat.par.roughness = 0.4
|
||||
|
||||
geo.par.material = mat.path
|
||||
|
||||
# Camera
|
||||
cam = root.create(cameraCOMP, 'cam1')
|
||||
cam.par.tx = 0; cam.par.ty = 0; cam.par.tz = 4
|
||||
cam.par.fov = 45
|
||||
cam.par.near = 0.1; cam.par.far = 100
|
||||
|
||||
# Key light
|
||||
key = root.create(lightCOMP, 'key_light')
|
||||
key.par.lighttype = 'point'
|
||||
key.par.tx = 3; key.par.ty = 3; key.par.tz = 3
|
||||
key.par.dimmer = 1.5
|
||||
|
||||
# Render
|
||||
render = root.create(renderTOP, 'render1')
|
||||
render.par.outputresolution = 'custom'
|
||||
render.par.resolutionw = 1920; render.par.resolutionh = 1080
|
||||
render.par.camera = cam.path
|
||||
render.par.geometry = geo.path
|
||||
render.par.lights = key.path # single light path; for multi, see below
|
||||
render.par.bgcolorr = 0; render.par.bgcolorg = 0; render.par.bgcolorb = 0
|
||||
```
|
||||
|
||||
For multiple lights, leave `par.lights` blank — Render TOP scans the network for all `lightCOMP` and `envlightCOMP` ops by default. To restrict to specific lights, set `par.lights = '/project1/key_light /project1/fill_light'` (space-separated paths).
|
||||
|
||||
---
|
||||
|
||||
## Light Types
|
||||
|
||||
| Type | What | Common params |
|
||||
|---|---|---|
|
||||
| `point` | Omnidirectional, falls off with distance | `dimmer`, `coneangle` (n/a), `attenuation` |
|
||||
| `directional` | Parallel rays, infinite distance (sun) | `dimmer`, light's rotation only matters |
|
||||
| `spot` | Cone, falls off with distance + angle | `coneangle`, `conedelta`, `dimmer` |
|
||||
| `cone` | Like spot but harder edge | same |
|
||||
| `area` | Rectangular soft light source | `sizex`, `sizey` |
|
||||
|
||||
For all: `colorr`, `colorg`, `colorb`, `tx/ty/tz`, `rx/ry/rz`, `dimmer`.
|
||||
|
||||
### Three-Point Lighting (Studio Setup)
|
||||
|
||||
```python
|
||||
# Key — main light, ~45° front
|
||||
key = root.create(lightCOMP, 'key')
|
||||
key.par.lighttype = 'point'
|
||||
key.par.tx = 4; key.par.ty = 3; key.par.tz = 4
|
||||
key.par.dimmer = 1.5
|
||||
key.par.colorr = 1.0; key.par.colorg = 0.95; key.par.colorb = 0.85
|
||||
|
||||
# Fill — softer, opposite side
|
||||
fill = root.create(lightCOMP, 'fill')
|
||||
fill.par.lighttype = 'area'
|
||||
fill.par.tx = -4; fill.par.ty = 2; fill.par.tz = 3
|
||||
fill.par.dimmer = 0.5
|
||||
fill.par.colorr = 0.7; fill.par.colorg = 0.8; fill.par.colorb = 1.0
|
||||
fill.par.sizex = 4; fill.par.sizey = 4
|
||||
|
||||
# Rim/back — outline from behind
|
||||
rim = root.create(lightCOMP, 'rim')
|
||||
rim.par.lighttype = 'spot'
|
||||
rim.par.tx = 0; rim.par.ty = 4; rim.par.tz = -4
|
||||
rim.par.coneangle = 30
|
||||
rim.par.dimmer = 1.0
|
||||
|
||||
# Optional: ambient lift to prevent pure-black shadows
|
||||
amb = root.create(ambientlightCOMP, 'ambient')
|
||||
amb.par.dimmer = 0.15
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Shadows
|
||||
|
||||
Spot and directional lights cast shadows when `par.shadowtype != 'none'`.
|
||||
|
||||
```python
|
||||
key.par.shadowtype = 'softshadow' # 'none' | 'hardshadow' | 'softshadow'
|
||||
key.par.shadowsize = 1024 # shadow map resolution
|
||||
key.par.shadowsoftness = 0.02 # softshadow only
|
||||
```
|
||||
|
||||
**Tips:**
|
||||
- Soft shadows are GPU-expensive. Start with `shadowsize = 1024` and only go higher (2048/4096) if shadow edges look pixelated at your resolution.
|
||||
- Set the spot light's `near`/`far` to JUST contain the scene. Wider range = wasted shadow map precision.
|
||||
- Multiple shadow-casting lights compound cost. Limit to 1-2 in real-time work; pre-bake the rest into the materials.
|
||||
|
||||
---
|
||||
|
||||
## Image-Based Lighting (IBL) / Environment Light
|
||||
|
||||
For realistic PBR materials you need a cubemap for reflections.
|
||||
|
||||
```python
|
||||
# Environment light from an HDR
|
||||
env = root.create(envlightCOMP, 'env')
|
||||
env.par.envmap = '/project1/cube_in' # path to a TOP that produces a cubemap
|
||||
env.par.envlightmap = ... # diffuse irradiance map (often same as envmap)
|
||||
env.par.dimmer = 1.0
|
||||
|
||||
# Cubemap source — option A: built-in cubeTOP from 6 faces
|
||||
cube = root.create(cubeTOP, 'cube_in')
|
||||
# (assign 6 face TOPs)
|
||||
|
||||
# Option B: HDR equirectangular → cubemap conversion
|
||||
# Use a moviefileinTOP loading .hdr or .exr, then projectTOP type='cubemapfromequirect'
|
||||
hdr = root.create(moviefileinTOP, 'hdr_src')
|
||||
hdr.par.file = '/path/to/environment.hdr'
|
||||
|
||||
proj = root.create(projectTOP, 'cube_proj')
|
||||
proj.par.projecttype = 'cubemapfromequirect'
|
||||
proj.inputConnectors[0].connect(hdr)
|
||||
```
|
||||
|
||||
PBR materials sample the environment automatically when `envlightCOMP` is in the scene. Verify param names with `td_get_par_info(op_type='envlightCOMP')` — TD versions vary.
|
||||
|
||||
---
|
||||
|
||||
## PBR Material Setup
|
||||
|
||||
```python
|
||||
mat = root.create(pbrMAT, 'pbr_metal')
|
||||
mat.par.basecolorr = 0.95; mat.par.basecolorg = 0.65; mat.par.basecolorb = 0.4
|
||||
mat.par.metallic = 1.0
|
||||
mat.par.roughness = 0.25
|
||||
mat.par.specularlevel = 0.5
|
||||
mat.par.emitcolorr = 0; mat.par.emitcolorg = 0; mat.par.emitcolorb = 0
|
||||
|
||||
# Texture maps
|
||||
mat.par.basecolormap = '/project1/textures/albedo' # TOP path
|
||||
mat.par.metallicroughnessmap = '/project1/textures/mr' # G=roughness, B=metallic (glTF convention)
|
||||
mat.par.normalmap = '/project1/textures/normal'
|
||||
mat.par.emitmap = '/project1/textures/emit'
|
||||
mat.par.occlusionmap = '/project1/textures/ao'
|
||||
```
|
||||
|
||||
**Material idioms:**
|
||||
|
||||
| Look | metallic | roughness | basecolor |
|
||||
|---|---|---|---|
|
||||
| Brushed steel | 1.0 | 0.4 | (0.7, 0.7, 0.7) |
|
||||
| Polished gold | 1.0 | 0.1 | (1.0, 0.85, 0.4) |
|
||||
| Plastic | 0.0 | 0.5 | mid-saturated |
|
||||
| Rubber | 0.0 | 0.9 | dark |
|
||||
| Glass | 0.0 | 0.05 | (1, 1, 1), low alpha + transmission |
|
||||
| Glowing emitter | 0.0 | 1.0 | dark, high `emitcolor` |
|
||||
|
||||
For glass/transmission, recent TD versions support `transmission` in PBR; older versions need glslMAT.
|
||||
|
||||
---
|
||||
|
||||
## Multi-Camera Setups
|
||||
|
||||
For comparison views, instant replay, multi-screen mapping, etc.
|
||||
|
||||
```python
|
||||
# Camera A — main scene
|
||||
cam_a = root.create(cameraCOMP, 'cam_main')
|
||||
cam_a.par.tz = 5
|
||||
|
||||
# Camera B — orbiting top-down
|
||||
cam_b = root.create(cameraCOMP, 'cam_top')
|
||||
cam_b.par.ty = 6; cam_b.par.rx = -90
|
||||
|
||||
# Render each via separate Render TOPs
|
||||
render_a = root.create(renderTOP, 'render_main')
|
||||
render_a.par.camera = cam_a.path
|
||||
render_a.par.geometry = geo.path
|
||||
|
||||
render_b = root.create(renderTOP, 'render_top')
|
||||
render_b.par.camera = cam_b.path
|
||||
render_b.par.geometry = geo.path
|
||||
```
|
||||
|
||||
Composite both with a `multiplyTOP`/`compositeTOP` for picture-in-picture, or route to separate `windowCOMP`s for multi-display.
|
||||
|
||||
### Camera animation
|
||||
|
||||
Drive camera params via expressions (orbit), animationCOMP (waypoint), or LFO (oscillation):
|
||||
|
||||
```python
|
||||
# Orbiting camera
|
||||
cam_a.par.tx.mode = ParMode.EXPRESSION
|
||||
cam_a.par.tx.expr = "cos(absTime.seconds * 0.3) * 6"
|
||||
cam_a.par.tz.mode = ParMode.EXPRESSION
|
||||
cam_a.par.tz.expr = "sin(absTime.seconds * 0.3) * 6"
|
||||
cam_a.par.lookat = '/project1/scene_geo' # auto-aim at target
|
||||
```
|
||||
|
||||
`par.lookat` is the simplest "always look at target" mechanism.
|
||||
|
||||
### Depth of field
|
||||
|
||||
PBR + Render TOP supports DOF when `par.dof = 'on'`.
|
||||
|
||||
```python
|
||||
render.par.dof = 'on'
|
||||
render.par.focusdistance = 5.0
|
||||
render.par.aperture = 0.05 # blur strength
|
||||
render.par.bokehshape = 'hexagon'
|
||||
```
|
||||
|
||||
DOF is GPU-heavy. Render at lower res then upscale for performance.
|
||||
|
||||
---
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **Render TOP shows black** — most common cause: no light. Even with PBR you need at least one `lightCOMP` or `envlightCOMP`. Add an `ambientlightCOMP` at low dimmer as a safety net.
|
||||
2. **Material doesn't appear** — `geo.par.material` must be a string PATH, not the material op itself. Use `mat.path`, not `mat`.
|
||||
3. **Lights ignored** — by default Render TOP picks up ALL `lightCOMP`s in the network. If you have leftover lights from another scene, they leak in. Set `par.lights` explicitly.
|
||||
4. **PBR looks flat** — without an `envlightCOMP` providing reflections, PBR materials look like Phong. Add one even if you don't have an HDR (use a `constantTOP` cubemap as fallback).
|
||||
5. **Shadow acne / striping** — increase `par.shadowbias` slightly. Tune per-light.
|
||||
6. **Camera inside geometry** — if `cam.par.tz` is INSIDE a sphere, you see the inside (or nothing if backface culled). Move the camera further out.
|
||||
7. **Light range too small** — point lights have implicit attenuation. Far-away geometry receives little light. Increase `par.dimmer` or move lights closer.
|
||||
8. **Multiple cameras conflict** — one render TOP = one camera. Don't try to share. Use multiple render TOPs.
|
||||
9. **Wrong handedness** — TD is right-handed Y-up. Imported assets from Z-up apps (Blender, Maya in Z-up) need a 90° X rotation on the geo COMP.
|
||||
10. **Cooking budget** — PBR + IBL + shadows + DOF at 1080p60 is fine on modern GPUs but 4K + 4 lights + soft shadows + DOF will tank. Profile via `td_get_perf` and downgrade settings before adding more.
|
||||
|
||||
---
|
||||
|
||||
## Quick Recipes
|
||||
|
||||
| Goal | Recipe |
|
||||
|---|---|
|
||||
| Studio portrait | 3-point rig (key + fill + rim) + ambient + PBR mat + DOF |
|
||||
| Outdoor daylight | One directional `lightCOMP` (sun) + envlight (sky HDR) + soft shadows |
|
||||
| Dramatic / film noir | Single spot light from upper side, hard shadows, deep ambient = 0.05 |
|
||||
| Abstract / dreamy | Multiple area lights at low dimmer, no shadows, `bloomTOP` post |
|
||||
| Product render | Three-point + IBL + neutral PBR + `bgcolorr=g=b=1` (white seamless) |
|
||||
| Game-style | Phong MAT + 1-2 lights + no IBL + flat ambient (cheap, stylized) |
|
||||
| Wireframe + solid | Two render TOPs (one with wireframeMAT, one with PBR), composite via `addTOP` |
|
||||
| Orbiting camera | `par.lookat` + expressions on tx/tz using sin/cos |
|
||||
@@ -0,0 +1,221 @@
|
||||
# Animation Reference
|
||||
|
||||
Patterns for time-based motion — keyframes, LFOs, timers, easing, expression-driven animation.
|
||||
|
||||
Always call `td_get_par_info` for the op type before setting params. Param names below reflect TD 2025.32 but verify if errors fire.
|
||||
|
||||
---
|
||||
|
||||
## Time Sources
|
||||
|
||||
TD has three time references — pick the right one.
|
||||
|
||||
| Expression | Behavior | Use for |
|
||||
|---|---|---|
|
||||
| `absTime.seconds` | Wall-clock seconds since TD started. Never resets. | Continuous motion, GLSL `uTime`, infinite loops |
|
||||
| `absTime.frame` | Wall-clock frame count. | Frame-accurate triggers |
|
||||
| `me.time.frame` | Local component frame count (resets on play/stop). | Per-COMP animation timeline |
|
||||
| `me.time.seconds` | Local component seconds. | Same, in seconds |
|
||||
|
||||
**Rule:** for shaders and continuous motion use `absTime.seconds`. For triggered/looping animations inside a COMP use `me.time.*`.
|
||||
|
||||
---
|
||||
|
||||
## LFO CHOP — Cyclic Motion
|
||||
|
||||
The simplest periodic driver. Fast, GPU-cheap, expression-friendly.
|
||||
|
||||
```python
|
||||
lfo = root.create(lfoCHOP, 'rot_driver')
|
||||
lfo.par.type = 'sin' # 'sin' | 'cos' | 'ramp' | 'square' | 'triangle' | 'pulse'
|
||||
lfo.par.frequency = 0.25 # cycles per second
|
||||
lfo.par.amplitude = 1.0
|
||||
lfo.par.offset = 0.0
|
||||
lfo.par.phase = 0.0 # 0-1, useful for offsetting parallel LFOs
|
||||
```
|
||||
|
||||
**Drive a parameter via export:**
|
||||
|
||||
```python
|
||||
op('/project1/geo1').par.rx.mode = ParMode.EXPRESSION
|
||||
op('/project1/geo1').par.rx.expr = "op('rot_driver')['chan1'] * 360"
|
||||
```
|
||||
|
||||
**Multiple synced LFOs (X/Y/Z rotation with phase offsets):**
|
||||
Create one LFO with three channels and phase-offset each, or use three LFOs and offset their `phase` params (0.0, 0.33, 0.66).
|
||||
|
||||
---
|
||||
|
||||
## Timer CHOP — Triggered Sequences
|
||||
|
||||
For run-once animations, beat-locked sequences, or stage-based logic.
|
||||
|
||||
```python
|
||||
timer = root.create(timerCHOP, 'fade_timer')
|
||||
timer.par.length = 4.0 # cycle length in seconds
|
||||
timer.par.cycle = False # run once vs. loop
|
||||
timer.par.outputseconds = True
|
||||
```
|
||||
|
||||
Output channels: `timer_fraction` (0→1 across the cycle), `running`, `done`, `cycles`.
|
||||
|
||||
**Start the timer:**
|
||||
```python
|
||||
timer.par.start.pulse()
|
||||
```
|
||||
|
||||
**Drive a fade:**
|
||||
```python
|
||||
op('/project1/level1').par.opacity.mode = ParMode.EXPRESSION
|
||||
op('/project1/level1').par.opacity.expr = "op('fade_timer')['timer_fraction']"
|
||||
```
|
||||
|
||||
**Easing on the timer fraction** — apply in the expression itself:
|
||||
|
||||
```python
|
||||
# Smoothstep: ease in/out
|
||||
expr = "smoothstep(0, 1, op('fade_timer')['timer_fraction'])"
|
||||
# Cubic ease-out: 1 - (1-t)^3
|
||||
expr = "1 - pow(1 - op('fade_timer')['timer_fraction'], 3)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Pattern CHOP — Custom Curves
|
||||
|
||||
For arbitrary waveforms (saw ramps, easing curves, custom envelopes).
|
||||
|
||||
```python
|
||||
pat = root.create(patternCHOP, 'envelope')
|
||||
pat.par.type = 'gaussian' # 'gaussian' | 'ramp' | 'square' | 'sin' | etc.
|
||||
pat.par.length = 60 # samples
|
||||
pat.par.cyclelength = 1.0 # seconds at TD framerate
|
||||
```
|
||||
|
||||
Combine with `lookupCHOP` to remap a 0-1 driver through a custom curve.
|
||||
|
||||
---
|
||||
|
||||
## Animation COMP — Keyframe-Based
|
||||
|
||||
For multi-keyframe motion graphics. Each animationCOMP holds channels with keyframes editable in the Animation Editor.
|
||||
|
||||
```python
|
||||
anim = root.create(animationCOMP, 'intro_anim')
|
||||
# By default has channels chan1..chanN; access via:
|
||||
# op('intro_anim').par.length, .par.play, .par.cue, etc.
|
||||
|
||||
# Drive a parameter from a channel
|
||||
op('/project1/text1').par.tx.mode = ParMode.EXPRESSION
|
||||
op('/project1/text1').par.tx.expr = "op('intro_anim/out1')['chan1']"
|
||||
```
|
||||
|
||||
**Keyframes are typically edited in the UI** (Animation Editor), but can be set via `keyframes` table internally. For programmatic keyframe creation, use `td_execute_python`:
|
||||
|
||||
```python
|
||||
# Get the channel CHOP inside an animationCOMP
|
||||
ch = op('/project1/intro_anim/chans')
|
||||
# Insert a key (advanced API — verify with td_get_par_info(op_type='animationCOMP'))
|
||||
ch.appendKey('chan1', frame=0, value=0.0, expression=None)
|
||||
ch.appendKey('chan1', frame=120, value=1.0)
|
||||
```
|
||||
|
||||
For most use cases, drive params with LFO/Timer/Pattern CHOPs instead — simpler and scriptable.
|
||||
|
||||
---
|
||||
|
||||
## Easing in Expressions
|
||||
|
||||
TD's expression evaluator supports Python math. Common easing forms:
|
||||
|
||||
```python
|
||||
# Linear
|
||||
"t"
|
||||
|
||||
# Smoothstep (classic ease-in-out)
|
||||
"smoothstep(0, 1, t)"
|
||||
|
||||
# Ease-out cubic
|
||||
"1 - pow(1 - t, 3)"
|
||||
|
||||
# Ease-in cubic
|
||||
"pow(t, 3)"
|
||||
|
||||
# Ease-in-out cubic
|
||||
"3*t*t - 2*t*t*t"
|
||||
|
||||
# Bounce (manual, simplified)
|
||||
"abs(sin(t * 6.28 * 3) * (1 - t))"
|
||||
```
|
||||
|
||||
Where `t` is `op('fade_timer')['timer_fraction']` or any 0-1 driver.
|
||||
|
||||
---
|
||||
|
||||
## Filter CHOP — Smoothing Existing Channels
|
||||
|
||||
Smooth out jittery values (e.g., audio analysis, sensor data) before driving visuals.
|
||||
|
||||
```python
|
||||
filt = root.create(filterCHOP, 'smooth')
|
||||
filt.par.filter = 'gaussian' # or 'lowpass'
|
||||
filt.par.width = 0.5 # smoothing window in seconds
|
||||
filt.inputConnectors[0].connect(op('raw_signal'))
|
||||
```
|
||||
|
||||
**WARNING:** Do NOT use Filter CHOP on AudioSpectrum output in timeslice mode — it expands the sample count and averages bins to near-zero. See `audio-reactive.md`.
|
||||
|
||||
---
|
||||
|
||||
## Lag CHOP — Asymmetric Attack/Release
|
||||
|
||||
Different speeds for rising vs. falling values. Standard for visualizing audio envelopes.
|
||||
|
||||
```python
|
||||
lag = root.create(lagCHOP, 'env_smooth')
|
||||
lag.par.lag1 = 0.02 # attack (rise time, seconds)
|
||||
lag.par.lag2 = 0.30 # release (fall time, seconds)
|
||||
lag.inputConnectors[0].connect(op('raw_envelope'))
|
||||
```
|
||||
|
||||
Fast attack, slow release = classic VU-meter feel.
|
||||
|
||||
---
|
||||
|
||||
## Per-Frame Driving via Script DAT
|
||||
|
||||
For complex per-frame logic that doesn't fit expressions, use a `executeDAT` (`onFrameStart` callback) or a `chopExecuteDAT`.
|
||||
|
||||
```python
|
||||
# In an executeDAT (frameStart):
|
||||
def onFrameStart(frame):
|
||||
t = absTime.seconds
|
||||
op('/project1/circle').par.tx = math.sin(t * 2.0) * 3.0
|
||||
op('/project1/circle').par.ty = math.cos(t * 2.0) * 3.0
|
||||
return
|
||||
```
|
||||
|
||||
Heavy logic should still be in CHOPs (CPU-cheap, deterministic). Reserve scripts for one-shots or non-realtime branching.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **Frame rate dependency** — `me.time.frame` is in TD project frames (default 60). If your project rate changes, motion speed changes. Use `seconds` for rate-independent timing.
|
||||
2. **Cooking budget** — every CHOP that drives a parameter cooks every frame. Consolidate drivers (one big mathCHOP > many small ones).
|
||||
3. **Expression mode** — params default to `CONSTANT`. `par.X.expr = ...` is ignored unless `par.X.mode = ParMode.EXPRESSION`.
|
||||
4. **Animation editor edits** — keyframes set via UI live in the animationCOMP's internal keyframe table. They survive save/reopen. Programmatic keys via `appendKey()` work but verify the API with `td_get_docs(topic='animation')` first.
|
||||
5. **Looping animations** — for seamless loops, `length` must equal `cyclelength` and the start/end values must match. Otherwise expect a visible jump.
|
||||
|
||||
---
|
||||
|
||||
## Quick Recipes
|
||||
|
||||
| Goal | Simplest path |
|
||||
|---|---|
|
||||
| Continuous rotation | LFO CHOP `type='ramp'`, expr → `geo.par.rx` |
|
||||
| Fade in over 2s | Timer CHOP `length=2`, smoothstep expr → `level.par.opacity` |
|
||||
| Pulse on every beat | `triggerCHOP` from audio → drive scale via expression |
|
||||
| 3D Lissajous orbit | Two LFOs with different freq, drive `tx`/`ty`/`tz` |
|
||||
| Random jitter | `noiseCHOP` (low-freq) added to position |
|
||||
| Timed scene switch | Timer CHOP → switchTOP/CHOP `index` |
|
||||
@@ -0,0 +1,175 @@
|
||||
# Audio-Reactive Reference
|
||||
|
||||
Patterns for driving visuals from audio — spectrum analysis, beat detection, envelope following.
|
||||
|
||||
## Audio Input
|
||||
|
||||
```python
|
||||
# Live input from audio interface
|
||||
audio_in = root.create(audiodeviceinCHOP, 'audio_in')
|
||||
audio_in.par.rate = 44100
|
||||
|
||||
# OR: from audio file (for testing)
|
||||
audio_file = root.create(audiofileinCHOP, 'audio_in')
|
||||
audio_file.par.file = '/path/to/track.wav'
|
||||
audio_file.par.play = True
|
||||
audio_file.par.repeat = 'on' # NOT par.loop
|
||||
audio_file.par.playmode = 'locked'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Audio Band Extraction (Verified TD 2025.32460)
|
||||
|
||||
Use `audiofilterCHOP` for band separation (NOT `selectCHOP` by channel index):
|
||||
|
||||
```python
|
||||
# Audio input
|
||||
af = root.create(audiofileinCHOP, 'audio_in')
|
||||
af.par.file = path
|
||||
af.par.play = True
|
||||
af.par.repeat = 'on'
|
||||
af.par.playmode = 'locked'
|
||||
|
||||
# Low band: lowpass @ 250Hz
|
||||
flt_low = root.create(audiofilterCHOP, 'flt_low')
|
||||
flt_low.par.filter = 'lowpass'
|
||||
flt_low.par.cutofffrequency = 250
|
||||
flt_low.par.rolloff = 2
|
||||
flt_low.inputConnectors[0].connect(af)
|
||||
|
||||
# Mid band: highpass@250 → lowpass@4000
|
||||
flt_mid_hp = root.create(audiofilterCHOP, 'flt_mid_hp')
|
||||
flt_mid_hp.par.filter = 'highpass'
|
||||
flt_mid_hp.par.cutofffrequency = 250
|
||||
flt_mid_hp.par.rolloff = 2
|
||||
flt_mid_hp.inputConnectors[0].connect(af)
|
||||
|
||||
flt_mid_lp = root.create(audiofilterCHOP, 'flt_mid_lp')
|
||||
flt_mid_lp.par.filter = 'lowpass'
|
||||
flt_mid_lp.par.cutofffrequency = 4000
|
||||
flt_mid_lp.par.rolloff = 2
|
||||
flt_mid_lp.inputConnectors[0].connect(flt_mid_hp)
|
||||
|
||||
# High band: highpass @ 4000Hz
|
||||
flt_high = root.create(audiofilterCHOP, 'flt_high')
|
||||
flt_high.par.filter = 'highpass'
|
||||
flt_high.par.cutofffrequency = 4000
|
||||
flt_high.par.rolloff = 2
|
||||
flt_high.inputConnectors[0].connect(af)
|
||||
|
||||
# Per-band: RMS → lag → gain → clamp
|
||||
for name, filt in [('low', flt_low), ('mid', flt_mid_lp), ('high', flt_high)]:
|
||||
rms = root.create(analyzeCHOP, f'rms_{name}')
|
||||
rms.par.function = 'rmspower' # NOT 'rms'
|
||||
rms.inputConnectors[0].connect(filt)
|
||||
|
||||
lag = root.create(lagCHOP, f'lag_{name}')
|
||||
lag.par.lag1 = 0.05 # attack (NOT par.lagin)
|
||||
lag.par.lag2 = 0.25 # release (NOT par.lagout)
|
||||
lag.inputConnectors[0].connect(rms)
|
||||
|
||||
math = root.create(mathCHOP, f'scale_{name}')
|
||||
math.par.gain = 8.0
|
||||
math.inputConnectors[0].connect(lag)
|
||||
|
||||
# mathCHOP has NO par.clamp — use limitCHOP
|
||||
lim = root.create(limitCHOP, f'clamp_{name}')
|
||||
lim.par.type = 'clamp'
|
||||
lim.par.min = 0.0
|
||||
lim.par.max = 1.0
|
||||
lim.inputConnectors[0].connect(math)
|
||||
|
||||
null = root.create(nullCHOP, f'out_{name}')
|
||||
null.inputConnectors[0].connect(lim)
|
||||
null.viewer = True
|
||||
```
|
||||
|
||||
**Key TD 2025 corrections:**
|
||||
- `analyzeCHOP.par.function = 'rmspower'` NOT `'rms'`
|
||||
- `lagCHOP.par.lag1` / `par.lag2` NOT `par.lagin` / `par.lagout`
|
||||
- `mathCHOP` has NO `par.clamp` — use separate `limitCHOP`
|
||||
|
||||
---
|
||||
|
||||
## Beat / Onset Detection
|
||||
|
||||
### Kick Detection (slope → trigger)
|
||||
|
||||
```python
|
||||
slope = root.create(slopeCHOP, 'kick_slope')
|
||||
slope.inputConnectors[0].connect(op('out_low'))
|
||||
|
||||
trig = root.create(triggerCHOP, 'kick_trig')
|
||||
trig.par.threshold = 0.12
|
||||
trig.par.attack = 0.005 # NOT par.attacktime
|
||||
trig.par.decay = 0.15 # NOT par.decaytime
|
||||
trig.par.triggeron = 'increase'
|
||||
trig.inputConnectors[0].connect(slope)
|
||||
|
||||
kick_out = root.create(nullCHOP, 'out_kick')
|
||||
kick_out.inputConnectors[0].connect(trig)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Passing Audio to GLSL
|
||||
|
||||
```python
|
||||
glsl.par.vec0name = 'uLow'
|
||||
glsl.par.vec0valuex.expr = "op('out_low')['chan1']"
|
||||
glsl.par.vec0valuex.mode = ParMode.EXPRESSION
|
||||
|
||||
glsl.par.vec1name = 'uKick'
|
||||
glsl.par.vec1valuex.expr = "op('out_kick')['chan1']"
|
||||
glsl.par.vec1valuex.mode = ParMode.EXPRESSION
|
||||
```
|
||||
|
||||
```glsl
|
||||
uniform float uLow;
|
||||
uniform float uKick;
|
||||
float scale = 1.0 + uKick * 0.4 + uLow * 0.2;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Standard Audio Bus Pattern
|
||||
|
||||
Recommended structure:
|
||||
|
||||
```
|
||||
audiodeviceinCHOP (audio_in)
|
||||
↓
|
||||
[null_audio_in]
|
||||
├──→ audiofilterCHOP (lowpass@250) → analyzeCHOP → lagCHOP → mathCHOP → limitCHOP → null
|
||||
├──→ audiofilterCHOP (bandpass@250-4k) → analyzeCHOP → lagCHOP → mathCHOP → limitCHOP → null
|
||||
├──→ audiofilterCHOP (highpass@4k) → analyzeCHOP → lagCHOP → mathCHOP → limitCHOP → null
|
||||
│
|
||||
└──→ slopeCHOP → triggerCHOP (beat_trigger)
|
||||
```
|
||||
|
||||
Keep this entire bus inside a `baseCOMP` (e.g., `audio_bus`) and reference via paths from visual networks.
|
||||
|
||||
---
|
||||
|
||||
## MIDI Input
|
||||
|
||||
```python
|
||||
midi_in = root.create(midiinCHOP, 'midi_in')
|
||||
midi_in.par.device = 0 # Check midiinDAT for device index
|
||||
# Outputs channels named by MIDI note/CC: 'ch1n60', 'ch1c74', etc.
|
||||
|
||||
# Map CC to a parameter
|
||||
op('bloom1').par.threshold.mode = ParMode.EXPRESSION
|
||||
op('bloom1').par.threshold.expr = "op('midi_in')['ch1c74'][0]"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CRITICAL: DO NOT use Lag CHOP for spectrum smoothing
|
||||
|
||||
Lag CHOP in timeslice mode expands 256-sample spectrum to 1600-2400 samples, averaging all values to near-zero (~1e-06). The shader receives no usable data. Use `mathCHOP(gain=8)` directly, or smooth in GLSL via temporal lerp with a feedback texture.
|
||||
|
||||
Verified:
|
||||
- Without Lag CHOP: bass bins = 5.0-5.4 (strong, usable)
|
||||
- With Lag CHOP: ALL bins = 0.000001 (dead)
|
||||
@@ -0,0 +1,352 @@
|
||||
# DAT-Based Scripting Reference
|
||||
|
||||
TD's event/callback model — Python that runs in response to network events. The full set of "Execute DATs" plus their idiomatic patterns.
|
||||
|
||||
For arbitrary Python execution (not callback-based), see `python-api.md`. For the MCP's `td_execute_python` tool, see `mcp-tools.md`.
|
||||
|
||||
---
|
||||
|
||||
## The Execute DAT Family
|
||||
|
||||
Every type watches one kind of event source and fires Python on changes.
|
||||
|
||||
| DAT | Watches | Use for |
|
||||
|---|---|---|
|
||||
| `chopExecuteDAT` | A CHOP's channel values | Audio triggers, threshold callbacks, state machines on numeric input |
|
||||
| `datExecuteDAT` | A DAT's content (table cells, text) | Reacting to data updates from APIs, parsing webDAT responses |
|
||||
| `parameterExecuteDAT` | A parameter's value or pulse | Reacting to user-changed params, custom pulse buttons |
|
||||
| `panelExecuteDAT` | A panel COMP's interaction | Button clicks, slider drags, field commits |
|
||||
| `opExecuteDAT` | Operator lifecycle | New operator created, deleted, name changed |
|
||||
| `executeDAT` | Project lifecycle, frame events | Run-once setup, per-frame logic, save/load hooks |
|
||||
|
||||
All have a docked DAT with predefined callback functions. You only fill in the bodies of the ones you care about.
|
||||
|
||||
---
|
||||
|
||||
## chopExecuteDAT — Numeric Triggers
|
||||
|
||||
```python
|
||||
ce = root.create(chopExecuteDAT, 'kick_handler')
|
||||
ce.par.chop = '/project1/audio/out_kick' # source CHOP
|
||||
ce.par.offtoon = True # fire when channel rises above 0
|
||||
ce.par.ontooff = False
|
||||
ce.par.whileon = False
|
||||
ce.par.valuechange = False
|
||||
```
|
||||
|
||||
In the docked callback DAT:
|
||||
|
||||
```python
|
||||
def offToOn(channel, sampleIndex, val, prev):
|
||||
"""Channel went from 0 to non-zero. Classic beat trigger."""
|
||||
op('/project1/strobe').par.flash.pulse()
|
||||
op('/project1/scene').par.index = (op('/project1/scene').par.index + 1) % 8
|
||||
return
|
||||
|
||||
def onToOff(channel, sampleIndex, val, prev):
|
||||
"""Channel went from non-zero to 0."""
|
||||
return
|
||||
|
||||
def whileOn(channel, sampleIndex, val, prev):
|
||||
"""Fires every frame while channel is non-zero. Use sparingly."""
|
||||
return
|
||||
|
||||
def valueChange(channel, sampleIndex, val, prev):
|
||||
"""Fires every frame the value changes (continuous). Heavy."""
|
||||
return
|
||||
```
|
||||
|
||||
`channel` is a `Channel` object — `.name`, `.owner`, `.vals[]`. Use `channel.name == 'chan1'` to filter.
|
||||
|
||||
**Threshold-based custom triggers:** wire the source CHOP through a `triggerCHOP` first to get clean 0/1 pulses, then watch with `offtoon`.
|
||||
|
||||
---
|
||||
|
||||
## datExecuteDAT — Table/Text Changes
|
||||
|
||||
```python
|
||||
de = root.create(datExecuteDAT, 'api_response')
|
||||
de.par.dat = '/project1/api/web1' # source DAT
|
||||
de.par.tablechange = True # any cell change
|
||||
de.par.cellchange = False
|
||||
de.par.rowchange = False
|
||||
de.par.colchange = False
|
||||
```
|
||||
|
||||
```python
|
||||
def onTableChange(dat):
|
||||
"""Whole table changed (including text DAT content updates)."""
|
||||
if dat.numRows == 0:
|
||||
return
|
||||
# If it's a webDAT response, parse JSON
|
||||
import json
|
||||
try:
|
||||
data = json.loads(dat.text)
|
||||
except json.JSONDecodeError:
|
||||
debug(f'Bad JSON: {dat.text[:100]}')
|
||||
return
|
||||
# Write to a CHOP
|
||||
op('/project1/api_value').par.value0 = float(data.get('count', 0))
|
||||
return
|
||||
|
||||
def onCellChange(dat, cells, prev):
|
||||
"""Specific cells changed."""
|
||||
for cell in cells:
|
||||
# cell.row, cell.col, cell.val
|
||||
pass
|
||||
return
|
||||
```
|
||||
|
||||
`debug()` prints to the textport — readable via `td_read_textport`.
|
||||
|
||||
---
|
||||
|
||||
## parameterExecuteDAT — Param Changes & Pulse
|
||||
|
||||
```python
|
||||
pe = root.create(parameterExecuteDAT, 'comp_params')
|
||||
pe.par.op = '/project1/my_component' # COMP whose params to watch
|
||||
pe.par.parameters = '*' # or specific names like 'Intensity Reset'
|
||||
pe.par.valuechange = True
|
||||
pe.par.pulse = True
|
||||
```
|
||||
|
||||
```python
|
||||
def onValueChange(par, prev):
|
||||
"""par is a Par object. par.name, par.eval(), par.owner."""
|
||||
if par.name == 'Intensity':
|
||||
op('/project1/bloom').par.threshold = par.eval()
|
||||
return
|
||||
|
||||
def onPulse(par):
|
||||
"""Pulse param was triggered."""
|
||||
if par.name == 'Reset':
|
||||
op('/project1/scene').par.index = 0
|
||||
op('/project1/audio_player').par.cuepoint = 0
|
||||
op('/project1/audio_player').par.cuepulse.pulse()
|
||||
return
|
||||
|
||||
def onExpressionChange(par, val, prev):
|
||||
"""User changed the expression on a param."""
|
||||
return
|
||||
|
||||
def onExportChange(par, val, prev):
|
||||
"""Export source changed."""
|
||||
return
|
||||
|
||||
def onModeChange(par, val, prev):
|
||||
"""Param mode changed (CONSTANT / EXPRESSION / EXPORT / etc)."""
|
||||
return
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## panelExecuteDAT — UI Events
|
||||
|
||||
For interactive control surfaces. See `panel-ui.md` for the full panel COMP context.
|
||||
|
||||
```python
|
||||
pe = root.create(panelExecuteDAT, 'btn_handler')
|
||||
pe.par.panel = '/project1/play_btn'
|
||||
pe.par.click = True # mouse click events
|
||||
pe.par.value = True # state changes (toggle)
|
||||
pe.par.lockedchange = False
|
||||
```
|
||||
|
||||
```python
|
||||
def onOffToOn(panelValue):
|
||||
"""Panel value rose to 1 (button pressed, slider crossed threshold)."""
|
||||
op('/project1/scene_timer').par.start.pulse()
|
||||
return
|
||||
|
||||
def onOnToOff(panelValue):
|
||||
"""Panel value dropped to 0."""
|
||||
return
|
||||
|
||||
def onValueChange(panelValue):
|
||||
"""Continuous: every frame the value changes."""
|
||||
val = panelValue.eval()
|
||||
op('/project1/master').par.opacity = val
|
||||
return
|
||||
|
||||
def onClick(panelValue):
|
||||
"""Discrete click event, fires once per click."""
|
||||
return
|
||||
```
|
||||
|
||||
`panelValue` is a `Par` object on the panel COMP.
|
||||
|
||||
---
|
||||
|
||||
## opExecuteDAT — Operator Lifecycle
|
||||
|
||||
Watches creation/deletion/renaming of operators in a parent COMP.
|
||||
|
||||
```python
|
||||
oe = root.create(opExecuteDAT, 'lifecycle')
|
||||
oe.par.op = '/project1'
|
||||
oe.par.create = True
|
||||
oe.par.destroy = True
|
||||
oe.par.namechange = True
|
||||
oe.par.flagchange = False
|
||||
```
|
||||
|
||||
```python
|
||||
def onCreate(opCreated):
|
||||
"""A new operator was created. Useful for auto-applying conventions."""
|
||||
if opCreated.OPType == 'glslTOP':
|
||||
# Always wrap with a null
|
||||
n = opCreated.parent().create(nullTOP, opCreated.name + '_out')
|
||||
n.inputConnectors[0].connect(opCreated)
|
||||
return
|
||||
|
||||
def onDestroy(opDestroyed):
|
||||
"""Operator was deleted. opDestroyed.path is still valid for one frame."""
|
||||
return
|
||||
|
||||
def onNameChange(opChanged):
|
||||
"""Operator was renamed."""
|
||||
return
|
||||
```
|
||||
|
||||
Useful for dev-time scaffolding (auto-create downstream nullTOPs, auto-name conventions). Disable in production projects to avoid surprise side effects.
|
||||
|
||||
---
|
||||
|
||||
## executeDAT — Project Lifecycle & Per-Frame
|
||||
|
||||
The catch-all. Gets you hooks into project start, save, load, frame-start, frame-end.
|
||||
|
||||
```python
|
||||
exec_dat = root.create(executeDAT, 'lifecycle')
|
||||
exec_dat.par.start = True
|
||||
exec_dat.par.create = True
|
||||
exec_dat.par.framestart = True
|
||||
exec_dat.par.frameend = False
|
||||
```
|
||||
|
||||
```python
|
||||
def onStart():
|
||||
"""Project just started cooking. Run once."""
|
||||
op('/project1/scene').par.index = 0
|
||||
debug('Project started')
|
||||
return
|
||||
|
||||
def onCreate():
|
||||
"""Component was just created (only fires for component executeDATs, not project root)."""
|
||||
return
|
||||
|
||||
def onFrameStart(frame):
|
||||
"""Per-frame, BEFORE network cooks. Heavy logic here = bottleneck."""
|
||||
return
|
||||
|
||||
def onFrameEnd(frame):
|
||||
"""Per-frame, AFTER network cooks. Use for capture, recording, post-network logic."""
|
||||
return
|
||||
|
||||
def onPlayStateChange(playing):
|
||||
"""Project play/pause toggled."""
|
||||
return
|
||||
|
||||
def onProjectPreSave():
|
||||
"""Right before saving the .toe file."""
|
||||
return
|
||||
|
||||
def onProjectPostSave():
|
||||
return
|
||||
```
|
||||
|
||||
Heavy per-frame logic in `onFrameStart` is one of the top performance regressions in TD projects. Use CHOPs for per-frame computation, scripts for events.
|
||||
|
||||
---
|
||||
|
||||
## Pattern: Triggering an Animation Sequence on Beat
|
||||
|
||||
```python
|
||||
# Source: a kick trigger CHOP
|
||||
# Goal: on each kick, run a 1.5s scale pulse + color flash
|
||||
|
||||
# Setup (create once)
|
||||
animator = root.create(timerCHOP, 'pulse_anim')
|
||||
animator.par.length = 1.5
|
||||
animator.par.cycle = False
|
||||
|
||||
# Param expressions on visual targets:
|
||||
op('logo').par.sx.expr = "1.0 + (1 - op('pulse_anim')['timer_fraction']) * 0.3"
|
||||
op('logo').par.sx.mode = ParMode.EXPRESSION
|
||||
op('logo').par.sy.expr = "1.0 + (1 - op('pulse_anim')['timer_fraction']) * 0.3"
|
||||
op('logo').par.sy.mode = ParMode.EXPRESSION
|
||||
|
||||
# In a chopExecuteDAT watching the kick CHOP:
|
||||
def offToOn(channel, sampleIndex, val, prev):
|
||||
op('pulse_anim').par.start.pulse()
|
||||
return
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Pattern: Live Editing a CHOP from API Data
|
||||
|
||||
```python
|
||||
# webDAT polls an API every 5 seconds
|
||||
# datExecuteDAT parses the response and writes to a constantCHOP
|
||||
|
||||
def onTableChange(dat):
|
||||
import json
|
||||
try:
|
||||
data = json.loads(dat.text)
|
||||
except:
|
||||
return
|
||||
target = op('/project1/external_state')
|
||||
target.par.name0 = 'temperature'
|
||||
target.par.value0 = float(data['temp_c'])
|
||||
target.par.name1 = 'humidity'
|
||||
target.par.value1 = float(data['humidity'])
|
||||
return
|
||||
```
|
||||
|
||||
Visuals just reference `op('external_state')['temperature']` — they update live.
|
||||
|
||||
---
|
||||
|
||||
## Pattern: Self-Cleaning Network
|
||||
|
||||
```python
|
||||
# An opExecuteDAT watching for orphaned helper ops, deleting them after their parent disappears
|
||||
|
||||
def onDestroy(opDestroyed):
|
||||
parent_name = opDestroyed.name
|
||||
helper = op(f'/project1/{parent_name}_helper')
|
||||
if helper:
|
||||
helper.destroy()
|
||||
return
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **Callbacks crash silently** — exceptions print to the textport but don't show up in the UI. Always `td_clear_textport` before debugging, then `td_read_textport` after.
|
||||
2. **`debug()` vs `print()`** — both write to textport, but `debug()` includes the file/line of the calling DAT. Prefer `debug()` for scripts.
|
||||
3. **`val` is the new value, `prev` is old** — easy to swap. Always: `def offToOn(channel, sampleIndex, val, prev)`. Check parameter order in TD docs if confused.
|
||||
4. **`whileOn` and `valueChange` are per-frame** — heavy. Avoid unless absolutely needed. Drive via expressions instead.
|
||||
5. **Callbacks don't run during cooking-paused state** — if the parent COMP has `allowCooking=False`, callbacks freeze. Useful for "disable me" toggles.
|
||||
6. **`par` vs `panelValue`** — parameterExecuteDAT gives `par` (a Par object), panelExecuteDAT gives `panelValue` (also a Par-like object). Both have `.name` and `.eval()` but their context differs.
|
||||
7. **`opExecuteDAT` fires for itself** — when you create an opExecuteDAT, it can fire `onCreate` for itself if `par.create=True` and parent matches. Filter by `if opCreated == me: return`.
|
||||
8. **Reload behavior** — when reloading an extension (`td_reinit_extension`), all callback DATs reset their internal state. Module-level vars are lost. Persist state in tableDATs or the docked DAT itself, not in module globals.
|
||||
9. **Cooking dependencies** — if a callback writes to an op that's upstream of the callback's source, you get a cooking loop. TD warns about it but doesn't always block. Keep dataflow one-directional.
|
||||
10. **Active flag** — every Execute DAT has `par.active`. False = silent. Easy to toggle for testing without deleting wiring.
|
||||
|
||||
---
|
||||
|
||||
## Quick Recipes
|
||||
|
||||
| Goal | Setup |
|
||||
|---|---|
|
||||
| Beat trigger | `chopExecuteDAT.par.offtoon=True` watching a `triggerCHOP` |
|
||||
| API response handler | `datExecuteDAT.par.tablechange=True` watching a `webDAT` |
|
||||
| Custom button → action | `parameterExecuteDAT.par.pulse=True` watching a custom pulse param |
|
||||
| Slider → continuous param | `panelExecuteDAT.par.value=True` watching a `sliderCOMP` |
|
||||
| Run-once setup | `executeDAT.par.start=True` with logic in `onStart()` |
|
||||
| Per-frame metrics | `executeDAT.par.frameend=True` recording values to a CHOP |
|
||||
| Auto-name new ops | `opExecuteDAT.par.create=True` enforcing naming conventions |
|
||||
@@ -0,0 +1,322 @@
|
||||
# External Data Reference
|
||||
|
||||
Network and device I/O — HTTP requests, WebSockets, MQTT, Serial, TCP, UDP. For MIDI/OSC specifically see `midi-osc.md`.
|
||||
|
||||
Common production needs:
|
||||
- API polling / webhook ingestion
|
||||
- Real-time data streams (sensors, market data, chat)
|
||||
- IoT device control (Arduino, ESP32, smart lights)
|
||||
- Inter-application messaging
|
||||
- Hosting a tiny TD-side HTTP server for remote control
|
||||
|
||||
---
|
||||
|
||||
## Web DAT — HTTP Requests
|
||||
|
||||
```python
|
||||
web = root.create(webDAT, 'api_call')
|
||||
web.par.url = 'https://api.example.com/v1/status'
|
||||
web.par.fetchmethod = 'get' # 'get' | 'post' | 'put' | 'delete'
|
||||
web.par.format = 'auto' # 'auto' | 'text' | 'json'
|
||||
web.par.timeout = 5.0
|
||||
```
|
||||
|
||||
**Triggering a request:**
|
||||
|
||||
`webDAT` does NOT auto-fetch on cook. Trigger explicitly:
|
||||
|
||||
```python
|
||||
web.par.fetch.pulse()
|
||||
```
|
||||
|
||||
Or via expression on a CHOP value-change (chopExecuteDAT — see `dat-scripting.md`).
|
||||
|
||||
**Authentication headers:**
|
||||
|
||||
Use `webclientDAT` (more flexible) or set `webDAT` headers via the headers DAT:
|
||||
|
||||
```python
|
||||
web_headers = root.create(tableDAT, 'headers')
|
||||
web_headers.appendRow(['Authorization', 'Bearer YOUR_TOKEN'])
|
||||
web_headers.appendRow(['Accept', 'application/json'])
|
||||
web.par.headers = web_headers.path
|
||||
```
|
||||
|
||||
**Parsing JSON response:**
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
def onTableChange(dat):
|
||||
response = dat.text # raw response body
|
||||
data = json.loads(response)
|
||||
# Update a tableDAT or store in a constantCHOP for downstream use
|
||||
op('/project1/api_status').par.value0 = data['count']
|
||||
return
|
||||
```
|
||||
|
||||
Wire this in a `datExecuteDAT` watching the webDAT.
|
||||
|
||||
**Polling pattern:**
|
||||
|
||||
```python
|
||||
# timerCHOP fires every N seconds
|
||||
timer = root.create(timerCHOP, 'poll_timer')
|
||||
timer.par.length = 5.0
|
||||
timer.par.cycle = True
|
||||
|
||||
# chopExecuteDAT on the timer's 'cycles' channel pulses the webDAT
|
||||
def offToOn(channel, sampleIndex, val, prev):
|
||||
op('/project1/api_call').par.fetch.pulse()
|
||||
return
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Web Client DAT — More Robust HTTP
|
||||
|
||||
`webclientDAT` is the modern replacement for `webDAT` — supports streaming responses, chunked transfer, custom auth.
|
||||
|
||||
```python
|
||||
client = root.create(webclientDAT, 'api')
|
||||
client.par.method = 'POST'
|
||||
client.par.url = 'https://api.example.com/events'
|
||||
client.par.uploadtype = 'json'
|
||||
client.par.uploaddata = '{"event": "scene_change", "scene": 3}'
|
||||
client.par.request.pulse()
|
||||
```
|
||||
|
||||
Output goes to its child `webclient1_response` DAT. Use a `datExecuteDAT` to react.
|
||||
|
||||
---
|
||||
|
||||
## Web Server DAT — TD as HTTP Server
|
||||
|
||||
Hosts a tiny HTTP server inside TD. Useful for:
|
||||
- Status/health endpoints
|
||||
- Remote control from a phone or another machine
|
||||
- Webhook receivers from external services
|
||||
|
||||
```python
|
||||
server = root.create(webserverDAT, 'control_server')
|
||||
server.par.port = 8080
|
||||
server.par.active = True
|
||||
|
||||
# Define handler in the docked callback DAT
|
||||
```
|
||||
|
||||
In the auto-created `webserver1_callbacks` DAT:
|
||||
|
||||
```python
|
||||
def onHTTPRequest(webServerDAT, request, response):
|
||||
path = request['uri']
|
||||
if path == '/status':
|
||||
response['statusCode'] = 200
|
||||
response['data'] = '{"fps": 60, "scene": "active"}'
|
||||
elif path == '/scene':
|
||||
idx = int(request['args'].get('index', 0))
|
||||
op('/project1/scene_switch').par.index = idx
|
||||
response['statusCode'] = 200
|
||||
response['data'] = 'OK'
|
||||
else:
|
||||
response['statusCode'] = 404
|
||||
response['data'] = 'Not Found'
|
||||
return response
|
||||
```
|
||||
|
||||
Test from terminal: `curl http://localhost:8080/status`.
|
||||
|
||||
**Security:** No auth by default. Bind to localhost only or add a token check in the callback. Never expose to the public internet without auth.
|
||||
|
||||
---
|
||||
|
||||
## WebSocket DAT — Bidirectional Real-Time
|
||||
|
||||
For low-latency bidirectional streams (chat, live data feeds, controllers).
|
||||
|
||||
### Client
|
||||
|
||||
```python
|
||||
ws = root.create(websocketDAT, 'ws_client')
|
||||
ws.par.netaddress = 'wss://api.example.com/socket'
|
||||
ws.par.active = True
|
||||
```
|
||||
|
||||
In the docked callbacks DAT:
|
||||
|
||||
```python
|
||||
def onConnect(dat):
|
||||
dat.sendText('{"action": "subscribe", "channel": "ticks"}')
|
||||
return
|
||||
|
||||
def onReceiveText(dat, rowIndex, message):
|
||||
# message is a string; parse JSON, dispatch to ops
|
||||
import json
|
||||
data = json.loads(message)
|
||||
op('/project1/price_chop').par.value0 = data['price']
|
||||
return
|
||||
|
||||
def onDisconnect(dat):
|
||||
# Optionally schedule a reconnect
|
||||
return
|
||||
```
|
||||
|
||||
### Server
|
||||
|
||||
```python
|
||||
ws = root.create(websocketDAT, 'ws_server')
|
||||
ws.par.mode = 'server'
|
||||
ws.par.port = 9001
|
||||
ws.par.active = True
|
||||
```
|
||||
|
||||
Same callback structure with an additional `clientID` arg.
|
||||
|
||||
---
|
||||
|
||||
## MQTT — Pub/Sub for IoT
|
||||
|
||||
```python
|
||||
mqtt = root.create(mqttClientDAT, 'iot')
|
||||
mqtt.par.brokeraddress = 'broker.hivemq.com'
|
||||
mqtt.par.brokerport = 1883
|
||||
mqtt.par.clientid = 'td_install_01'
|
||||
mqtt.par.connect.pulse()
|
||||
|
||||
# Subscribe in callbacks DAT:
|
||||
def onConnect(dat):
|
||||
dat.subscribe('home/lights/+', qos=1)
|
||||
return
|
||||
|
||||
def onReceive(dat, topic, payload, qos, retained, dup):
|
||||
# payload is bytes — decode if JSON
|
||||
msg = payload.decode('utf-8')
|
||||
# Dispatch by topic
|
||||
return
|
||||
|
||||
# Publish from anywhere:
|
||||
op('iot').publish('show/scene', 'sunset', qos=0, retain=False)
|
||||
```
|
||||
|
||||
For Mosquitto / HiveMQ self-hosted brokers use the same setup with `tcp://192.168.x.x` and your local port.
|
||||
|
||||
---
|
||||
|
||||
## Serial DAT — Arduino, USB Devices
|
||||
|
||||
```python
|
||||
serial = root.create(serialDAT, 'arduino')
|
||||
serial.par.port = '/dev/cu.usbmodem14101' # macOS — check Arduino IDE
|
||||
# Windows: 'COM3', 'COM4', etc.
|
||||
serial.par.baudrate = 115200
|
||||
serial.par.active = True
|
||||
```
|
||||
|
||||
In callbacks:
|
||||
|
||||
```python
|
||||
def onReceive(dat, rowIndex, line):
|
||||
# Each newline-terminated line from Arduino arrives here
|
||||
parts = line.split(',')
|
||||
op('/project1/sensors').par.value0 = float(parts[0])
|
||||
op('/project1/sensors').par.value1 = float(parts[1])
|
||||
return
|
||||
```
|
||||
|
||||
Send to Arduino:
|
||||
```python
|
||||
op('arduino').send('LED_ON\n')
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## TCP/IP DAT — Custom Protocols
|
||||
|
||||
For talking to non-HTTP servers (game servers, custom protocols, legacy systems).
|
||||
|
||||
```python
|
||||
tcp = root.create(tcpipDAT, 'show_control')
|
||||
tcp.par.netaddress = '192.168.1.50'
|
||||
tcp.par.port = 7000
|
||||
tcp.par.protocol = 'tcp' # 'tcp' | 'udp'
|
||||
tcp.par.active = True
|
||||
```
|
||||
|
||||
Send / receive via callbacks similar to websocketDAT.
|
||||
|
||||
For UDP-only (fire-and-forget, no connection), use `udpoutDAT` + `udpinDAT` — simpler but unreliable across networks.
|
||||
|
||||
---
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### REST API → Visual
|
||||
|
||||
```
|
||||
timerCHOP (5s loop)
|
||||
→ chopExecuteDAT (pulse webDAT.par.fetch on cycle)
|
||||
→ webDAT (returns JSON)
|
||||
→ datExecuteDAT (parse, write to constantCHOP)
|
||||
→ CHOP drives glsl uniform → visuals
|
||||
```
|
||||
|
||||
### Webhook receiver
|
||||
|
||||
```
|
||||
webserverDAT (port 8080, /webhook endpoint)
|
||||
→ callback writes to a tableDAT log + triggers a scene change
|
||||
```
|
||||
|
||||
### Real-time stock/crypto ticker
|
||||
|
||||
```
|
||||
websocketDAT (subscribe to feed)
|
||||
→ onReceiveText callback parses JSON
|
||||
→ writes to constantCHOP
|
||||
→ drives bar chart / typography animation
|
||||
```
|
||||
|
||||
### IoT-controlled installation
|
||||
|
||||
```
|
||||
MQTT → callback dispatches by topic
|
||||
→ /lights/main → constantCHOP drives lighting render
|
||||
→ /audio/volume → mathCHOP for master fader
|
||||
```
|
||||
|
||||
### Two-way phone control
|
||||
|
||||
```
|
||||
WebSocket server in TD
|
||||
→ simple HTML page on phone connects, sends slider values
|
||||
→ callback writes to ops
|
||||
→ TD pushes status back via dat.sendText() to phone UI
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **`webDAT` doesn't auto-fetch** — must explicitly pulse `par.fetch`. Easy to forget.
|
||||
2. **Blocking on slow APIs** — `webDAT` runs on the cook thread. A 30s API call freezes TD for 30s. Use `webclientDAT` (async) for anything potentially slow.
|
||||
3. **WebSocket reconnection** — TD does NOT auto-reconnect on disconnect. Implement backoff in `onDisconnect`.
|
||||
4. **Serial port permissions on macOS** — TD needs Full Disk Access OR the port needs to be unlocked via `sudo chmod 666 /dev/cu.usbmodem...` per session.
|
||||
5. **MQTT broker connection state** — `mqttClientDAT` may show `connected=true` but messages don't flow if QoS is wrong or topic ACL blocks. Check broker logs.
|
||||
6. **JSON parse errors crash callbacks silently** — wrap parses in try/except and log to textport. Otherwise the callback just stops firing.
|
||||
7. **Firewall on Windows** — first time `webserverDAT` binds, Windows pops a firewall dialog. Approve it or the server is unreachable.
|
||||
8. **CORS** — `webserverDAT` doesn't add CORS headers by default. If serving a webapp from a different origin, add `Access-Control-Allow-Origin: *` in the response.
|
||||
9. **Polling vs push** — polling burns API quota. Always prefer WebSocket / webhook / MQTT for high-frequency data.
|
||||
10. **Floating-point parsing** — sensor data over Serial often comes as strings. `float()` will crash on `'\n'` or `'NaN'`. Validate before converting.
|
||||
|
||||
---
|
||||
|
||||
## Quick Recipes
|
||||
|
||||
| Goal | Op chain |
|
||||
|---|---|
|
||||
| Periodic API fetch | `timerCHOP` → `chopExecuteDAT` pulses → `webDAT` → `datExecuteDAT` parses |
|
||||
| Webhook receiver | `webserverDAT` (port + path), callback writes to ops |
|
||||
| Real-time stream | `websocketDAT` client → onReceiveText → CHOP/DAT |
|
||||
| Arduino sensor → visual | `serialDAT` → callback → `constantCHOP` → expression on visual op |
|
||||
| TD ↔ phone control | `websocketDAT` server + simple HTML page on phone |
|
||||
| MQTT IoT integration | `mqttClientDAT` subscribe → callback dispatches by topic |
|
||||
@@ -0,0 +1,121 @@
|
||||
# Geometry COMP Reference
|
||||
|
||||
## Creating Geometry COMPs
|
||||
|
||||
```python
|
||||
geo = root.create(geometryCOMP, 'geo1')
|
||||
# Remove default torus
|
||||
for c in list(geo.children):
|
||||
if c.valid: c.destroy()
|
||||
# Build your shape inside
|
||||
```
|
||||
|
||||
## Correct Pattern (shapes inside geo)
|
||||
|
||||
```python
|
||||
# Create shape INSIDE the geo COMP
|
||||
box = geo.create(boxSOP, 'cube')
|
||||
box.par.sizex = 1.5; box.par.sizey = 1.5; box.par.sizez = 1.5
|
||||
|
||||
# For POP-based geometry (TD 099), POPs must be inside:
|
||||
sph = geo.create(spherePOP, 'shape')
|
||||
out1 = geo.create(outPOP, 'out1')
|
||||
out1.inputConnectors[0].connect(sph.outputConnectors[0])
|
||||
```
|
||||
|
||||
## DO NOT: Common Mistakes
|
||||
|
||||
```python
|
||||
# BAD: Don't create geometry at parent level and wire into COMP
|
||||
box = root.create(boxPOP, 'box1') # ← outside geo, won't render
|
||||
|
||||
# BAD: Don't reference parent operators from inside COMP
|
||||
choptopop1.par.chop = '../null1' # ← hidden dependency, breaks on move
|
||||
```
|
||||
|
||||
## Instancing
|
||||
|
||||
```python
|
||||
geo.par.instancing = True
|
||||
geo.par.instanceop = 'sopto1' # relative path to CHOP/SOP with instance data
|
||||
geo.par.instancetx = 'tx'
|
||||
geo.par.instancety = 'ty'
|
||||
geo.par.instancetz = 'tz'
|
||||
```
|
||||
|
||||
### Instance Attribute Names by OP Type
|
||||
|
||||
| OP Type | Attribute Names |
|
||||
|---------|-----------------|
|
||||
| CHOP | Channel names: `tx`, `ty`, `tz` |
|
||||
| SOP/POP | `P(0)`, `P(1)`, `P(2)` for position |
|
||||
| DAT | Column header names from first row |
|
||||
| TOP | `r`, `g`, `b`, `a` |
|
||||
|
||||
### Mixed Data Sources
|
||||
|
||||
```python
|
||||
geo.par.instanceop = 'pos_chop' # Position from CHOP
|
||||
geo.par.instancetx = 'tx'
|
||||
geo.par.instancecolorop = 'color_top' # Color from TOP
|
||||
geo.par.instancecolorr = 'r'
|
||||
```
|
||||
|
||||
## Rendering Setup
|
||||
|
||||
```python
|
||||
# Camera
|
||||
cam = root.create(cameraCOMP, 'cam1')
|
||||
cam.par.tx = 0; cam.par.ty = 0; cam.par.tz = 4
|
||||
|
||||
# Render TOP
|
||||
render = root.create(renderTOP, 'render1')
|
||||
render.par.outputresolution = 'custom'
|
||||
render.par.resolutionw = 1280; render.par.resolutionh = 720
|
||||
render.par.camera = cam.path
|
||||
render.par.geometry = geo.path # accepts path string
|
||||
```
|
||||
|
||||
## POPs vs SOPs for Rendering
|
||||
|
||||
In TD 099, `geometryCOMP` renders **POPs** but NOT SOPs. A `boxSOP` inside a geometry COMP is invisible — no errors.
|
||||
|
||||
```python
|
||||
# WRONG — SOPs don't render (invisible, no errors)
|
||||
box = geo.create(boxSOP, 'cube') # ✗ invisible
|
||||
|
||||
# CORRECT — POPs render
|
||||
box = geo.create(boxPOP, 'cube') # ✓ visible
|
||||
```
|
||||
|
||||
| SOP | POP | Notes |
|
||||
|-----|-----|-------|
|
||||
| `boxSOP` | `boxPOP` | `sizex/y/z`, `surftype` |
|
||||
| `sphereSOP` | `spherePOP` | `radx/y/z`, `freq`, `type` (geodesic/grid/sharedpoles/tetrahedron) |
|
||||
| `torusSOP` | `torusPOP` | TD auto-creates in new geo COMPs |
|
||||
| `circleSOP` | `circlePOP` | |
|
||||
| `gridSOP` | `gridPOP` | |
|
||||
| `tubeSOP` | `tubePOP` | |
|
||||
|
||||
New geometry COMPs auto-create: `in1` (inPOP), `out1` (outPOP), `torus1` (torusPOP). Always clean before building.
|
||||
|
||||
## Morphing Between Shapes (switchPOP)
|
||||
|
||||
```python
|
||||
sw = geo.create(switchPOP, 'shape_switch')
|
||||
sw.par.index.expr = 'int(absTime.seconds / 3) % 4'
|
||||
sw.inputConnectors[0].connect(tetra.outputConnectors[0]) # shape 0
|
||||
sw.inputConnectors[1].connect(box.outputConnectors[0]) # shape 1
|
||||
sw.inputConnectors[2].connect(octa.outputConnectors[0]) # shape 2
|
||||
sw.inputConnectors[3].connect(sphere.outputConnectors[0]) # shape 3
|
||||
|
||||
out = geo.create(outPOP, 'out1')
|
||||
out.inputConnectors[0].connect(sw.outputConnectors[0])
|
||||
```
|
||||
|
||||
`spherePOP.par.type` options: `geodesic`, `grid`, `sharedpoles`, `tetrahedron`. Use `tetrahedron` for platonic solid polyhedra.
|
||||
|
||||
## Misc
|
||||
|
||||
- `connect()` replaces existing connections — no need to disconnect first
|
||||
- `project.name` returns the TOE filename, `project.folder` returns the directory
|
||||
@@ -0,0 +1,151 @@
|
||||
# GLSL Reference
|
||||
|
||||
## Uniforms
|
||||
|
||||
```
|
||||
TouchDesigner GLSL
|
||||
─────────────────────────────
|
||||
vec0name = 'uTime' → uniform float uTime;
|
||||
vec0valuex = 1.0 → uTime value
|
||||
```
|
||||
|
||||
### Pass Time
|
||||
|
||||
```python
|
||||
glsl_op.par.vec0name = 'uTime'
|
||||
glsl_op.par.vec0valuex.mode = ParMode.EXPRESSION
|
||||
glsl_op.par.vec0valuex.expr = 'absTime.seconds'
|
||||
```
|
||||
|
||||
```glsl
|
||||
uniform float uTime;
|
||||
void main() { float t = uTime * 0.5; }
|
||||
```
|
||||
|
||||
### Built-in Uniforms (TOP)
|
||||
|
||||
```glsl
|
||||
// Output resolution (always available)
|
||||
vec2 res = uTDOutputInfo.res.zw;
|
||||
|
||||
// Input texture (only when inputs connected)
|
||||
vec2 inputRes = uTD2DInfos[0].res.zw;
|
||||
vec4 color = texture(sTD2DInputs[0], vUV.st);
|
||||
|
||||
// UV coordinates
|
||||
vUV.st // 0-1 texture coords
|
||||
```
|
||||
|
||||
**IMPORTANT:** `uTD2DInfos` requires input textures. For standalone shaders use `uTDOutputInfo`.
|
||||
|
||||
## Built-in Utility Functions
|
||||
|
||||
```glsl
|
||||
// Noise
|
||||
float TDPerlinNoise(vec2/vec3/vec4 v);
|
||||
float TDSimplexNoise(vec2/vec3/vec4 v);
|
||||
|
||||
// Color conversion
|
||||
vec3 TDHSVToRGB(vec3 c);
|
||||
vec3 TDRGBToHSV(vec3 c);
|
||||
|
||||
// Matrix transforms
|
||||
mat4 TDTranslate(float x, float y, float z);
|
||||
mat3 TDRotateX/Y/Z(float radians);
|
||||
mat3 TDRotateOnAxis(float radians, vec3 axis);
|
||||
mat3 TDScale(float x, float y, float z);
|
||||
mat3 TDRotateToVector(vec3 forward, vec3 up);
|
||||
mat3 TDCreateRotMatrix(vec3 from, vec3 to); // vectors must be normalized
|
||||
|
||||
// Resolution struct
|
||||
struct TDTexInfo {
|
||||
vec4 res; // (1/width, 1/height, width, height)
|
||||
vec4 depth;
|
||||
};
|
||||
|
||||
// Output (always use this — handles sRGB correctly)
|
||||
fragColor = TDOutputSwizzle(color);
|
||||
|
||||
// Instancing (MAT only)
|
||||
int TDInstanceID();
|
||||
```
|
||||
|
||||
## glslTOP
|
||||
|
||||
Docked DATs created automatically:
|
||||
- `glsl1_pixel` — Pixel shader
|
||||
- `glsl1_compute` — Compute shader
|
||||
- `glsl1_info` — Compile info
|
||||
|
||||
### Pixel Shader Template
|
||||
|
||||
```glsl
|
||||
out vec4 fragColor;
|
||||
void main() {
|
||||
vec4 color = texture(sTD2DInputs[0], vUV.st);
|
||||
fragColor = TDOutputSwizzle(color);
|
||||
}
|
||||
```
|
||||
|
||||
### Compute Shader Template
|
||||
|
||||
```glsl
|
||||
layout (local_size_x = 8, local_size_y = 8) in;
|
||||
void main() {
|
||||
vec4 color = texelFetch(sTD2DInputs[0], ivec2(gl_GlobalInvocationID.xy), 0);
|
||||
TDImageStoreOutput(0, gl_GlobalInvocationID, color);
|
||||
}
|
||||
```
|
||||
|
||||
### Update Shader
|
||||
|
||||
```python
|
||||
op('/project1/glsl1_pixel').text = shader_code
|
||||
op('/project1/glsl1').cook(force=True)
|
||||
# Check errors:
|
||||
print(op('/project1/glsl1_info').text)
|
||||
```
|
||||
|
||||
## glslMAT
|
||||
|
||||
Docked DATs:
|
||||
- `glslmat1_vertex` — Vertex shader (param: `vdat`)
|
||||
- `glslmat1_pixel` — Pixel shader (param: `pdat`)
|
||||
- `glslmat1_info` — Compile info
|
||||
|
||||
Note: MAT uses `vdat`/`pdat`, TOP uses `vertexdat`/`pixeldat`.
|
||||
|
||||
### Vertex Shader Template
|
||||
|
||||
```glsl
|
||||
uniform float uTime;
|
||||
void main() {
|
||||
vec3 pos = TDPos();
|
||||
pos.z += sin(pos.x * 3.0 + uTime) * 0.2;
|
||||
vec4 worldSpacePos = TDDeform(pos);
|
||||
gl_Position = TDWorldToProj(worldSpacePos);
|
||||
}
|
||||
```
|
||||
|
||||
## Bayer 8x8 Dither Matrix
|
||||
|
||||
Reusable ordered dither function for retro/print aesthetics:
|
||||
|
||||
```glsl
|
||||
float bayer8(vec2 pos) {
|
||||
int x = int(mod(pos.x, 8.0)), y = int(mod(pos.y, 8.0)), idx = x + y * 8;
|
||||
int b[64] = int[64](
|
||||
0,32,8,40,2,34,10,42,48,16,56,24,50,18,58,26,
|
||||
12,44,4,36,14,46,6,38,60,28,52,20,62,30,54,22,
|
||||
3,35,11,43,1,33,9,41,51,19,59,27,49,17,57,25,
|
||||
15,47,7,39,13,45,5,37,63,31,55,23,61,29,53,21
|
||||
);
|
||||
return float(b[idx]) / 64.0;
|
||||
}
|
||||
```
|
||||
|
||||
## glslPOP / glsladvancedPOP / glslcopyPOP
|
||||
|
||||
All use compute shaders. Docked DATs follow naming convention:
|
||||
- `glsl1_compute` / `glsladv1_compute`
|
||||
- `glslcopy1_ptCompute` / `glslcopy1_vertCompute` / `glslcopy1_primCompute`
|
||||
@@ -0,0 +1,131 @@
|
||||
# Layout Compositor Reference
|
||||
|
||||
Patterns for building modular multi-panel grids — useful for HUD interfaces, data dashboards, and multi-source visual composites.
|
||||
|
||||
## Layout Approaches
|
||||
|
||||
| Approach | Best For | Notes |
|
||||
|----------|----------|-------|
|
||||
| `layoutTOP` | Fixed grid, quick setup | GPU, simple tiling |
|
||||
| Container COMP + `overTOP` | Full control, mixed-size panels | More setup, very flexible |
|
||||
| GLSL compositor | Procedural / BSP-style | Most powerful, more complex |
|
||||
|
||||
---
|
||||
|
||||
## layoutTOP
|
||||
|
||||
Built-in grid compositor — fastest path for uniform tile grids.
|
||||
|
||||
```python
|
||||
layout = root.create(layoutTOP, 'layout1')
|
||||
layout.par.resolutionw = 1920
|
||||
layout.par.resolutionh = 1080
|
||||
layout.par.cols = 3
|
||||
layout.par.rows = 2
|
||||
layout.par.gap = 4
|
||||
```
|
||||
|
||||
Connect inputs (up to cols×rows):
|
||||
```python
|
||||
layout.inputConnectors[0].connect(op('panel_radar'))
|
||||
layout.inputConnectors[1].connect(op('panel_wave'))
|
||||
layout.inputConnectors[2].connect(op('panel_data'))
|
||||
```
|
||||
|
||||
**Variable-width columns:** Not directly supported. Use overTOP approach for non-uniform grids.
|
||||
|
||||
---
|
||||
|
||||
## Container COMP Grid
|
||||
|
||||
Build each element as its own `containerCOMP`. Compose with `overTOP`:
|
||||
|
||||
```python
|
||||
def create_panel(root, name, width, height, x=0, y=0):
|
||||
panel = root.create(containerCOMP, name)
|
||||
panel.par.w = width
|
||||
panel.par.h = height
|
||||
panel.viewer = True
|
||||
return panel
|
||||
|
||||
# Composite with overTOP chain
|
||||
over1 = root.create(overTOP, 'over1')
|
||||
over1.inputConnectors[0].connect(panel_radar)
|
||||
over1.inputConnectors[1].connect(panel_wave)
|
||||
over1.par.topx2 = 0
|
||||
over1.par.topy2 = 512
|
||||
```
|
||||
|
||||
**Tip:** Use a `resolutionTOP` before each `overTOP` input if panels are different sizes.
|
||||
|
||||
---
|
||||
|
||||
## Panel Dividers (GLSL)
|
||||
|
||||
```glsl
|
||||
out vec4 fragColor;
|
||||
uniform vec2 uGridDivisions; // e.g. vec2(3, 2) for 3 cols, 2 rows
|
||||
uniform float uLineWidth; // pixels
|
||||
uniform vec4 uLineColor; // e.g. vec4(0.0, 1.0, 0.8, 0.6) for cyan
|
||||
|
||||
void main() {
|
||||
vec2 res = uTDOutputInfo.res.zw;
|
||||
vec2 uv = vUV.st;
|
||||
vec4 bg = texture(sTD2DInputs[0], uv);
|
||||
|
||||
float lineW = uLineWidth / res.x;
|
||||
float lineH = uLineWidth / res.y;
|
||||
|
||||
float vDiv = 0.0;
|
||||
for (float i = 1.0; i < uGridDivisions.x; i++) {
|
||||
float x = i / uGridDivisions.x;
|
||||
vDiv = max(vDiv, step(abs(uv.x - x), lineW));
|
||||
}
|
||||
|
||||
float hDiv = 0.0;
|
||||
for (float i = 1.0; i < uGridDivisions.y; i++) {
|
||||
float y = i / uGridDivisions.y;
|
||||
hDiv = max(hDiv, step(abs(uv.y - y), lineH));
|
||||
}
|
||||
|
||||
float line = max(vDiv, hDiv);
|
||||
vec4 result = mix(bg, uLineColor, line * uLineColor.a);
|
||||
fragColor = TDOutputSwizzle(result);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Element Library Pattern
|
||||
|
||||
Each visual element lives in its own `baseCOMP` as a reusable `.tox`:
|
||||
|
||||
### Standard Interface
|
||||
```
|
||||
inputs:
|
||||
- in_audio (CHOP) — audio envelope / beat data
|
||||
- in_data (CHOP) — optional data stream
|
||||
- in_control (CHOP) — intensity, color, speed params
|
||||
|
||||
outputs:
|
||||
- out_top (TOP) — rendered element
|
||||
```
|
||||
|
||||
### Network Structure
|
||||
```
|
||||
/project1/
|
||||
audio_bus/ ← all audio analysis (see audio-reactive.md)
|
||||
elements/
|
||||
elem_radar/ ← baseCOMP with out_top
|
||||
elem_wave/
|
||||
elem_data/
|
||||
compositor/
|
||||
layout1 ← layoutTOP or overTOP chain
|
||||
dividers1 ← GLSL divider lines
|
||||
postfx/ ← bloom → chrom → CRT stack (see postfx.md)
|
||||
null_out ← final output
|
||||
output/
|
||||
windowCOMP ← full-screen output
|
||||
```
|
||||
|
||||
**Key principle:** Elements don't know about each other. The compositor assembles them. Audio bus is referenced by all elements but lives separately.
|
||||
@@ -0,0 +1,382 @@
|
||||
# twozero MCP Tools Reference
|
||||
|
||||
36 tools from twozero MCP v2.774+ (April 2026).
|
||||
All tools accept an optional `target_instance` param for multi-TD-instance scenarios.
|
||||
|
||||
## Execution & Scripting
|
||||
|
||||
### td_execute_python
|
||||
|
||||
Execute Python code inside TouchDesigner and return the result. Has full access to TD Python API (op, project, app, etc). Print statements and the last expression value are captured. Best for: wiring connections (inputConnectors), setting expressions (par.X.expr/mode), querying parameter names, and batch creation scripts (5+ operators). For creating 1-4 operators, prefer td_create_operator instead.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `code` | string | yes | Python code to execute in TouchDesigner |
|
||||
|
||||
## Network & Structure
|
||||
|
||||
### td_get_network
|
||||
|
||||
Get the operator network structure in TouchDesigner (TD) at a given path. Returns compact list: name OPType flags. First line is full path of queried op. Flags: ch:N=children count, !cook=allowCooking off, bypass, private=isPrivate, blocked:reason, "comment text". depth=0 (default) = current level only. depth=1 = one level of children (indented). To explore deeper, call again on a specific COMP path. System operators (/ui, /sys) are hidden by default.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | no | Network path to inspect, e.g. '/' or '/project1' |
|
||||
| `depth` | integer | no | How many levels deep to recurse. 0=current level only (recommended), 1=include direct children of COMPs |
|
||||
| `includeSystem` | boolean | no | Include system operators (/ui, /sys). Default false. |
|
||||
| `nodeXY` | boolean | no | Include nodeX,nodeY coordinates. Default false. |
|
||||
|
||||
### td_create_operator
|
||||
|
||||
Create a new operator (node) in TouchDesigner (TD). Preferred way to create operators — handles viewport positioning, viewer flag, and docked ops automatically. For batch creation (5+ ops), you may use td_execute_python with a script instead, but then call td_get_hints('construction') first for correct parameter names and layout rules. Supports all TD operator types: TOP, CHOP, SOP, DAT, COMP, MAT. If parent is omitted, creates in the currently open network at the user's viewport position. When building a container: first create baseCOMP (no parent), then create children with parent=compPath.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `type` | string | yes | Operator type, e.g. 'textDAT', 'constantCHOP', 'noiseTOP', 'transformTOP', 'baseCOMP' |
|
||||
| `parent` | string | no | Path to the parent operator. If omitted, uses the currently open network in TD. |
|
||||
| `name` | string | no | Name for the new operator (optional, TD auto-names if omitted) |
|
||||
| `parameters` | object | no | Key-value pairs of parameters to set on the created operator |
|
||||
|
||||
### td_find_op
|
||||
|
||||
Find operators by name and/or type across the project. Returns TSV: path, OPType, flags. Flags: bypass, !cook, private, blocked:reason. Use td_search to search inside code/expressions; use td_find_op to find operators themselves.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `name` | string | no | Substring to match in operator name (case-insensitive). E.g. 'noise' finds noise1, noise2, myNoise. |
|
||||
| `type` | string | no | Substring to match in OPType (case-insensitive). E.g. 'noiseTOP', 'baseCOMP', 'CHOP'. Use exact type for precision or partial for broader matches. |
|
||||
| `root` | string | no | Root operator path to search from. Default '/project1'. |
|
||||
| `max_results` | number | no | Maximum results to return. Default 50. |
|
||||
| `max_depth` | number | no | Max recursion depth from root. Default unlimited. |
|
||||
| `detail` | `basic` / `summary` | no | Result detail level. 'basic' = name/path/type (fast). 'summary' = + connections, non-default pars, expressions. Default 'basic'. |
|
||||
|
||||
### td_search
|
||||
|
||||
Search for text across all code (DAT scripts), parameter expressions, and string parameter values in the TD project. Returns TSV: path, kind (code/expression/parameter/ref), line, text. JSON when context>0. Words are OR-matched. Use quotes for exact phrases: 'GetLogin "op('login')"'. Use count_only=true to quickly check if something is referenced without fetching full results.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `query` | string | yes | Search query. Multiple words = OR (any match). Wrap in quotes for exact phrase. Example: 'GetLogin getLogin' finds either. |
|
||||
| `root` | string | no | Root operator path to search from. Default '/project1'. |
|
||||
| `scope` | `all` / `code` / `editable` / `expressions` / `parameters` | no | What to search. 'code' = DAT scripts only (fast, ~0.05s). 'editable' = only editable code (skips inherited/ref DATs). 'expressions' = parameter expressions only. 'parameters' = string parameter values only. 'all' = everything (slow, ~1.5s due to parameter scan). Default 'all'. |
|
||||
| `case_sensitive` | boolean | no | Case-sensitive matching. Default false. |
|
||||
| `max_results` | number | no | Maximum results to return. Default 50. |
|
||||
| `context` | number | no | Lines to show before/after each code match. Saves td_read_dat calls. Default 0. |
|
||||
| `count_only` | boolean | no | Return only match count, not results. Fast existence check. |
|
||||
| `max_depth` | number | no | Max recursion depth from root. Default unlimited. |
|
||||
|
||||
### td_navigate_to
|
||||
|
||||
Navigate the TouchDesigner Network Editor viewport to show a specific operator. Opens the operator's parent network and centers the view on it. Use this to show the user where a problem is, or to navigate to an operator before modifying it.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the operator to navigate to, e.g. '/project1/noise1' |
|
||||
|
||||
## Operator Inspection
|
||||
|
||||
### td_get_operator_info
|
||||
|
||||
Get information about a specific operator (node) in TouchDesigner (TD). detail='summary': connections, non-default pars, expressions, CHOP channels (compact). detail='full': all of the above PLUS every parameter with value/default/label.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Full path to the operator, e.g. '/project1/noise1' |
|
||||
| `detail` | `summary` / `full` | no | Level of detail. 'summary' = connections, expressions, non-default pars, custom pars (pulse marked), CHOP channels. 'full' = summary + all parameters. Default 'full'. |
|
||||
|
||||
### td_get_operators_info
|
||||
|
||||
Get information about multiple operators in one call. Returns an array of operator info objects. Use instead of calling td_get_operator_info multiple times.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `paths` | array | yes | Array of full operator paths, e.g. ['/project1/null1', '/project1/null2'] |
|
||||
| `detail` | `summary` / `full` | no | Level of detail. Default 'summary'. |
|
||||
|
||||
### td_get_par_info
|
||||
|
||||
Get parameter names and details for a TouchDesigner operator type. Without specific pars: returns compact list of all parameters with their names, types, and menu options. With pars: returns full details (help text, menu values, style) for specific parameters. Use this when you need to know exact parameter names before setting them.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `op_type` | string | yes | TD operator type name, e.g. 'noiseTOP', 'blurTOP', 'lfoCHOP', 'compositeTOP' |
|
||||
| `pars` | array | no | Optional list of specific parameter names to get full details for |
|
||||
|
||||
## Parameter Setting
|
||||
|
||||
### td_set_operator_pars
|
||||
|
||||
Set parameters and flags on an operator in TouchDesigner (TD). Safer than td_execute_python for simple parameter changes. Can set values, toggle bypass/viewer, without writing Python code.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the operator |
|
||||
| `parameters` | object | no | Key-value pairs of parameters to set |
|
||||
| `bypass` | boolean | no | Set bypass state of the operator (not available on COMPs) |
|
||||
| `viewer` | boolean | no | Set viewer state of the operator |
|
||||
| `allowCooking` | boolean | no | Set cooking flag on a COMP. When False, internal network stops cooking (0 CPU). COMP-only. |
|
||||
|
||||
## Data Read/Write
|
||||
|
||||
### td_read_dat
|
||||
|
||||
Read the text content of a DAT operator in TouchDesigner (TD). Returns content with line numbers. Use to read scripts, extensions, GLSL shaders, table data.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the DAT operator |
|
||||
| `start_line` | integer | no | Start line (1-based). Omit to read from beginning. |
|
||||
| `end_line` | integer | no | End line (inclusive). Omit to read to end. |
|
||||
|
||||
### td_write_dat
|
||||
|
||||
Write or patch text content of a DAT operator in TouchDesigner (TD). Can do full replacement or StrReplace-style patching (old_text -> new_text). Use for editing scripts, extensions, shaders. Does NOT reinit extensions automatically.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the DAT operator |
|
||||
| `text` | string | no | Full replacement text. Use this OR old_text+new_text, not both. |
|
||||
| `old_text` | string | no | Text to find and replace (must be unique in the DAT) |
|
||||
| `new_text` | string | no | Replacement text |
|
||||
| `replace_all` | boolean | no | If true, replaces ALL occurrences of old_text (default: false, requires unique match) |
|
||||
|
||||
### td_read_chop
|
||||
|
||||
Read CHOP channel sample data. Returns channel values as arrays. Use when you need the actual sample values (animation curves, lookup tables, waveforms), not just the summary from td_get_operator_info.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the CHOP operator |
|
||||
| `channels` | array | no | Channel names to read. Omit to read all channels. |
|
||||
| `start` | integer | no | Start sample index (0-based). Omit to read from beginning. |
|
||||
| `end` | integer | no | End sample index (inclusive). Omit to read to end. |
|
||||
|
||||
### td_read_textport
|
||||
|
||||
Read the last N lines from the TouchDesigner (TD) log/textport (console output). Use this to see errors, warnings and print output from TD.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `lines` | integer | no | Number of recent lines to return |
|
||||
|
||||
### td_clear_textport
|
||||
|
||||
Clear the MCP textport log buffer. Use this before starting a debug session or an edit-run-check loop to keep td_read_textport output focused and minimal.
|
||||
|
||||
No parameters (other than optional `target_instance`).
|
||||
|
||||
## Visual Capture
|
||||
|
||||
### td_get_screenshot
|
||||
|
||||
Get a screenshot of an operator's viewer in TouchDesigner (TD). Saves the image to a file and returns the file path. Use your file-reading tool to view the image. Shows what the operator looks like in its viewer (TOP output, CHOP waveform graph, SOP geometry, DAT table, parameter UI, etc). Use this to visually inspect any operator, or to generate images via TD for use in your project. TWO-STEP ASYNC USAGE: Step 1 — call with 'path' to start: returns {'status': 'pending', 'requestId': '...'}. Step 2 — call with 'request_id' to retrieve: returns {'file': '/tmp/.../opname_id.jpg'}. Then read the file to see the image. If step 2 still returns pending, make one other tool call then retry.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | no | Full operator path to screenshot, e.g. '/project1/noise1'. Required for step 1. |
|
||||
| `request_id` | string | no | Request ID from step 1 to retrieve the completed screenshot. |
|
||||
| `max_size` | integer | no | Max pixel size for the longer side (default 512). Use 0 for original operator resolution (useful for pixel-accurate UI work). Higher values (e.g. 1024) for more detail. |
|
||||
| `output_path` | string | no | Optional absolute path where the image should be saved (e.g. '/Users/me/project/render.png'). If omitted, saved to /tmp/pisang_mcp/screenshots/. Use absolute paths — TD's working directory may differ from the agent's. |
|
||||
| `as_top` | boolean | no | If true, captures the operator directly as a TOP (bypasses the viewer renderer), preserving alpha/transparency. Only works for TOP operators — if the target is not a TOP, falls back to the viewer automatically. Use this when you need a clean PNG with alpha, e.g. to save a generated image for use in another project. |
|
||||
| `format` | `auto` / `jpg` / `png` | no | Image format. 'auto' (default): JPEG for viewer mode, PNG for as_top=true. 'jpg': always JPEG (smaller). 'png': always PNG (lossless). |
|
||||
|
||||
### td_get_screenshots
|
||||
|
||||
Get screenshots of multiple operators in one batch. Saves images to files and returns file paths. Use your file-reading tool to view images. TWO-STEP ASYNC USAGE: Step 1 — call with 'paths' array to start: returns {'status': 'pending', 'batchId': '...', 'total': N}. Step 2 — call with 'batch_id' to retrieve: returns {'files': [{op, file}, ...]}. Then read the files to see the images. If still processing returns {'status': 'pending', 'ready': K, 'total': N}.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `paths` | array | no | List of full operator paths to screenshot. Required for step 1. |
|
||||
| `batch_id` | string | no | Batch ID from step 1 to retrieve completed screenshots. |
|
||||
| `max_size` | integer | no | Max pixel size for longer side (default 512). Use 0 for original resolution. |
|
||||
| `as_top` | boolean | no | If true, captures TOP operators directly (preserves alpha). Non-TOP operators fall back to viewer. |
|
||||
| `output_dir` | string | no | Optional absolute path to a directory. Each screenshot saved as <opname>.jpg or .png inside it and kept on disk. |
|
||||
| `format` | `auto` / `jpg` / `png` | no | Image format. 'auto' (default): JPEG for viewer mode, PNG for as_top=true. 'jpg': always JPEG (smaller). 'png': always PNG (lossless). |
|
||||
|
||||
### td_get_screen_screenshot
|
||||
|
||||
Capture a screenshot of the actual screen via TD's screenGrabTOP. Saves the image to a file and returns the file path. Use your file-reading tool to view the image. Unlike td_get_screenshot (operator viewer), this shows what the user literally sees on their monitor — TD windows, UI panels, everything. Use when simulating mouse/keyboard input to verify what happened on screen. Workflow: td_get_screen_screenshot → read file → td_input_execute → wait idle → td_get_screen_screenshot again. TWO-STEP ASYNC: Step 1 — call without request_id: returns {'status':'pending','requestId':'...'}. Step 2 — call with request_id: returns {'file': '/tmp/.../screen_id.jpg', 'info': '...metadata...'}. Then read the file to see the image. The requestId also stays usable with td_screen_point_to_global for later coordinate lookup. crop_x/y/w/h are in ACTUAL SCREEN PIXELS (not image pixels). Crops exceeding screen bounds are auto-clamped. SMART DEFAULTS: max_size is auto when omitted — 1920 for full screen (good overview), max(crop_w,crop_h) for cropped (guarantees 1:1 scale). At 1:1 scale: screen_coord = crop_origin + image_pixel. Otherwise use the formula from metadata.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `request_id` | string | no | Request ID from step 1 to retrieve the completed screenshot. |
|
||||
| `max_size` | integer | no | Max pixel size for the longer side. Auto when omitted: 1920 for full screen, max(crop_w,crop_h) for cropped (1:1). Set explicitly to override. |
|
||||
| `crop_x` | integer | no | Left edge in screen pixels. |
|
||||
| `crop_y` | integer | no | Top edge in screen pixels (y=0 at top of screen). |
|
||||
| `crop_w` | integer | no | Width in pixels. |
|
||||
| `crop_h` | integer | no | Height in pixels. |
|
||||
| `display` | integer | no | Screen index (default 0 = primary display). |
|
||||
|
||||
## Context & Focus
|
||||
|
||||
### td_get_focus
|
||||
|
||||
Get the current user focus in TouchDesigner (TD): which network is open, selected operators, current operator, and rollover (what is under the mouse cursor). IMPORTANT: when the user says 'this operator' or 'вот этот', they mean the SELECTED/CURRENT operator, NOT the rollover. Rollover is just incidental mouse position and should be ignored for intent. Pass screenshots=true to immediately start a screenshot batch for all selected operators — response includes a 'screenshots' field with batchId; retrieve with td_get_screenshots(batch_id=...).
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `screenshots` | boolean | no | If true, start a screenshot batch for all selected operators. Retrieve with td_get_screenshots(batch_id=...). |
|
||||
| `max_size` | integer | no | Max screenshot size when screenshots=true (default 512). |
|
||||
| `as_top` | boolean | no | Passed to the screenshot batch when screenshots=true. |
|
||||
|
||||
### td_get_errors
|
||||
|
||||
Find errors and warnings in TouchDesigner (TD) operators. Checks operator errors, warnings, AND broken parameter expressions (missing channels, bad references, etc). Also includes recent script errors from the log (tracebacks), grouped and deduplicated — e.g. 1000 identical mouse-move errors shown as ×1000 with one entry. If path is given, checks that operator and its children. If no path, checks the currently open network. Use '/' for entire project. Use when user says something is broken, has errors, red nodes, горит ошибка, etc. TIP: call td_clear_textport before reproducing an error to keep log focused. TIP: combine with td_get_perf when user says 'тупит/лагает' to check both errors and performance.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | no | Path to check. If omitted, checks the current network. Use '/' to scan entire project. |
|
||||
| `recursive` | boolean | no | Check children recursively (default true) |
|
||||
| `include_log` | boolean | no | Include recent script errors from log, grouped by unique signature (default true). Use td_clear_textport before reproducing an error to keep results focused. |
|
||||
|
||||
### td_get_perf
|
||||
|
||||
Get performance data from TouchDesigner (TD). Returns TSV: header with fps/budget/memory summary, then slowest operators sorted by cook time. Columns: path, OPType, cpu/cook(ms), gpu/cook(ms), cpu/s, gpu/s, rate, flags. Use when user reports lag, low FPS, slow performance, тупит, тормозит.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | no | Path to profile. If omitted, profiles the current network. Use '/' for entire project. |
|
||||
| `top` | integer | no | Number of slowest operators to return |
|
||||
|
||||
## Documentation
|
||||
|
||||
### td_get_docs
|
||||
|
||||
Get comprehensive documentation on a TouchDesigner topic. Unlike td_get_hints (compact tips), this returns in-depth reference material. Call without arguments to see available topics with descriptions. Call with a topic name to get the full documentation.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `topic` | string | no | Topic to get docs for. Omit to list available topics. |
|
||||
|
||||
### td_get_hints
|
||||
|
||||
Get TouchDesigner tips and common patterns for a topic. Call this BEFORE creating operators or writing TD Python code to learn correct parameter names, expressions, and idiomatic approaches. Available topics: animation, noise, connections, parameters, scripting, construction, ui_analysis, panel_layout, screenshots, input_simulation, undo. IMPORTANT: always call with topic='construction' before building multi-operator setups to get correct TOP/CHOP parameter names, compositeTOP input ordering, and layout guidelines. IMPORTANT: always call with topic='input_simulation' before using td_input_execute to learn focus recovery, coordinate systems, and testing workflow.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `topic` | string | yes | Topic to get hints for. Available: 'animation', 'noise', 'connections', 'parameters', 'scripting', 'construction', 'ui_analysis', 'panel_layout', 'screenshots', 'input_simulation', 'undo', 'networking', 'all' |
|
||||
|
||||
### td_agents_md
|
||||
|
||||
Read, write, or update the agents_md documentation inside a COMP container. agents_md is a Markdown textDAT describing the container's purpose, structure, and conventions. action='read': returns content + staleness check (compares documented children vs live state). action='update': refreshes auto-generated sections (children list, connections) from live state, preserves human-written sections. action='write': sets full content, creates the DAT if missing.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the COMP container |
|
||||
| `action` | `read` / `update` / `write` | yes | read=get content+staleness, update=refresh auto sections, write=set content |
|
||||
| `content` | string | no | Markdown content (only for action='write') |
|
||||
|
||||
## Input Automation
|
||||
|
||||
### td_input_execute
|
||||
|
||||
Send a sequence of mouse/keyboard commands to TouchDesigner. Commands execute sequentially with smooth bezier movement. Returns immediately — poll td_input_status() until status='idle' before proceeding. Command types: 'focus' — bring TD to foreground. 'move' — smooth mouse move: {type,x,y,duration,easing}. 'click' — click: {type,x,y,button,hold,duration,easing}. hold=seconds to hold down. duration=smooth move before click. 'dblclick' — double click: {type,x,y,duration}. 'mousedown'/'mouseup' — {type,x,y,button}. 'key' — keystroke: {type,keys} e.g. 'ctrl+z','tab','escape','shift+f5'. Requires Accessibility permission on Mac. 'type' — human-like typing: {type,text,wpm,variance} — layout-independent Unicode, variable timing. 'wait' — pause: {type,duration}. 'scroll' — {type,x,y,dx,dy,steps} — human-like scroll: moves mouse to (x,y) first, then sends dy (vertical, +up) and dx (horizontal, +right) as multiple ticks with natural timing. steps=4 by default. Mouse commands may include coord_space='logical' (default) or coord_space='physical'. On macOS, 'physical' means actual screen pixels from td_get_screen_screenshot and is converted to CGEvent logical coords automatically. Top-level coord_space applies to commands that do not override it. on_error: 'stop' (default) clears queue on error; 'continue' skips failed command. IMPORTANT: call td_get_hints('input_simulation') before first use to learn focus recovery, coordinate systems, and testing workflow.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `commands` | array | yes | List of command dicts to execute in sequence. |
|
||||
| `coord_space` | `logical` / `physical` | no | Default coordinate space for mouse commands that do not specify their own coord_space. 'logical' uses CGEvent coords directly. 'physical' uses actual screen pixels from td_get_screen_screenshot and is auto-converted on macOS. |
|
||||
| `on_error` | `stop` / `continue` | no | What to do on error. Default 'stop'. |
|
||||
|
||||
### td_input_status
|
||||
|
||||
Get current status of the td_input command queue. Poll this after td_input_execute until status='idle'. Returns: status ('idle'/'running'), current command, queue_remaining, last error.
|
||||
|
||||
No parameters (other than optional `target_instance`).
|
||||
|
||||
### td_input_clear
|
||||
|
||||
Clear the td_input command queue and stop current execution immediately.
|
||||
|
||||
No parameters (other than optional `target_instance`).
|
||||
|
||||
### td_op_screen_rect
|
||||
|
||||
Get the screen coordinates of an operator node in the network editor. Returns {x,y,w,h,cx,cy} where cx,cy is the center for clicking. Use this to find where to click on a specific operator. Only works if the operator's parent network is currently open in a network editor pane.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Full path to the operator, e.g. '/project1/myComp/noise1' |
|
||||
|
||||
### td_click_screen_point
|
||||
|
||||
Resolve a point inside a previous td_get_screen_screenshot result and click it. Pass the screenshot request_id plus either normalized u/v or image_x/image_y. Queues a td_input click using physical screen coordinates, so it works directly with screenshot-derived points. Use duration/easing to control the cursor travel before the click.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `request_id` | string | yes | Request ID originally returned by td_get_screen_screenshot. |
|
||||
| `u` | number | no | Normalized horizontal position inside the screenshot region (0=left, 1=right). Use with v. |
|
||||
| `v` | number | no | Normalized vertical position inside the screenshot region (0=top, 1=bottom). Use with u. |
|
||||
| `image_x` | number | no | Horizontal pixel coordinate inside the returned screenshot image. Use with image_y. |
|
||||
| `image_y` | number | no | Vertical pixel coordinate inside the returned screenshot image. Use with image_x. |
|
||||
| `button` | `left` / `right` / `middle` | no | Mouse button to click. Default left. |
|
||||
| `hold` | number | no | Seconds to hold the mouse button down before releasing. |
|
||||
| `duration` | number | no | Seconds for the cursor to travel to the target before clicking. |
|
||||
| `easing` | `linear` / `ease-in` / `ease-out` / `ease-in-out` | no | Cursor movement easing for the pre-click travel. |
|
||||
| `focus` | boolean | no | If true, bring TD to the front before clicking and wait briefly for focus to settle. |
|
||||
|
||||
### td_screen_point_to_global
|
||||
|
||||
Convert a point inside a previous td_get_screen_screenshot result into absolute screen coordinates. Pass the screenshot request_id plus either normalized u/v (0..1 inside that screenshot region) or image_x/image_y in returned image pixels. Returns absolute physical screen coordinates, logical coordinates, and a ready-to-use td_input_execute payload. Metadata is kept for the most recent screen screenshots so multiple agents can resolve points later by request_id.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `request_id` | string | yes | Request ID originally returned by td_get_screen_screenshot. |
|
||||
| `u` | number | no | Normalized horizontal position inside the screenshot region (0=left, 1=right). Use with v. |
|
||||
| `v` | number | no | Normalized vertical position inside the screenshot region (0=top, 1=bottom). Use with u. |
|
||||
| `image_x` | number | no | Horizontal pixel coordinate inside the returned screenshot image. Use with image_y. |
|
||||
| `image_y` | number | no | Vertical pixel coordinate inside the returned screenshot image. Use with image_x. |
|
||||
|
||||
## System
|
||||
|
||||
### td_list_instances
|
||||
|
||||
List all running TouchDesigner (TD) instances with active MCP servers. Returns port, project name, PID, and instanceId for each instance. Call this at the start of every conversation to discover available instances and choose which one to work with. instanceId is stable for the lifetime of a TD process and is used as target_instance in all other tool calls.
|
||||
|
||||
No parameters (other than optional `target_instance`).
|
||||
|
||||
### td_project_quit
|
||||
|
||||
Save and/or close the current TouchDesigner (TD) project. Can save before closing. Reports if project has unsaved changes. To close a different instance, pass target_instance=instanceId. WARNING: this will shut down the MCP server on that instance.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `save` | boolean | no | Save the project before closing. Default true. |
|
||||
| `force` | boolean | no | Force close without save dialog. Default false. |
|
||||
|
||||
### td_reinit_extension
|
||||
|
||||
Reinitialize an extension on a COMP in TouchDesigner (TD). Call this AFTER finishing all code edits via td_write_dat to apply changes. Do NOT call after every small edit - batch your changes first.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `path` | string | yes | Path to the COMP with the extension |
|
||||
|
||||
### td_dev_log
|
||||
|
||||
Read the last N entries from the MCP dev log. Only available when Devmode is enabled. Shows request/response history.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `count` | integer | no | Number of recent log entries to return |
|
||||
|
||||
### td_clear_dev_log
|
||||
|
||||
Clear the current MCP dev log by closing the old file and starting a fresh one. Only available when Devmode is enabled.
|
||||
|
||||
No parameters (other than optional `target_instance`).
|
||||
|
||||
### td_test_session
|
||||
|
||||
Manage test sessions, bug reports, and conversation export. IMPORTANT: Do NOT proactively suggest exporting chat or submitting reports. These are tools for specific situations: - export_chat / submit_report: ONLY when the user encounters a BUG with the plugin or TouchDesigner and wants to report it, or when the user explicitly asks to export the conversation. Never suggest this at session end or as routine action. USER PHRASES → ACTIONS: 'разбор тестовых сессий' / 'analyze test sessions' → list, then pull, read meta.json → index.jsonl → calls/. 'разбор репортов' / 'analyze user reports' → list with session='user', then pull by name. 'экспортируй чат' / 'export chat' → (1) export_chat_id → marker, (2) export_chat with session=marker. 'сообщи о проблеме' / 'report bug' → export chat, review for privacy, then submit_report with summary + tags + result_op=file_path. ACTIONS: export_chat_id | export_chat | submit_report | start | note | import_chat | end | list | pull. list: default=auto-detect repo. session='user' for user_reports (dev only). pull: auto-searches both repos. Auto-detects dev vs user Hub access.
|
||||
|
||||
| Param | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `action` | `export_chat_id` / `export_chat` / `submit_report` / `start` / `note` / `import_chat` / `end` / `list` / `pull` | yes | Action: export_chat_id / export_chat / submit_report / start / note / import_chat / end / list / pull |
|
||||
| `prompt` | string | no | (start) The test prompt/task description |
|
||||
| `tags` | array | no | (start) Tags for categorization, e.g. ['ui', 'layout'] |
|
||||
| `text` | string | no | (note) Observation text. (import_chat) Full conversation text. |
|
||||
| `outcome` | `success` / `partial` / `failure` | no | (end) Result: success / partial / failure |
|
||||
| `summary` | string | no | (end) Brief summary of what happened |
|
||||
| `result_op` | string | no | (end) Path to operator to save as result.tox |
|
||||
| `session` | string | no | (pull) Session name or substring to download |
|
||||
@@ -0,0 +1,211 @@
|
||||
# MIDI / OSC Reference
|
||||
|
||||
External controller input and output — MIDI hardware, TouchOSC mobile UIs, OSC routing across the network.
|
||||
|
||||
For audio-driven MIDI patterns (track triggers from spectrum analysis), see also `audio-reactive.md`.
|
||||
|
||||
---
|
||||
|
||||
## MIDI Input — Hardware Controllers
|
||||
|
||||
### Discovery
|
||||
|
||||
List connected MIDI devices first. Use a `midiinDAT` to enumerate:
|
||||
|
||||
```python
|
||||
mdat = root.create(midiinDAT, 'mid_devices')
|
||||
# Read available device names from the DAT after one cook
|
||||
```
|
||||
|
||||
Or via Python directly:
|
||||
|
||||
```python
|
||||
# In td_execute_python
|
||||
import td
|
||||
devices = [d for d in op.MIDI.devices] # verify with td_get_docs('midi')
|
||||
```
|
||||
|
||||
Verify the API with `td_get_docs(topic='midi')` since this varies between TD versions.
|
||||
|
||||
### MIDI In CHOP
|
||||
|
||||
Standard pattern:
|
||||
|
||||
```python
|
||||
midi_in = root.create(midiinCHOP, 'midi_in')
|
||||
midi_in.par.device = 0 # device index from discovery
|
||||
midi_in.par.activechan = True
|
||||
```
|
||||
|
||||
Output channels follow the convention `chCcN` and `chCnN`:
|
||||
- `ch1c74` — channel 1, CC 74
|
||||
- `ch1n60` — channel 1, note 60 (middle C) — value is velocity 0-127
|
||||
|
||||
**Map a CC to a parameter:**
|
||||
|
||||
```python
|
||||
op('/project1/bloom1').par.threshold.mode = ParMode.EXPRESSION
|
||||
op('/project1/bloom1').par.threshold.expr = "op('midi_in')['ch1c74'][0] / 127.0"
|
||||
```
|
||||
|
||||
**Map a note as a trigger:**
|
||||
|
||||
Notes in `midiinCHOP` output velocity while held, 0 when released. Use a `triggerCHOP` to convert a held note into pulses:
|
||||
|
||||
```python
|
||||
trig = root.create(triggerCHOP, 'note_trig')
|
||||
trig.par.threshold = 1
|
||||
trig.par.triggeron = 'increase'
|
||||
trig.inputConnectors[0].connect(op('midi_in'))
|
||||
# Filter to a single channel via a selectCHOP if desired
|
||||
```
|
||||
|
||||
### MIDI Learn Pattern
|
||||
|
||||
Build a reusable learn pattern when you don't know the controller's CC layout in advance:
|
||||
|
||||
1. Drop a `midiinCHOP` and `selectCHOP` after it.
|
||||
2. User wiggles the controller knob.
|
||||
3. Use `td_read_chop` on the midiinCHOP to identify which channel is non-zero — that's the active CC.
|
||||
4. Set the `selectCHOP.par.channames` to that channel name.
|
||||
5. Save the mapping to a `tableDAT` so it persists across sessions.
|
||||
|
||||
---
|
||||
|
||||
## MIDI Output
|
||||
|
||||
```python
|
||||
midi_out = root.create(midioutCHOP, 'midi_out')
|
||||
midi_out.par.device = 0
|
||||
midi_out.par.outputformat = 'continuous' # 'continuous' | 'event'
|
||||
|
||||
# Drive an output: send out a CC mapped from any 0-1 source
|
||||
src = root.create(constantCHOP, 'cc_src')
|
||||
src.par.name0 = 'ch1c20'
|
||||
src.par.value0 = 0.5
|
||||
midi_out.inputConnectors[0].connect(src)
|
||||
```
|
||||
|
||||
For note events specifically, use `event` mode and pulse the value with a `pulseCHOP` or `triggerCHOP`.
|
||||
|
||||
---
|
||||
|
||||
## OSC Input — Network Control
|
||||
|
||||
OSC is the more flexible cousin of MIDI. Used heavily for:
|
||||
- TouchOSC / Lemur mobile control surfaces
|
||||
- Show control systems (QLab, Watchout)
|
||||
- Inter-application sync (Ableton via Max for Live, Resolume, etc.)
|
||||
|
||||
### OSC In CHOP
|
||||
|
||||
```python
|
||||
osc_in = root.create(oscinCHOP, 'osc_in')
|
||||
osc_in.par.port = 7000 # listen on UDP 7000
|
||||
osc_in.par.localaddress = '' # empty = all interfaces
|
||||
osc_in.par.queued = False # immediate vs. queued processing
|
||||
```
|
||||
|
||||
Each incoming OSC address becomes a channel. `/scene/1/intensity` becomes a channel named `scene_1_intensity` (TD sanitizes slashes to underscores).
|
||||
|
||||
**Common gotcha:** TD only creates the channel after the FIRST message arrives at that address. Send a "hello" message from the controller during setup, or pre-declare channel names manually.
|
||||
|
||||
### OSC In DAT (for raw events)
|
||||
|
||||
Use a `oscinDAT` when you need full message access (multiple typed args, addresses with brackets/regex).
|
||||
|
||||
```python
|
||||
osc_dat = root.create(oscinDAT, 'osc_events')
|
||||
osc_dat.par.port = 7001
|
||||
# Each row: timestamp, address, type tags, args...
|
||||
```
|
||||
|
||||
Drive logic via a `datExecuteDAT` watching the `oscinDAT`:
|
||||
|
||||
```python
|
||||
def onTableChange(dat):
|
||||
last = dat[dat.numRows - 1, 'message']
|
||||
parsed = last.val.split()
|
||||
addr = parsed[0]
|
||||
args = parsed[1:]
|
||||
if addr == '/scene/trigger':
|
||||
op('/project1/scene_switcher').par.index = int(args[0])
|
||||
return
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## OSC Output — Sending to External Apps
|
||||
|
||||
```python
|
||||
osc_out = root.create(oscoutCHOP, 'osc_out')
|
||||
osc_out.par.netaddress = '127.0.0.1' # destination IP
|
||||
osc_out.par.port = 9000
|
||||
|
||||
# Channel names become OSC addresses
|
||||
src = root.create(constantCHOP, 'send')
|
||||
src.par.name0 = 'scene/intensity' # → /scene/intensity
|
||||
src.par.value0 = 0.7
|
||||
osc_out.inputConnectors[0].connect(src)
|
||||
```
|
||||
|
||||
**Channel-to-address mapping:** TD prepends `/` automatically. Use `/` in channel names to nest.
|
||||
|
||||
For one-shot string/typed messages, use `oscoutDAT` and call `.sendOSC(address, args)`:
|
||||
|
||||
```python
|
||||
op('osc_out_dat').sendOSC('/scene/trigger', [1, 'fade'])
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## TouchOSC / Mobile UI Pattern
|
||||
|
||||
Common setup for live VJ control from a phone/tablet:
|
||||
|
||||
1. **Configure TouchOSC layout** — assign each control an OSC address like `/vj/master`, `/vj/scene/1`, etc.
|
||||
2. **Find your machine's LAN IP** — TouchOSC needs to point at it.
|
||||
3. **TD listens** on `oscinCHOP.par.port = 8000` (or whichever).
|
||||
4. **Map channels to params** via expressions:
|
||||
|
||||
```python
|
||||
op('/project1/master_level').par.opacity.mode = ParMode.EXPRESSION
|
||||
op('/project1/master_level').par.opacity.expr = "op('osc_in')['vj_master']"
|
||||
```
|
||||
|
||||
5. **Send feedback** to the controller via `oscoutCHOP` — useful for syncing state across multiple devices.
|
||||
|
||||
---
|
||||
|
||||
## Network / Multi-Machine
|
||||
|
||||
OSC over LAN works out-of-the-box. For multi-TD-instance sync (e.g., projection cluster):
|
||||
|
||||
- One TD acts as **master**, broadcasts `/sync/...` over OSC
|
||||
- Worker TDs run `oscinCHOP` listening on the same port
|
||||
- Use UDP **broadcast address** (e.g., `192.168.1.255`) on the master's `oscoutCHOP.par.netaddress` to hit all peers
|
||||
|
||||
For reliability over WAN, use `webserverDAT` or `websocketDAT` with an external relay instead — UDP loss is invisible.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **MIDI device indexing** — device `0` is whichever device TD enumerated first. Reorder may shift it. Pin by name when possible.
|
||||
2. **OSC channel names** — TD doesn't create a channel until the first message lands. New channels invalidate cooked dependents on first arrival, causing a one-frame stutter.
|
||||
3. **OSC queued mode** — `par.queued = True` defers processing to a single per-frame batch. Lower latency but messages arriving same frame collapse to the last value. Off for triggers, on for continuous knobs.
|
||||
4. **MIDI clock vs. transport** — `midiinCHOP` reports clock if available. Use `midisyncCHOP` (if your TD version exposes it) or compute BPM from clock pulses (24 per quarter note).
|
||||
5. **Latency** — wired MIDI is ~1-3ms. WiFi OSC is 10-30ms with jitter. Use wired for tight beat-locked work.
|
||||
6. **Port conflicts** — only one process can bind a UDP port on most OS. If `oscinCHOP` shows no traffic, check that another app (Max, Ableton, etc.) isn't already listening on that port.
|
||||
|
||||
---
|
||||
|
||||
## Quick Recipes
|
||||
|
||||
| Goal | Op chain |
|
||||
|---|---|
|
||||
| Knob → bloom intensity | `midiinCHOP` → expression on `bloom.par.threshold` |
|
||||
| Note → scene change | `midiinCHOP` → `triggerCHOP` → `selectCHOP` → drive `switchTOP.par.index` |
|
||||
| Phone slider → master fader | TouchOSC `/master` → `oscinCHOP` → expression on output `level.par.opacity` |
|
||||
| TD → Resolume scene trigger | `oscoutCHOP` channel `composition/layers/1/clips/1/connect` → Resolume listening on 7000 |
|
||||
| Multi-projector sync | Master TD `oscoutCHOP` broadcast → workers `oscinCHOP` |
|
||||
@@ -0,0 +1,966 @@
|
||||
# TouchDesigner Network Patterns
|
||||
|
||||
Complete network recipes for common creative coding tasks. Each pattern shows the operator chain, MCP tool calls to build it, and key parameter settings.
|
||||
|
||||
## Audio-Reactive Visuals
|
||||
|
||||
### Pattern 1: Audio Spectrum -> Noise Displacement
|
||||
|
||||
Audio drives noise parameters for organic, music-responsive textures.
|
||||
|
||||
```
|
||||
Audio File In CHOP -> Audio Spectrum CHOP -> Math CHOP (scale)
|
||||
|
|
||||
v (export to noise params)
|
||||
Noise TOP -> Level TOP -> Feedback TOP -> Composite TOP -> Null TOP (out)
|
||||
^ |
|
||||
|________________|
|
||||
```
|
||||
|
||||
**MCP Build Sequence:**
|
||||
|
||||
```
|
||||
1. td_create_operator(parent="/project1", type="audiofileinChop", name="audio_in")
|
||||
2. td_create_operator(parent="/project1", type="audiospectrumChop", name="spectrum")
|
||||
3. td_create_operator(parent="/project1", type="mathChop", name="spectrum_scale")
|
||||
4. td_create_operator(parent="/project1", type="noiseTop", name="noise1")
|
||||
5. td_create_operator(parent="/project1", type="levelTop", name="level1")
|
||||
6. td_create_operator(parent="/project1", type="feedbackTop", name="feedback1")
|
||||
7. td_create_operator(parent="/project1", type="compositeTop", name="comp1")
|
||||
8. td_create_operator(parent="/project1", type="nullTop", name="out")
|
||||
|
||||
9. td_set_operator_pars(path="/project1/audio_in",
|
||||
properties={"file": "/path/to/music.wav", "play": true})
|
||||
10. td_set_operator_pars(path="/project1/spectrum",
|
||||
properties={"size": 512})
|
||||
11. td_set_operator_pars(path="/project1/spectrum_scale",
|
||||
properties={"gain": 2.0, "postoff": 0.0})
|
||||
12. td_set_operator_pars(path="/project1/noise1",
|
||||
properties={"type": 1, "monochrome": false, "resolutionw": 1280, "resolutionh": 720,
|
||||
"period": 4.0, "harmonics": 3, "amp": 1.0})
|
||||
13. td_set_operator_pars(path="/project1/level1",
|
||||
properties={"opacity": 0.95, "gamma1": 0.75})
|
||||
14. td_set_operator_pars(path="/project1/feedback1",
|
||||
properties={"top": "/project1/comp1"})
|
||||
15. td_set_operator_pars(path="/project1/comp1",
|
||||
properties={"operand": 0})
|
||||
|
||||
16. td_execute_python: """
|
||||
op('/project1/audio_in').outputConnectors[0].connect(op('/project1/spectrum'))
|
||||
op('/project1/spectrum').outputConnectors[0].connect(op('/project1/spectrum_scale'))
|
||||
op('/project1/noise1').outputConnectors[0].connect(op('/project1/level1'))
|
||||
op('/project1/level1').outputConnectors[0].connect(op('/project1/comp1').inputConnectors[0])
|
||||
op('/project1/feedback1').outputConnectors[0].connect(op('/project1/comp1').inputConnectors[1])
|
||||
op('/project1/comp1').outputConnectors[0].connect(op('/project1/out'))
|
||||
"""
|
||||
|
||||
17. td_execute_python: """
|
||||
# Export spectrum values to drive noise parameters
|
||||
# This makes the noise react to audio frequencies
|
||||
op('/project1/noise1').par.seed.expr = "op('/project1/spectrum_scale')['chan1']"
|
||||
op('/project1/noise1').par.period.expr = "tdu.remap(op('/project1/spectrum_scale')['chan1'].eval(), 0, 1, 1, 8)"
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 2: Beat Detection -> Visual Pulses
|
||||
|
||||
Detect beats from audio and trigger visual events.
|
||||
|
||||
```
|
||||
Audio Device In CHOP -> Audio Spectrum CHOP -> Math CHOP (isolate bass)
|
||||
|
|
||||
Trigger CHOP (envelope)
|
||||
|
|
||||
[export to visual params]
|
||||
```
|
||||
|
||||
**Key parameter settings:**
|
||||
|
||||
```
|
||||
# Isolate bass frequencies (20-200 Hz)
|
||||
Math CHOP: chanop=1 (Add channels), range1low=0, range1high=10
|
||||
(first 10 FFT bins = bass frequencies with 512 FFT at 44100Hz)
|
||||
|
||||
# ADSR envelope on each beat
|
||||
Trigger CHOP: attack=0.02, peak=1.0, decay=0.3, sustain=0.0, release=0.1
|
||||
|
||||
# Export to visual: Scale, brightness, or color intensity
|
||||
td_execute_python: "op('/project1/level1').par.brightness1.expr = \"1.0 + op('/project1/trigger1')['chan1'] * 0.5\""
|
||||
```
|
||||
|
||||
### Pattern 3: Multi-Band Audio -> Multi-Layer Visuals
|
||||
|
||||
Split audio into frequency bands, drive different visual layers per band.
|
||||
|
||||
```
|
||||
Audio In -> Spectrum -> Audio Band EQ (3 bands: bass, mid, treble)
|
||||
|
|
||||
+---------+---------+
|
||||
| | |
|
||||
Bass Mids Treble
|
||||
| | |
|
||||
Noise TOP Circle TOP Text TOP
|
||||
(slow,dark) (mid,warm) (fast,bright)
|
||||
| | |
|
||||
+-----+----+----+----+
|
||||
| |
|
||||
Composite Composite
|
||||
|
|
||||
Out
|
||||
```
|
||||
|
||||
### Pattern 3b: Audio-Reactive GLSL Fractal (Proven Recipe)
|
||||
|
||||
Complete working recipe. Plays an MP3, runs FFT, feeds spectrum as a texture into a GLSL shader where inner fractal reacts to bass, outer to treble.
|
||||
|
||||
**Network:**
|
||||
```
|
||||
AudioFileIn CHOP → AudioSpectrum CHOP (FFT=512, outlength=256)
|
||||
→ Math CHOP (gain=10) → CHOP To TOP (256x2 spectrum texture, dataformat=r)
|
||||
↓
|
||||
Constant TOP (time, rgba32float) → GLSL TOP (input 0=time, input 1=spectrum) → Null → MovieFileOut
|
||||
↓
|
||||
AudioFileIn CHOP → Audio Device Out CHOP Record to .mov
|
||||
```
|
||||
|
||||
**Build via td_execute_python (one call per step for reliability):**
|
||||
|
||||
```python
|
||||
# Step 1: Audio chain
|
||||
# td_execute_python script:
|
||||
td_execute_python(code="""
|
||||
root = op('/project1')
|
||||
audio = root.create(audiofileinCHOP, 'audio_in')
|
||||
audio.par.file = '/path/to/music.mp3'
|
||||
audio.par.playmode = 0 # Locked to timeline
|
||||
audio.par.volume = 0.5
|
||||
|
||||
spec = root.create(audiospectrumCHOP, 'spectrum')
|
||||
audio.outputConnectors[0].connect(spec.inputConnectors[0])
|
||||
|
||||
math_n = root.create(mathCHOP, 'math_norm')
|
||||
spec.outputConnectors[0].connect(math_n.inputConnectors[0])
|
||||
math_n.par.gain = 5 # boost signal
|
||||
|
||||
resamp = root.create(resampleCHOP, 'resample_spec')
|
||||
math_n.outputConnectors[0].connect(resamp.inputConnectors[0])
|
||||
resamp.par.timeslice = True
|
||||
resamp.par.rate = 256
|
||||
|
||||
chop2top = root.create(choptoTOP, 'spectrum_tex')
|
||||
chop2top.par.chop = resamp # CHOP To TOP has NO input connectors — use par.chop reference
|
||||
|
||||
# Audio output (hear the music)
|
||||
aout = root.create(audiodeviceoutCHOP, 'audio_out')
|
||||
audio.outputConnectors[0].connect(aout.inputConnectors[0])
|
||||
result = 'audio chain ok'
|
||||
""")
|
||||
|
||||
# Step 2: Time driver (MUST be rgba32float — see pitfalls #6)
|
||||
# td_execute_python script:
|
||||
td_execute_python(code="""
|
||||
root = op('/project1')
|
||||
td = root.create(constantTOP, 'time_driver')
|
||||
td.par.format = 'rgba32float'
|
||||
td.par.outputresolution = 'custom'
|
||||
td.par.resolutionw = 1
|
||||
td.par.resolutionh = 1
|
||||
td.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
td.par.colorg.expr = "int(absTime.seconds / 1000.0)"
|
||||
result = 'time ok'
|
||||
""")
|
||||
|
||||
# Step 3: GLSL shader (write to /tmp, load from file)
|
||||
# td_execute_python script:
|
||||
td_execute_python(code="""
|
||||
root = op('/project1')
|
||||
glsl = root.create(glslTOP, 'audio_shader')
|
||||
glsl.par.outputresolution = 'custom'
|
||||
glsl.par.resolutionw = 1280
|
||||
glsl.par.resolutionh = 720
|
||||
|
||||
sd = root.create(textDAT, 'shader_code')
|
||||
sd.text = open('/tmp/my_shader.glsl').read()
|
||||
glsl.par.pixeldat = sd
|
||||
|
||||
# Wire: input 0 = time, input 1 = spectrum texture
|
||||
op('/project1/time_driver').outputConnectors[0].connect(glsl.inputConnectors[0])
|
||||
op('/project1/spectrum_tex').outputConnectors[0].connect(glsl.inputConnectors[1])
|
||||
result = 'glsl ok'
|
||||
""")
|
||||
|
||||
# Step 4: Output + recorder
|
||||
# td_execute_python script:
|
||||
td_execute_python(code="""
|
||||
root = op('/project1')
|
||||
out = root.create(nullTOP, 'output')
|
||||
op('/project1/audio_shader').outputConnectors[0].connect(out.inputConnectors[0])
|
||||
|
||||
rec = root.create(moviefileoutTOP, 'recorder')
|
||||
out.outputConnectors[0].connect(rec.inputConnectors[0])
|
||||
rec.par.type = 'movie'
|
||||
rec.par.file = '/tmp/output.mov'
|
||||
rec.par.videocodec = 'mjpa'
|
||||
result = 'output ok'
|
||||
""")
|
||||
```
|
||||
|
||||
**GLSL shader pattern (audio-reactive fractal):**
|
||||
```glsl
|
||||
out vec4 fragColor;
|
||||
|
||||
vec3 palette(float t) {
|
||||
vec3 a = vec3(0.5); vec3 b = vec3(0.5);
|
||||
vec3 c = vec3(1.0); vec3 d = vec3(0.263, 0.416, 0.557);
|
||||
return a + b * cos(6.28318 * (c * t + d));
|
||||
}
|
||||
|
||||
void main() {
|
||||
// Input 0 = time (1x1 rgba32float constant)
|
||||
// Input 1 = audio spectrum (256x2 CHOP To TOP, stereo — sample at y=0.25 for first channel)
|
||||
vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
float t = td.r + td.g * 1000.0;
|
||||
|
||||
vec2 res = uTDOutputInfo.res.zw;
|
||||
vec2 uv = (gl_FragCoord.xy * 2.0 - res) / min(res.x, res.y);
|
||||
vec2 uv0 = uv;
|
||||
vec3 finalColor = vec3(0.0);
|
||||
|
||||
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.25)).r;
|
||||
float mids = texture(sTD2DInputs[1], vec2(0.25, 0.25)).r;
|
||||
|
||||
for (float i = 0.0; i < 4.0; i++) {
|
||||
uv = fract(uv * (1.4 + bass * 0.3)) - 0.5;
|
||||
float d = length(uv) * exp(-length(uv0));
|
||||
|
||||
// Sample spectrum at distance: inner=bass, outer=treble
|
||||
float freq = texture(sTD2DInputs[1], vec2(clamp(d * 0.5, 0.0, 1.0), 0.25)).r;
|
||||
|
||||
vec3 col = palette(length(uv0) + i * 0.4 + t * 0.35);
|
||||
d = sin(d * (7.0 + bass * 4.0) + t * 1.5) / 8.0;
|
||||
d = abs(d);
|
||||
d = pow(0.012 / d, 1.2 + freq * 0.8 + bass * 0.5);
|
||||
finalColor += col * d;
|
||||
}
|
||||
|
||||
// Tone mapping
|
||||
finalColor = finalColor / (finalColor + vec3(1.0));
|
||||
fragColor = TDOutputSwizzle(vec4(finalColor, 1.0));
|
||||
}
|
||||
```
|
||||
|
||||
**Key insights from testing:**
|
||||
- `spectrum_tex` (CHOP To TOP) produces a 256x2 texture — x position = frequency, y=0.25 for first channel
|
||||
- Sampling at `vec2(0.05, 0.0)` gets bass, `vec2(0.65, 0.0)` gets treble
|
||||
- Sampling based on pixel distance (`d * 0.5`) makes inner fractal react to bass, outer to treble
|
||||
- `bass * 0.3` in the `fract()` zoom makes the fractal breathe with kicks
|
||||
- Math CHOP gain of 5 is needed because raw spectrum values are very small
|
||||
|
||||
## Generative Art
|
||||
|
||||
### Pattern 4: Feedback Loop with Transform
|
||||
|
||||
Classic generative technique — texture evolves through recursive transformation.
|
||||
|
||||
```
|
||||
Noise TOP -> Composite TOP -> Level TOP -> Null TOP (out)
|
||||
^ |
|
||||
| v
|
||||
Transform TOP <- Feedback TOP
|
||||
```
|
||||
|
||||
**MCP Build Sequence:**
|
||||
|
||||
```
|
||||
1. td_create_operator(parent="/project1", type="noiseTop", name="seed_noise")
|
||||
2. td_create_operator(parent="/project1", type="compositeTop", name="mix")
|
||||
3. td_create_operator(parent="/project1", type="transformTop", name="evolve")
|
||||
4. td_create_operator(parent="/project1", type="feedbackTop", name="fb")
|
||||
5. td_create_operator(parent="/project1", type="levelTop", name="color_correct")
|
||||
6. td_create_operator(parent="/project1", type="nullTop", name="out")
|
||||
|
||||
7. td_set_operator_pars(path="/project1/seed_noise",
|
||||
properties={"type": 1, "monochrome": false, "period": 2.0, "amp": 0.3,
|
||||
"resolutionw": 1280, "resolutionh": 720})
|
||||
8. td_set_operator_pars(path="/project1/mix",
|
||||
properties={"operand": 27}) # 27 = Screen blend
|
||||
9. td_set_operator_pars(path="/project1/evolve",
|
||||
properties={"sx": 1.003, "sy": 1.003, "rz": 0.5, "extend": 2}) # slight zoom + rotate, repeat edges
|
||||
10. td_set_operator_pars(path="/project1/fb",
|
||||
properties={"top": "/project1/mix"})
|
||||
11. td_set_operator_pars(path="/project1/color_correct",
|
||||
properties={"opacity": 0.98, "gamma1": 0.85})
|
||||
|
||||
12. td_execute_python: """
|
||||
op('/project1/seed_noise').outputConnectors[0].connect(op('/project1/mix').inputConnectors[0])
|
||||
op('/project1/fb').outputConnectors[0].connect(op('/project1/evolve'))
|
||||
op('/project1/evolve').outputConnectors[0].connect(op('/project1/mix').inputConnectors[1])
|
||||
op('/project1/mix').outputConnectors[0].connect(op('/project1/color_correct'))
|
||||
op('/project1/color_correct').outputConnectors[0].connect(op('/project1/out'))
|
||||
"""
|
||||
```
|
||||
|
||||
**Variations:**
|
||||
- Change Transform: `rz` (rotation), `sx/sy` (zoom), `tx/ty` (drift)
|
||||
- Change Composite operand: Screen (glow), Add (bright), Multiply (dark)
|
||||
- Add HSV Adjust in the feedback loop for color evolution
|
||||
- Add Blur for dreamlike softness
|
||||
- Replace Noise with a GLSL TOP for custom seed patterns
|
||||
|
||||
### Pattern 5: Instancing (Particle-Like Systems)
|
||||
|
||||
Render thousands of copies of geometry, each with unique position/rotation/scale driven by CHOP data or DATs.
|
||||
|
||||
```
|
||||
Table DAT (instance data) -> DAT to CHOP -> Geometry COMP (instancing on) -> Render TOP
|
||||
+ Sphere SOP (template geometry)
|
||||
+ Constant MAT (material)
|
||||
+ Camera COMP
|
||||
+ Light COMP
|
||||
```
|
||||
|
||||
**MCP Build Sequence:**
|
||||
|
||||
```
|
||||
1. td_create_operator(parent="/project1", type="tableDat", name="instance_data")
|
||||
2. td_create_operator(parent="/project1", type="geometryComp", name="geo1")
|
||||
3. td_create_operator(parent="/project1/geo1", type="sphereSop", name="sphere")
|
||||
4. td_create_operator(parent="/project1", type="constMat", name="mat1")
|
||||
5. td_create_operator(parent="/project1", type="cameraComp", name="cam1")
|
||||
6. td_create_operator(parent="/project1", type="lightComp", name="light1")
|
||||
7. td_create_operator(parent="/project1", type="renderTop", name="render1")
|
||||
|
||||
8. td_execute_python: """
|
||||
import random, math
|
||||
dat = op('/project1/instance_data')
|
||||
dat.clear()
|
||||
dat.appendRow(['tx', 'ty', 'tz', 'sx', 'sy', 'sz', 'cr', 'cg', 'cb'])
|
||||
for i in range(500):
|
||||
angle = i * 0.1
|
||||
r = 2 + i * 0.01
|
||||
dat.appendRow([
|
||||
str(math.cos(angle) * r),
|
||||
str(math.sin(angle) * r),
|
||||
str((i - 250) * 0.02),
|
||||
'0.05', '0.05', '0.05',
|
||||
str(random.random()),
|
||||
str(random.random()),
|
||||
str(random.random())
|
||||
])
|
||||
"""
|
||||
|
||||
9. td_set_operator_pars(path="/project1/geo1",
|
||||
properties={"instancing": true, "instancechop": "",
|
||||
"instancedat": "/project1/instance_data",
|
||||
"material": "/project1/mat1"})
|
||||
10. td_set_operator_pars(path="/project1/render1",
|
||||
properties={"camera": "/project1/cam1", "geometry": "/project1/geo1",
|
||||
"light": "/project1/light1",
|
||||
"resolutionw": 1280, "resolutionh": 720})
|
||||
11. td_set_operator_pars(path="/project1/cam1",
|
||||
properties={"tz": 10})
|
||||
```
|
||||
|
||||
### Pattern 6: Reaction-Diffusion (GLSL)
|
||||
|
||||
Classic Gray-Scott reaction-diffusion system running on the GPU.
|
||||
|
||||
```
|
||||
Text DAT (GLSL code) -> GLSL TOP (resolution, dat reference) -> Feedback TOP
|
||||
^ |
|
||||
|_______________________________________|
|
||||
Level TOP (out)
|
||||
```
|
||||
|
||||
**Key GLSL code (write to Text DAT via td_execute_python):**
|
||||
|
||||
```glsl
|
||||
// Gray-Scott reaction-diffusion
|
||||
uniform float feed; // 0.037
|
||||
uniform float kill; // 0.06
|
||||
uniform float dA; // 1.0
|
||||
uniform float dB; // 0.5
|
||||
|
||||
layout(location = 0) out vec4 fragColor;
|
||||
|
||||
void main() {
|
||||
vec2 uv = vUV.st;
|
||||
vec2 texel = 1.0 / uTDOutputInfo.res.zw;
|
||||
|
||||
vec4 c = texture(sTD2DInputs[0], uv);
|
||||
float a = c.r;
|
||||
float b = c.g;
|
||||
|
||||
// Laplacian (9-point stencil)
|
||||
float lA = 0.0, lB = 0.0;
|
||||
for(int dx = -1; dx <= 1; dx++) {
|
||||
for(int dy = -1; dy <= 1; dy++) {
|
||||
float w = (dx == 0 && dy == 0) ? -1.0 : (abs(dx) + abs(dy) == 1 ? 0.2 : 0.05);
|
||||
vec4 s = texture(sTD2DInputs[0], uv + vec2(dx, dy) * texel);
|
||||
lA += s.r * w;
|
||||
lB += s.g * w;
|
||||
}
|
||||
}
|
||||
|
||||
float reaction = a * b * b;
|
||||
float newA = a + (dA * lA - reaction + feed * (1.0 - a));
|
||||
float newB = b + (dB * lB + reaction - (kill + feed) * b);
|
||||
|
||||
fragColor = vec4(clamp(newA, 0.0, 1.0), clamp(newB, 0.0, 1.0), 0.0, 1.0);
|
||||
}
|
||||
```
|
||||
|
||||
## Video Processing
|
||||
|
||||
### Pattern 7: Video Effects Chain
|
||||
|
||||
Apply a chain of effects to a video file.
|
||||
|
||||
```
|
||||
Movie File In TOP -> HSV Adjust TOP -> Level TOP -> Blur TOP -> Composite TOP -> Null TOP (out)
|
||||
^
|
||||
Text TOP ---+
|
||||
```
|
||||
|
||||
**MCP Build Sequence:**
|
||||
|
||||
```
|
||||
1. td_create_operator(parent="/project1", type="moviefileinTop", name="video_in")
|
||||
2. td_create_operator(parent="/project1", type="hsvadjustTop", name="color")
|
||||
3. td_create_operator(parent="/project1", type="levelTop", name="levels")
|
||||
4. td_create_operator(parent="/project1", type="blurTop", name="blur")
|
||||
5. td_create_operator(parent="/project1", type="compositeTop", name="overlay")
|
||||
6. td_create_operator(parent="/project1", type="textTop", name="title")
|
||||
7. td_create_operator(parent="/project1", type="nullTop", name="out")
|
||||
|
||||
8. td_set_operator_pars(path="/project1/video_in",
|
||||
properties={"file": "/path/to/video.mp4", "play": true})
|
||||
9. td_set_operator_pars(path="/project1/color",
|
||||
properties={"hueoffset": 0.1, "saturationmult": 1.3})
|
||||
10. td_set_operator_pars(path="/project1/levels",
|
||||
properties={"brightness1": 1.1, "contrast": 1.2, "gamma1": 0.9})
|
||||
11. td_set_operator_pars(path="/project1/blur",
|
||||
properties={"sizex": 2, "sizey": 2})
|
||||
12. td_set_operator_pars(path="/project1/title",
|
||||
properties={"text": "My Video", "fontsizex": 48, "alignx": 1, "aligny": 1})
|
||||
|
||||
13. td_execute_python: """
|
||||
chain = ['video_in', 'color', 'levels', 'blur']
|
||||
for i in range(len(chain) - 1):
|
||||
op(f'/project1/{chain[i]}').outputConnectors[0].connect(op(f'/project1/{chain[i+1]}'))
|
||||
op('/project1/blur').outputConnectors[0].connect(op('/project1/overlay').inputConnectors[0])
|
||||
op('/project1/title').outputConnectors[0].connect(op('/project1/overlay').inputConnectors[1])
|
||||
op('/project1/overlay').outputConnectors[0].connect(op('/project1/out'))
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 8: Video Recording
|
||||
|
||||
Record the output to a file. **H.264/H.265 require a Commercial license** — use Motion JPEG (`mjpa`) on Non-Commercial.
|
||||
|
||||
```
|
||||
[any TOP chain] -> Null TOP -> Movie File Out TOP
|
||||
```
|
||||
|
||||
```python
|
||||
# Build via td_execute_python:
|
||||
root = op('/project1')
|
||||
|
||||
# Always put a Null TOP before the recorder
|
||||
null_out = root.op('out') # or create one
|
||||
rec = root.create(moviefileoutTOP, 'recorder')
|
||||
null_out.outputConnectors[0].connect(rec.inputConnectors[0])
|
||||
|
||||
rec.par.type = 'movie'
|
||||
rec.par.file = '/tmp/output.mov'
|
||||
rec.par.videocodec = 'mjpa' # Motion JPEG — works on Non-Commercial
|
||||
|
||||
# Start recording (par.record is a toggle — .record() method may not exist)
|
||||
rec.par.record = True
|
||||
# ... let TD run for desired duration ...
|
||||
rec.par.record = False
|
||||
|
||||
# For image sequences:
|
||||
# rec.par.type = 'imagesequence'
|
||||
# rec.par.imagefiletype = 'png'
|
||||
# rec.par.file.expr = "'/tmp/frames/out' + me.fileSuffix" # fileSuffix REQUIRED
|
||||
```
|
||||
|
||||
**Pitfalls:**
|
||||
- Setting `par.file` + `par.record = True` in the same script may race — use `run("...", delayFrames=2)`
|
||||
- `TOP.save()` called rapidly always captures the same frame — use MovieFileOut for animation
|
||||
- See `pitfalls.md` #25-27 for full details
|
||||
|
||||
### Pattern 8b: TD → External Pipeline (FFmpeg / Python / Post-Processing)
|
||||
|
||||
Export TD visuals for use in another tool (ffmpeg, Python, ASCII art, etc.). This is the standard workflow when you need to composite TD output with external processing (ASCII conversion, Python shader chains, ML inference, etc.).
|
||||
|
||||
**Step 1: Record to video in TD**
|
||||
|
||||
```python
|
||||
# Preferred: ProRes on macOS (lossless, Non-Commercial OK, ~55MB/s at 1280x720)
|
||||
rec.par.videocodec = 'prores'
|
||||
# Fallback for non-macOS: mjpa (Motion JPEG)
|
||||
# rec.par.videocodec = 'mjpa'
|
||||
rec.par.record = True
|
||||
# ... wait N seconds ...
|
||||
rec.par.record = False
|
||||
```
|
||||
|
||||
**Step 2: Extract frames with ffmpeg**
|
||||
|
||||
```bash
|
||||
# Extract all frames at 30fps
|
||||
ffmpeg -y -i /tmp/output.mov -vf 'fps=30' /tmp/frames/frame_%06d.png
|
||||
|
||||
# Or extract a specific duration
|
||||
ffmpeg -y -i /tmp/output.mov -t 25 -vf 'fps=30' /tmp/frames/frame_%06d.png
|
||||
|
||||
# Or extract specific frame range
|
||||
ffmpeg -y -i /tmp/output.mov -vf 'select=between(n\,0\,749)' -vsync vfr /tmp/frames/frame_%06d.png
|
||||
```
|
||||
|
||||
**Step 3: Process frames in Python**
|
||||
|
||||
```python
|
||||
from PIL import Image
|
||||
import os
|
||||
|
||||
frames_dir = '/tmp/frames'
|
||||
output_dir = '/tmp/processed'
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
for fname in sorted(os.listdir(frames_dir)):
|
||||
if not fname.endswith('.png'):
|
||||
continue
|
||||
img = Image.open(os.path.join(frames_dir, fname))
|
||||
# ... apply your processing ...
|
||||
img.save(os.path.join(output_dir, fname))
|
||||
```
|
||||
|
||||
**Step 4: Mux processed frames back with audio**
|
||||
|
||||
```bash
|
||||
# Create video from processed frames + audio with fade-out
|
||||
ffmpeg -y \
|
||||
-framerate 30 -i /tmp/processed/frame_%06d.png \
|
||||
-i /tmp/audio.mp3 \
|
||||
-c:v libx264 -pix_fmt yuv420p -crf 18 \
|
||||
-c:a aac -b:a 192k \
|
||||
-shortest \
|
||||
-af 'afade=t=out:st=23:d=2' \
|
||||
/tmp/final_output.mp4
|
||||
```
|
||||
|
||||
**Key considerations:**
|
||||
- Use ProRes for the TD recording step to avoid generation loss during compositing
|
||||
- Extract at the target output framerate (not TD's render framerate)
|
||||
- For audio-synced content, analyze the audio file separately in Python (scipy FFT) to get per-frame features (rms, spectral bands, beats) and drive compositing parameters
|
||||
- Always verify TD FPS > 0 before recording (see pitfalls #37, #38)
|
||||
|
||||
## Data Visualization
|
||||
|
||||
### Pattern 9: Table Data -> Bar Chart via Instancing
|
||||
|
||||
Visualize tabular data as a 3D bar chart.
|
||||
|
||||
```
|
||||
Table DAT (data) -> Script DAT (transform to instance format) -> DAT to CHOP
|
||||
|
|
||||
Box SOP -> Geometry COMP (instancing from CHOP) -> Render TOP -> Null TOP (out)
|
||||
+ PBR MAT
|
||||
+ Camera COMP
|
||||
+ Light COMP
|
||||
```
|
||||
|
||||
```python
|
||||
# Script DAT code to transform data to instance positions
|
||||
td_execute_python: """
|
||||
source = op('/project1/data_table')
|
||||
instance = op('/project1/instance_transform')
|
||||
instance.clear()
|
||||
instance.appendRow(['tx', 'ty', 'tz', 'sx', 'sy', 'sz', 'cr', 'cg', 'cb'])
|
||||
|
||||
for i in range(1, source.numRows):
|
||||
value = float(source[i, 'value'])
|
||||
name = source[i, 'name']
|
||||
instance.appendRow([
|
||||
str(i * 1.5), # x position (spread bars)
|
||||
str(value / 2), # y position (center bar vertically)
|
||||
'0', # z position
|
||||
'1', str(value), '1', # scale (height = data value)
|
||||
'0.2', '0.6', '1.0' # color (blue)
|
||||
])
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 9b: Audio-Reactive GLSL Fractal (Proven Recipe)
|
||||
|
||||
Audio spectrum drives a GLSL fractal shader directly via a spectrum texture input. Bass thickens inner fractal lines, mids twist rotation, highs light outer edges. **Always run discovery (SKILL.md Step 0) before using any param names from these recipes — they may differ in your TD version.**
|
||||
|
||||
```
|
||||
Audio File In CHOP → Audio Spectrum CHOP (FFT=512, outlength=256)
|
||||
→ Math CHOP (gain=10)
|
||||
→ CHOP To TOP (spectrum texture, 256x2, dataformat=r)
|
||||
↓ (input 1)
|
||||
Constant TOP (rgba32float, time) → GLSL TOP (audio-reactive shader) → Null TOP
|
||||
(input 0) ↑
|
||||
Text DAT (shader code)
|
||||
```
|
||||
|
||||
**Build via td_execute_python (complete working script):**
|
||||
|
||||
```python
|
||||
# td_execute_python script:
|
||||
td_execute_python(code="""
|
||||
import os
|
||||
root = op('/project1')
|
||||
|
||||
# Audio input
|
||||
audio = root.create(audiofileinCHOP, 'audio_in')
|
||||
audio.par.file = '/path/to/music.mp3'
|
||||
audio.par.playmode = 0 # Locked to timeline
|
||||
|
||||
# FFT analysis (output length manually set to 256 bins)
|
||||
spectrum = root.create(audiospectrumCHOP, 'spectrum')
|
||||
audio.outputConnectors[0].connect(spectrum.inputConnectors[0])
|
||||
spectrum.par.fftsize = '512'
|
||||
spectrum.par.outputmenu = 'setmanually'
|
||||
spectrum.par.outlength = 256
|
||||
|
||||
# THEN boost gain on the raw spectrum (NO Lag CHOP — see pitfall #34)
|
||||
math = root.create(mathCHOP, 'math_norm')
|
||||
spectrum.outputConnectors[0].connect(math.inputConnectors[0])
|
||||
math.par.gain = 10
|
||||
|
||||
# Spectrum → texture (256x2 image — stereo, sample at y=0.25 for first channel)
|
||||
# NOTE: choptoTOP has NO input connectors — use par.chop reference!
|
||||
spec_tex = root.create(choptoTOP, 'spectrum_tex')
|
||||
spec_tex.par.chop = math
|
||||
spec_tex.par.dataformat = 'r'
|
||||
spec_tex.par.layout = 'rowscropped'
|
||||
|
||||
# Time driver (rgba32float to avoid 0-1 clamping!)
|
||||
time_drv = root.create(constantTOP, 'time_driver')
|
||||
time_drv.par.format = 'rgba32float'
|
||||
time_drv.par.outputresolution = 'custom'
|
||||
time_drv.par.resolutionw = 1
|
||||
time_drv.par.resolutionh = 1
|
||||
time_drv.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
time_drv.par.colorg.expr = "int(absTime.seconds / 1000.0)"
|
||||
|
||||
# GLSL shader
|
||||
glsl = root.create(glslTOP, 'audio_shader')
|
||||
glsl.par.outputresolution = 'custom'
|
||||
glsl.par.resolutionw = 1280; glsl.par.resolutionh = 720
|
||||
|
||||
shader_dat = root.create(textDAT, 'shader_code')
|
||||
shader_dat.text = open('/tmp/shader.glsl').read()
|
||||
glsl.par.pixeldat = shader_dat
|
||||
|
||||
# Wire: input 0=time, input 1=spectrum
|
||||
time_drv.outputConnectors[0].connect(glsl.inputConnectors[0])
|
||||
spec_tex.outputConnectors[0].connect(glsl.inputConnectors[1])
|
||||
|
||||
# Output + audio playback
|
||||
out = root.create(nullTOP, 'output')
|
||||
glsl.outputConnectors[0].connect(out.inputConnectors[0])
|
||||
audio_out = root.create(audiodeviceoutCHOP, 'audio_out')
|
||||
audio.outputConnectors[0].connect(audio_out.inputConnectors[0])
|
||||
|
||||
result = 'network built'
|
||||
""")
|
||||
```
|
||||
|
||||
**GLSL shader (reads spectrum from input 1 texture):**
|
||||
|
||||
```glsl
|
||||
out vec4 fragColor;
|
||||
|
||||
vec3 palette(float t) {
|
||||
vec3 a = vec3(0.5); vec3 b = vec3(0.5);
|
||||
vec3 c = vec3(1.0); vec3 d = vec3(0.263, 0.416, 0.557);
|
||||
return a + b * cos(6.28318 * (c * t + d));
|
||||
}
|
||||
|
||||
void main() {
|
||||
vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
float t = td.r + td.g * 1000.0;
|
||||
|
||||
vec2 res = uTDOutputInfo.res.zw;
|
||||
vec2 uv = (gl_FragCoord.xy * 2.0 - res) / min(res.x, res.y);
|
||||
vec2 uv0 = uv;
|
||||
vec3 finalColor = vec3(0.0);
|
||||
|
||||
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.25)).r;
|
||||
float mids = texture(sTD2DInputs[1], vec2(0.25, 0.25)).r;
|
||||
float highs = texture(sTD2DInputs[1], vec2(0.65, 0.25)).r;
|
||||
|
||||
float ca = cos(t * (0.15 + mids * 0.3));
|
||||
float sa = sin(t * (0.15 + mids * 0.3));
|
||||
uv = mat2(ca, -sa, sa, ca) * uv;
|
||||
|
||||
for (float i = 0.0; i < 4.0; i++) {
|
||||
uv = fract(uv * (1.4 + bass * 0.3)) - 0.5;
|
||||
float d = length(uv) * exp(-length(uv0));
|
||||
float freq = texture(sTD2DInputs[1], vec2(clamp(d*0.5, 0.0, 1.0), 0.25)).r;
|
||||
vec3 col = palette(length(uv0) + i * 0.4 + t * 0.35);
|
||||
d = sin(d * (7.0 + bass * 4.0) + t * 1.5) / 8.0;
|
||||
d = abs(d);
|
||||
d = pow(0.012 / d, 1.2 + freq * 0.8 + bass * 0.5);
|
||||
finalColor += col * d;
|
||||
}
|
||||
|
||||
float glow = (0.03 + bass * 0.05) / (length(uv0) + 0.03);
|
||||
finalColor += vec3(0.4, 0.1, 0.7) * glow * (0.6 + 0.4 * sin(t * 2.5));
|
||||
|
||||
float ring = abs(length(uv0) - 0.4 - mids * 0.3);
|
||||
finalColor += vec3(0.1, 0.6, 0.8) * (0.005 / ring) * (0.2 + highs * 0.5);
|
||||
|
||||
finalColor *= smoothstep(0.0, 1.0, 1.0 - dot(uv0*0.55, uv0*0.55));
|
||||
finalColor = finalColor / (finalColor + vec3(1.0));
|
||||
|
||||
fragColor = TDOutputSwizzle(vec4(finalColor, 1.0));
|
||||
}
|
||||
```
|
||||
|
||||
**How spectrum sampling drives the visual:**
|
||||
- `texture(sTD2DInputs[1], vec2(x, 0.0)).r` — x position = frequency (0=bass, 1=treble)
|
||||
- Inner fractal iterations sample lower x → react to bass
|
||||
- Outer iterations sample higher x → react to treble
|
||||
- `bass * 0.3` on `fract()` scale → fractal zoom pulses with bass
|
||||
- `bass * 4.0` on sin frequency → line density pulses with bass
|
||||
- `mids * 0.3` on rotation speed → spiral twists faster during vocal/mid sections
|
||||
- `highs * 0.5` on ring opacity → high-frequency sparkle on outer ring
|
||||
|
||||
**Recording the output:** Use MovieFileOut TOP with `mjpa` codec (H.264 requires Commercial license). See pitfalls #25-27.
|
||||
|
||||
## GLSL Shaders
|
||||
|
||||
### Pattern 10: Custom Fragment Shader
|
||||
|
||||
Write a custom visual effect as a GLSL fragment shader.
|
||||
|
||||
```
|
||||
Text DAT (shader code) -> GLSL TOP -> Level TOP -> Null TOP (out)
|
||||
+ optional input TOPs for texture sampling
|
||||
```
|
||||
|
||||
**Common GLSL uniforms available in TouchDesigner:**
|
||||
|
||||
```glsl
|
||||
// Automatically provided by TD
|
||||
uniform vec4 uTDOutputInfo; // .res.zw = resolution
|
||||
|
||||
// NOTE: uTDCurrentTime does NOT exist in TD 099!
|
||||
// Feed time via a 1x1 Constant TOP (format=rgba32float):
|
||||
// t.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
// t.par.colorg.expr = "int(absTime.seconds / 1000.0)"
|
||||
// Then read in GLSL:
|
||||
// vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
// float t = td.r + td.g * 1000.0;
|
||||
|
||||
// Input textures (from connected TOP inputs)
|
||||
uniform sampler2D sTD2DInputs[1]; // array of input samplers
|
||||
|
||||
// From vertex shader
|
||||
in vec3 vUV; // UV coordinates (0-1 range)
|
||||
```
|
||||
|
||||
**Example: Plasma shader (using time from input texture)**
|
||||
|
||||
```glsl
|
||||
layout(location = 0) out vec4 fragColor;
|
||||
|
||||
void main() {
|
||||
vec2 uv = vUV.st;
|
||||
// Read time from Constant TOP input 0 (rgba32float format)
|
||||
vec4 td = texture(sTD2DInputs[0], vec2(0.5));
|
||||
float t = td.r + td.g * 1000.0;
|
||||
|
||||
float v1 = sin(uv.x * 10.0 + t);
|
||||
float v2 = sin(uv.y * 10.0 + t * 0.7);
|
||||
float v3 = sin((uv.x + uv.y) * 10.0 + t * 1.3);
|
||||
float v4 = sin(length(uv - 0.5) * 20.0 - t * 2.0);
|
||||
|
||||
float v = (v1 + v2 + v3 + v4) * 0.25;
|
||||
|
||||
vec3 color = vec3(
|
||||
sin(v * 3.14159 + 0.0) * 0.5 + 0.5,
|
||||
sin(v * 3.14159 + 2.094) * 0.5 + 0.5,
|
||||
sin(v * 3.14159 + 4.189) * 0.5 + 0.5
|
||||
);
|
||||
|
||||
fragColor = vec4(color, 1.0);
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 11: Multi-Pass GLSL (Ping-Pong)
|
||||
|
||||
For effects needing state across frames (particles, fluid, cellular automata), use GLSL Multi TOP with multiple passes or a Feedback TOP loop.
|
||||
|
||||
```
|
||||
GLSL Multi TOP (pass 0: simulation, pass 1: rendering)
|
||||
+ Text DAT (simulation shader)
|
||||
+ Text DAT (render shader)
|
||||
-> Level TOP -> Null TOP (out)
|
||||
^
|
||||
|__ Feedback TOP (feeds simulation state back)
|
||||
```
|
||||
|
||||
## Interactive Installations
|
||||
|
||||
### Pattern 12: Mouse/Touch -> Visual Response
|
||||
|
||||
```
|
||||
Mouse In CHOP -> Math CHOP (normalize to 0-1) -> [export to visual params]
|
||||
|
||||
# Or for touch/multi-touch:
|
||||
Multi Touch In DAT -> Script CHOP (parse touches) -> [export to visual params]
|
||||
```
|
||||
|
||||
```python
|
||||
# Normalize mouse position to 0-1 range
|
||||
td_execute_python: """
|
||||
op('/project1/noise1').par.offsetx.expr = "op('/project1/mouse_norm')['tx']"
|
||||
op('/project1/noise1').par.offsety.expr = "op('/project1/mouse_norm')['ty']"
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 13: OSC Control (from external software)
|
||||
|
||||
```
|
||||
OSC In CHOP (port 7000) -> Select CHOP (pick channels) -> [export to visual params]
|
||||
```
|
||||
|
||||
```
|
||||
1. td_create_operator(parent="/project1", type="oscinChop", name="osc_in")
|
||||
2. td_set_operator_pars(path="/project1/osc_in", properties={"port": 7000})
|
||||
|
||||
# OSC messages like /frequency 440 will appear as channel "frequency" with value 440
|
||||
# Export to any parameter:
|
||||
3. td_execute_python: "op('/project1/noise1').par.period.expr = \"op('/project1/osc_in')['frequency']\""
|
||||
```
|
||||
|
||||
### Pattern 14: MIDI Control (DJ/VJ)
|
||||
|
||||
```
|
||||
MIDI In CHOP (device) -> Select CHOP -> [export channels to visual params]
|
||||
```
|
||||
|
||||
Common MIDI mappings:
|
||||
- CC channels (knobs/faders): continuous 0-127, map to float params
|
||||
- Note On/Off: binary triggers, map to Trigger CHOP for envelopes
|
||||
- Velocity: intensity/brightness
|
||||
|
||||
## Live Performance
|
||||
|
||||
### Pattern 15: Multi-Source VJ Setup
|
||||
|
||||
```
|
||||
Source A (generative) ----+
|
||||
Source B (video) ---------+-- Switch/Cross TOP -- Level TOP -- Window COMP (output)
|
||||
Source C (camera) --------+
|
||||
^
|
||||
MIDI/OSC control selects active source and crossfade
|
||||
```
|
||||
|
||||
```python
|
||||
# MIDI CC1 controls which source is active (0-127 -> 0-2)
|
||||
td_execute_python: """
|
||||
op('/project1/switch1').par.index.expr = "int(op('/project1/midi_in')['cc1'] / 42)"
|
||||
"""
|
||||
|
||||
# MIDI CC2 controls crossfade between current and next
|
||||
td_execute_python: """
|
||||
op('/project1/cross1').par.cross.expr = "op('/project1/midi_in')['cc2'] / 127.0"
|
||||
"""
|
||||
```
|
||||
|
||||
### Pattern 16: Projection Mapping
|
||||
|
||||
```
|
||||
Content TOPs ----+
|
||||
|
|
||||
Stoner TOP (UV mapping) -> Composite TOP -> Window COMP (projector output)
|
||||
or
|
||||
Kantan Mapper COMP (external .tox)
|
||||
```
|
||||
|
||||
For projection mapping, the key is:
|
||||
1. Create your visual content as standard TOPs
|
||||
2. Use Stoner TOP or a third-party mapping tool to UV-map content to physical surfaces
|
||||
3. Output via Window COMP to the projector
|
||||
|
||||
### Pattern 17: Cue System
|
||||
|
||||
```
|
||||
Table DAT (cue list: cue_number, scene_name, duration, transition_type)
|
||||
|
|
||||
Script CHOP (cue state: current_cue, progress, next_cue_trigger)
|
||||
|
|
||||
[export to Switch/Cross TOPs to transition between scenes]
|
||||
```
|
||||
|
||||
```python
|
||||
td_execute_python: """
|
||||
# Simple cue system
|
||||
cue_table = op('/project1/cue_list')
|
||||
cue_state = op('/project1/cue_state')
|
||||
|
||||
def advance_cue():
|
||||
current = int(cue_state.par.value0.val)
|
||||
next_cue = min(current + 1, cue_table.numRows - 1)
|
||||
cue_state.par.value0.val = next_cue
|
||||
|
||||
scene = cue_table[next_cue, 'scene']
|
||||
duration = float(cue_table[next_cue, 'duration'])
|
||||
|
||||
# Set crossfade target and duration
|
||||
op('/project1/cross1').par.cross.val = 0
|
||||
# Animate cross to 1.0 over duration seconds
|
||||
# (use a Timer CHOP or LFO CHOP for smooth animation)
|
||||
"""
|
||||
```
|
||||
|
||||
## Networking
|
||||
|
||||
### Pattern 18: OSC Server/Client
|
||||
|
||||
```
|
||||
# Sending OSC
|
||||
OSC Out CHOP -> (network) -> external application
|
||||
|
||||
# Receiving OSC
|
||||
(network) -> OSC In CHOP -> Select CHOP -> [use values]
|
||||
```
|
||||
|
||||
### Pattern 19: NDI Video Streaming
|
||||
|
||||
```
|
||||
# Send video over network
|
||||
[any TOP chain] -> NDI Out TOP (source name)
|
||||
|
||||
# Receive video from network
|
||||
NDI In TOP (select source) -> [process as normal TOP]
|
||||
```
|
||||
|
||||
### Pattern 20: WebSocket Communication
|
||||
|
||||
```
|
||||
WebSocket DAT -> Script DAT (parse JSON messages) -> [update visuals]
|
||||
```
|
||||
|
||||
```python
|
||||
td_execute_python: """
|
||||
ws = op('/project1/websocket1')
|
||||
ws.par.address = 'ws://localhost:8080'
|
||||
ws.par.active = True
|
||||
|
||||
# In a DAT Execute callback (Script DAT watching WebSocket DAT):
|
||||
# def onTableChange(dat):
|
||||
# import json
|
||||
# msg = json.loads(dat.text)
|
||||
# op('/project1/noise1').par.seed.val = msg.get('seed', 0)
|
||||
"""
|
||||
```
|
||||
@@ -0,0 +1,106 @@
|
||||
# Operator Tips
|
||||
|
||||
## Wireframe Rendering Pattern
|
||||
|
||||
Reusable setup for wireframe geometry on black background:
|
||||
|
||||
```python
|
||||
# 1. Material
|
||||
mat = root.create(wireframeMAT, 'wire_mat')
|
||||
mat.par.colorr = 1.0; mat.par.colorg = 0.0; mat.par.colorb = 0.0
|
||||
mat.par.linewidth = 3
|
||||
|
||||
# 2. Geometry COMP
|
||||
geo = root.create(geometryCOMP, 'my_geo')
|
||||
geo.par.rx.expr = 'absTime.seconds * 30'
|
||||
geo.par.ry.expr = 'absTime.seconds * 45'
|
||||
geo.par.material = mat.path # NOTE: 'material' not 'mat'
|
||||
|
||||
# 3. Shape inside the geo
|
||||
box = geo.create(boxSOP, 'cube')
|
||||
box.par.sizex = 1.5; box.par.sizey = 1.5; box.par.sizez = 1.5
|
||||
|
||||
# 4. Camera
|
||||
cam = root.create(cameraCOMP, 'cam1')
|
||||
cam.par.tx = 0; cam.par.ty = 0; cam.par.tz = 4; cam.par.fov = 45
|
||||
|
||||
# 5. Render TOP
|
||||
render = root.create(renderTOP, 'render1')
|
||||
render.par.outputresolution = 'custom'
|
||||
render.par.resolutionw = 1280; render.par.resolutionh = 720
|
||||
render.par.bgcolorr = 0; render.par.bgcolorg = 0; render.par.bgcolorb = 0
|
||||
render.par.camera = cam.path
|
||||
render.par.geometry = geo.path
|
||||
|
||||
# 6. Output null
|
||||
out = root.create(nullTOP, 'out1')
|
||||
out.inputConnectors[0].connect(render.outputConnectors[0])
|
||||
```
|
||||
|
||||
**Key rules:**
|
||||
- Class names: `wireframeMAT` not `wireframeMat` (all-caps suffix)
|
||||
- Geometry SOPs/POPs go INSIDE the geo comp
|
||||
- Material: `geo.par.material` not `geo.par.mat`
|
||||
- Render geometry: `render.par.geometry = geo.path` (string path)
|
||||
- `wireframeMAT.par.wireframemode = 'topology'` for clean wireframe (vs `'tesselated'` for triangle edges)
|
||||
- Alternative: Use `renderTOP.par.overridemat` instead of per-geo material
|
||||
|
||||
## Feedback TOP
|
||||
|
||||
### Basic Structure
|
||||
|
||||
```
|
||||
input (initial state) ──┐
|
||||
├──→ feedback_top ──→ processing ──→ null_out
|
||||
│ ↑
|
||||
└── par.top = 'null_out' ────────────────┘
|
||||
```
|
||||
|
||||
### Setup Pattern
|
||||
|
||||
```python
|
||||
# 1. Processing chain
|
||||
glsl = root.create(glslTOP, 'sim')
|
||||
null_out = root.create(nullTOP, 'null_out')
|
||||
glsl.outputConnectors[0].connect(null_out.inputConnectors[0])
|
||||
|
||||
# 2. Feedback referencing null_out
|
||||
feedback = root.create(feedbackTOP, 'feedback')
|
||||
feedback.par.top = 'null_out'
|
||||
|
||||
# 3. Black initial state
|
||||
const_init = root.create(constantTOP, 'const_init')
|
||||
const_init.par.colorr = 0; const_init.par.colorg = 0; const_init.par.colorb = 0
|
||||
|
||||
# 4. Wire: initial → feedback, feedback → processing
|
||||
feedback.inputConnectors[0].connect(const_init)
|
||||
glsl.inputConnectors[0].connect(feedback)
|
||||
|
||||
# 5. Reset to apply initial state
|
||||
feedback.par.resetpulse.pulse()
|
||||
```
|
||||
|
||||
### Common Errors
|
||||
|
||||
| Error | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| "Not enough sources specified" | No input connected | Connect initial state TOP |
|
||||
| Unexpected initial pattern | Wrong initial state | Use Constant TOP (black) |
|
||||
|
||||
### Tips
|
||||
|
||||
1. Use float format for simulations: `glsl.par.format = 'rgba32float'`
|
||||
2. Reset after setup: `feedback.par.resetpulse.pulse()`
|
||||
3. Match resolutions — feedback, processing, and initial state must match
|
||||
4. Soft boundary prevents edge artifacts:
|
||||
```glsl
|
||||
float edge = 3.0 * texel.x;
|
||||
float bx = smoothstep(0.0, edge, uv.x) * smoothstep(0.0, edge, 1.0 - uv.x);
|
||||
float by = smoothstep(0.0, edge, uv.y) * smoothstep(0.0, edge, 1.0 - uv.y);
|
||||
value *= bx * by;
|
||||
```
|
||||
|
||||
### Use Cases
|
||||
- **Wave Simulation** — R=height, G=velocity, black initial state
|
||||
- **Cellular Automata** — white=alive, black=dead, random noise initial state
|
||||
- **Trail / Motion Blur** — blend current frame with feedback, black initial
|
||||
@@ -0,0 +1,239 @@
|
||||
# TouchDesigner Operator Reference
|
||||
|
||||
## Operator Families Overview
|
||||
|
||||
TouchDesigner has 6 operator families. Each family processes a specific data type and is color-coded in the UI. Operators can only connect to others of the SAME family (with cross-family converters as the bridge).
|
||||
|
||||
## TOPs — Texture Operators (Purple)
|
||||
|
||||
2D image/texture processing on the GPU. The workhorse of visual output.
|
||||
|
||||
### Generators (create images from nothing)
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Noise TOP | `noiseTop` | `type` (0-6), `monochrome`, `seed`, `period`, `harmonics`, `exponent`, `amp`, `offset`, `resolutionw/h` | Procedural noise textures — Perlin, Simplex, Sparse, etc. Foundation of generative art. |
|
||||
| Constant TOP | `constantTop` | `colorr/g/b/a`, `resolutionw/h` | Solid color. Use as background or blend input. |
|
||||
| Text TOP | `textTop` | `text`, `fontsizex`, `fontfile`, `alignx/y`, `colorr/g/b` | Render text to texture. Supports multi-line, word wrap. |
|
||||
| Ramp TOP | `rampTop` | `type` (0=horizontal, 1=vertical, 2=radial, 3=circular), `phase`, `period` | Gradient textures for masking, color mapping. |
|
||||
| Circle TOP | `circleTop` | `radiusx/y`, `centerx/y`, `width` | Circles, rings, ellipses. |
|
||||
| Rectangle TOP | `rectangleTop` | `sizex/y`, `centerx/y`, `softness` | Rectangles with optional softness. |
|
||||
| GLSL TOP | `glslTop` | `dat` (points to shader DAT), `resolutionw/h`, `outputformat`, custom uniforms | Custom fragment shaders. Most powerful TOP for custom visuals. |
|
||||
| GLSL Multi TOP | `glslmultiTop` | `dat`, `numinputs`, `numoutputs`, `numcomputepasses` | Multi-pass GLSL with compute shaders. Advanced. |
|
||||
| Render TOP | `renderTop` | `camera`, `geometry`, `lights`, `resolutionw/h` | Renders 3D scenes (SOPs + MATs + Camera/Light COMPs). |
|
||||
|
||||
### Filters (modify a single input)
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Level TOP | `levelTop` | `opacity`, `brightness1/2`, `gamma1/2`, `contrast`, `invert`, `blacklevel/whitelevel` | Brightness, contrast, gamma, levels. Essential color correction. |
|
||||
| Blur TOP | `blurTop` | `sizex/y`, `type` (0=Gaussian, 1=Box, 2=Bartlett) | Gaussian/box blur. |
|
||||
| Transform TOP | `transformTop` | `tx/ty`, `sx/sy`, `rz`, `pivotx/y`, `extend` (0=Hold, 1=Zero, 2=Repeat, 3=Mirror) | Translate, scale, rotate textures. |
|
||||
| HSV Adjust TOP | `hsvadjustTop` | `hueoffset`, `saturationmult`, `valuemult` | HSV color adjustments. |
|
||||
| Lookup TOP | `lookupTop` | (input: texture + lookup table) | Color remapping via lookup table texture. |
|
||||
| Edge TOP | `edgeTop` | `type` (0=Sobel, 1=Frei-Chen) | Edge detection. |
|
||||
| Displace TOP | `displaceTop` | `scalex/y` | Pixel displacement using a second input as displacement map. |
|
||||
| Flip TOP | `flipTop` | `flipx`, `flipy`, `flop` (diagonal) | Mirror/flip textures. |
|
||||
| Crop TOP | `cropTop` | `cropleft/right/top/bottom` | Crop region of texture. |
|
||||
| Resolution TOP | `resolutionTop` | `resolutionw/h`, `outputresolution` | Resize textures. |
|
||||
| Null TOP | `nullTop` | (none significant) | Pass-through. Use for organization, referencing, feedback delay. |
|
||||
| Cache TOP | `cacheTop` | `length`, `step` | Store N frames of history. Useful for trails, time effects. |
|
||||
|
||||
### Compositors (combine multiple inputs)
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Composite TOP | `compositeTop` | `operand` (0-31: Over, Add, Multiply, Screen, etc.) | Blend two textures with standard compositing modes. |
|
||||
| Over TOP | `overTop` | (simple alpha compositing) | Layer with alpha. Simpler than Composite. |
|
||||
| Add TOP | `addTop` | (additive blend) | Additive blending. Great for glow, light effects. |
|
||||
| Multiply TOP | `multiplyTop` | (multiplicative blend) | Multiply blend. Good for masking, darkening. |
|
||||
| Switch TOP | `switchTop` | `index` (0-based) | Switch between multiple inputs by index. |
|
||||
| Cross TOP | `crossTop` | `cross` (0.0-1.0) | Crossfade between two inputs. |
|
||||
|
||||
### I/O (input/output)
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Movie File In TOP | `moviefileinTop` | `file`, `speed`, `trim`, `index` | Load video files, image sequences. |
|
||||
| Movie File Out TOP | `moviefileoutTop` | `file`, `type` (codec), `record` (toggle) | Record/export video files. |
|
||||
| NDI In TOP | `ndiinTop` | `sourcename` | Receive NDI video streams. |
|
||||
| NDI Out TOP | `ndioutTop` | `sourcename` | Send NDI video streams. |
|
||||
| Syphon Spout In/Out TOP | `syphonspoutinTop` / `syphonspoutoutTop` | `servername` | Inter-app texture sharing. |
|
||||
| Video Device In TOP | `videodeviceinTop` | `device` | Webcam/capture card input. |
|
||||
| Feedback TOP | `feedbackTop` | `top` (path to the TOP to feed back) | One-frame delay feedback. Essential for recursive effects. |
|
||||
|
||||
### Converters
|
||||
|
||||
| Operator | Type Name | Direction | Use |
|
||||
|----------|-----------|-----------|-----|
|
||||
| CHOP to TOP | `choptopTop` | CHOP -> TOP | Visualize channel data as texture (waveform, spectrum display). |
|
||||
| TOP to CHOP | `topchopChop` | TOP -> CHOP | Sample texture pixels as channel data. |
|
||||
|
||||
## CHOPs — Channel Operators (Green)
|
||||
|
||||
Time-varying numeric data: audio, animation curves, sensor data, control signals.
|
||||
|
||||
### Generators
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Constant CHOP | `constantChop` | `name0/value0`, `name1/value1`... | Static named channels. Control panel for parameters. |
|
||||
| LFO CHOP | `lfoChop` | `frequency`, `type` (0=Sin, 1=Tri, 2=Square, 3=Ramp, 4=Pulse), `amp`, `offset`, `phase` | Low frequency oscillator. Animation driver. |
|
||||
| Noise CHOP | `noiseChop` | `type`, `roughness`, `period`, `amp`, `seed`, `channels` | Smooth random motion. Organic animation. |
|
||||
| Pattern CHOP | `patternChop` | `type` (0=Sine, 1=Triangle, ...), `length`, `cycles` | Generate waveform patterns. |
|
||||
| Timer CHOP | `timerChop` | `length`, `play`, `cue`, `cycles` | Countdown/count-up timer with cue points. |
|
||||
| Count CHOP | `countChop` | `threshold`, `limittype`, `limitmin/max` | Event counter with wrapping/clamping. |
|
||||
|
||||
### Audio
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Audio File In CHOP | `audiofileinChop` | `file`, `volume`, `play`, `speed`, `trim` | Play audio files. |
|
||||
| Audio Device In CHOP | `audiodeviceinChop` | `device`, `channels` | Live microphone/line input. |
|
||||
| Audio Spectrum CHOP | `audiospectrumChop` | `size` (FFT size), `outputformat` (0=Power, 1=Magnitude) | FFT frequency analysis. |
|
||||
| Audio Band EQ CHOP | `audiobandeqChop` | `bands`, `gaindb` per band | Frequency band isolation. |
|
||||
| Audio Device Out CHOP | `audiodeviceoutChop` | `device` | Audio playback output. |
|
||||
|
||||
### Math/Logic
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Math CHOP | `mathChop` | `preoff`, `gain`, `postoff`, `chanop` (0=Off, 1=Add, 2=Subtract, 3=Multiply...) | Math operations on channels. The Swiss army knife. |
|
||||
| Logic CHOP | `logicChop` | `preop` (0=Off, 1=AND, 2=OR, 3=XOR, 4=NAND), `convert` | Boolean logic on channels. |
|
||||
| Filter CHOP | `filterChop` | `type` (0=Low Pass, 1=Band Pass, 2=High Pass, 3=Notch), `cutofffreq`, `filterwidth` | Smooth, dampen, filter signals. |
|
||||
| Lag CHOP | `lagChop` | `lag1/2`, `overshoot1/2` | Smooth transitions with overshoot. |
|
||||
| Limit CHOP | `limitChop` | `type` (0=Clamp, 1=Loop, 2=ZigZag), `min/max` | Clamp or wrap channel values. |
|
||||
| Speed CHOP | `speedChop` | (none significant) | Integrate values (velocity to position, acceleration to velocity). |
|
||||
| Trigger CHOP | `triggerChop` | `attack`, `peak`, `decay`, `sustain`, `release` | ADSR envelope from trigger events. |
|
||||
| Select CHOP | `selectChop` | `chop` (path), `channames` | Reference channels from another CHOP. |
|
||||
| Merge CHOP | `mergeChop` | `align` (0=Extend, 1=Trim to First, 2=Trim to Shortest) | Combine channels from multiple CHOPs. |
|
||||
| Null CHOP | `nullChop` | (none significant) | Pass-through for organization and referencing. |
|
||||
|
||||
### Input Devices
|
||||
|
||||
| Operator | Type Name | Use |
|
||||
|----------|-----------|-----|
|
||||
| Mouse In CHOP | `mouseinChop` | Mouse position, buttons, wheel. |
|
||||
| Keyboard In CHOP | `keyboardinChop` | Keyboard key states. |
|
||||
| MIDI In CHOP | `midiinChop` | MIDI note/CC input. |
|
||||
| OSC In CHOP | `oscinChop` | OSC message input (network). |
|
||||
|
||||
## SOPs — Surface Operators (Blue)
|
||||
|
||||
3D geometry: points, polygons, NURBS, meshes.
|
||||
|
||||
### Generators
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Grid SOP | `gridSop` | `rows`, `cols`, `sizex/y`, `type` (0=Polygon, 1=Mesh, 2=NURBS) | Flat grid mesh. Foundation for displacement, instancing. |
|
||||
| Sphere SOP | `sphereSop` | `type`, `rows`, `cols`, `radius` | Sphere geometry. |
|
||||
| Box SOP | `boxSop` | `sizex/y/z` | Box geometry. |
|
||||
| Torus SOP | `torusSop` | `radiusx/y`, `rows`, `cols` | Donut shape. |
|
||||
| Circle SOP | `circleSop` | `type`, `radius`, `divs` | Circle/ring geometry. |
|
||||
| Line SOP | `lineSop` | `dist`, `points` | Line segments. |
|
||||
| Text SOP | `textSop` | `text`, `fontsizex`, `fontfile`, `extrude` | 3D text geometry. |
|
||||
|
||||
### Modifiers
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Transform SOP | `transformSop` | `tx/ty/tz`, `rx/ry/rz`, `sx/sy/sz` | Transform geometry (translate, rotate, scale). |
|
||||
| Noise SOP | `noiseSop` | `type`, `amp`, `period`, `roughness` | Deform geometry with noise. |
|
||||
| Sort SOP | `sortSop` | `ptsort`, `primsort` | Reorder points/primitives. |
|
||||
| Facet SOP | `facetSop` | `unique`, `consolidate`, `computenormals` | Normals, consolidation, unique points. |
|
||||
| Merge SOP | `mergeSop` | (none significant) | Combine multiple geometry inputs. |
|
||||
| Null SOP | `nullSop` | (none significant) | Pass-through. |
|
||||
|
||||
## DATs — Data Operators (White)
|
||||
|
||||
Text, tables, scripts, network data.
|
||||
|
||||
### Core
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Table DAT | `tableDat` | (edit content directly) | Spreadsheet-like data tables. |
|
||||
| Text DAT | `textDat` | (edit content directly) | Arbitrary text content. Shader code, configs, scripts. |
|
||||
| Script DAT | `scriptDat` | `language` (0=Python, 1=C++) | Custom callbacks and DAT processing. |
|
||||
| CHOP Execute DAT | `chopexecDat` | `chop` (path to watch), callbacks | Trigger Python on CHOP value changes. |
|
||||
| DAT Execute DAT | `datexecDat` | `dat` (path to watch) | Trigger Python on DAT content changes. |
|
||||
| Panel Execute DAT | `panelexecDat` | `panel` | Trigger Python on UI panel events. |
|
||||
|
||||
### I/O
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Web DAT | `webDat` | `url`, `fetchmethod` (0=GET, 1=POST) | HTTP requests. API integration. |
|
||||
| TCP/IP DAT | `tcpipDat` | `address`, `port`, `mode` | TCP networking. |
|
||||
| OSC In DAT | `oscinDat` | `port` | Receive OSC as text messages. |
|
||||
| Serial DAT | `serialDat` | `port`, `baudrate` | Serial port communication (Arduino, etc.). |
|
||||
| File In DAT | `fileinDat` | `file` | Read text files. |
|
||||
| File Out DAT | `fileoutDat` | `file`, `write` | Write text files. |
|
||||
|
||||
### Conversions
|
||||
|
||||
| Operator | Type Name | Direction | Use |
|
||||
|----------|-----------|-----------|-----|
|
||||
| DAT to CHOP | `dattochopChop` | DAT -> CHOP | Convert table data to channels. |
|
||||
| CHOP to DAT | `choptodatDat` | CHOP -> DAT | Convert channel data to table rows. |
|
||||
| SOP to DAT | `soptodatDat` | SOP -> DAT | Extract geometry data as table. |
|
||||
|
||||
## MATs — Material Operators (Yellow)
|
||||
|
||||
Materials for 3D rendering in Render TOP / Geometry COMP.
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Phong MAT | `phongMat` | `diff_colorr/g/b`, `spec_colorr/g/b`, `shininess`, `colormap`, `normalmap` | Classic Phong shading. Simple, fast. |
|
||||
| PBR MAT | `pbrMat` | `basecolorr/g/b`, `metallic`, `roughness`, `normalmap`, `emitcolorr/g/b` | Physically-based rendering. Realistic materials. |
|
||||
| GLSL MAT | `glslMat` | `dat` (shader DAT), custom uniforms | Custom vertex + fragment shaders for 3D. |
|
||||
| Constant MAT | `constMat` | `colorr/g/b`, `colormap` | Flat unlit color/texture. No shading. |
|
||||
| Point Sprite MAT | `pointspriteMat` | `colormap`, `scale` | Render points as camera-facing sprites. Great for particles. |
|
||||
| Wireframe MAT | `wireframeMat` | `colorr/g/b`, `width` | Wireframe rendering. |
|
||||
| Depth MAT | `depthMat` | `near`, `far` | Render depth buffer as grayscale. |
|
||||
|
||||
## COMPs — Component Operators (Gray)
|
||||
|
||||
Containers, 3D scene elements, UI components.
|
||||
|
||||
### 3D Scene
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Geometry COMP | `geometryComp` | `material` (path), `instancechop` (path), `instancing` (toggle) | Renders geometry with material. Instancing host. |
|
||||
| Camera COMP | `cameraComp` | `tx/ty/tz`, `rx/ry/rz`, `fov`, `near/far` | Camera for Render TOP. |
|
||||
| Light COMP | `lightComp` | `lighttype` (0=Point, 1=Directional, 2=Spot, 3=Cone), `dimmer`, `colorr/g/b` | Lighting for 3D scenes. |
|
||||
| Ambient Light COMP | `ambientlightComp` | `dimmer`, `colorr/g/b` | Ambient lighting. |
|
||||
| Environment Light COMP | `envlightComp` | `envmap` | Image-based lighting (IBL). |
|
||||
|
||||
### Containers
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Container COMP | `containerComp` | `w`, `h`, `bgcolor1/2/3` | UI container. Holds other COMPs for panel layouts. |
|
||||
| Base COMP | `baseComp` | (none significant) | Generic container. Networks-inside-networks. |
|
||||
| Replicator COMP | `replicatorComp` | `template`, `operatorsdat` | Clone a template operator N times from a table. |
|
||||
|
||||
### Utilities
|
||||
|
||||
| Operator | Type Name | Key Parameters | Use |
|
||||
|----------|-----------|---------------|-----|
|
||||
| Window COMP | `windowComp` | `winw/h`, `winoffsetx/y`, `monitor`, `borders` | Output window for display/projection. |
|
||||
| Select COMP | `selectComp` | `rowcol`, `panel` | Select and display content from elsewhere. |
|
||||
| Engine COMP | `engineComp` | `tox`, `externaltox` | Load external .tox components. Sub-process isolation. |
|
||||
|
||||
## Cross-Family Converter Summary
|
||||
|
||||
| From | To | Operator | Type Name |
|
||||
|------|-----|----------|-----------|
|
||||
| CHOP | TOP | CHOP to TOP | `choptopTop` |
|
||||
| TOP | CHOP | TOP to CHOP | `topchopChop` |
|
||||
| DAT | CHOP | DAT to CHOP | `dattochopChop` |
|
||||
| CHOP | DAT | CHOP to DAT | `choptodatDat` |
|
||||
| SOP | CHOP | SOP to CHOP | `soptochopChop` |
|
||||
| CHOP | SOP | CHOP to SOP | `choptosopSop` |
|
||||
| SOP | DAT | SOP to DAT | `soptodatDat` |
|
||||
| DAT | SOP | DAT to SOP | `dattosopSop` |
|
||||
| SOP | TOP | (use Render TOP + Geometry COMP) | — |
|
||||
| TOP | SOP | TOP to SOP | `toptosopSop` |
|
||||
@@ -0,0 +1,281 @@
|
||||
# Panel & UI Reference
|
||||
|
||||
Interactive control surfaces inside TouchDesigner — buttons, sliders, fields, custom parameter pages, panel callbacks. For HUD overlays (rendered text on visuals) see `layout-compositor.md`.
|
||||
|
||||
Use cases:
|
||||
- VJ control rack (master fader, scene buttons, FX toggles)
|
||||
- Installation operator console
|
||||
- Self-contained TOX components with their own parameter UIs
|
||||
- Phone-style touch interfaces displayed on a tablet
|
||||
|
||||
---
|
||||
|
||||
## Two Layers of UI
|
||||
|
||||
| Layer | What it is | Use for |
|
||||
|---|---|---|
|
||||
| **Custom Parameters** | Params on any COMP, edited like built-in TD params | Configurable components, presets, "settings" panels |
|
||||
| **Panel COMPs** | Visible widgets (button, slider, field) inside a containerCOMP | Interactive control surfaces, real-time UIs |
|
||||
|
||||
Combine both: build a containerCOMP with panel widgets that read/write custom parameters on a parent component.
|
||||
|
||||
---
|
||||
|
||||
## Custom Parameters
|
||||
|
||||
Add user-editable params to any COMP. Params persist with the COMP, drive expressions, and survive save/reload.
|
||||
|
||||
```python
|
||||
# Add a custom page to a baseCOMP
|
||||
comp = op('/project1/my_component')
|
||||
page = comp.appendCustomPage('Controls')
|
||||
|
||||
# Add typed params
|
||||
page.appendFloat('Intensity', label='Intensity')[0] # returns a Par
|
||||
page.appendInt('Count', label='Count')[0]
|
||||
page.appendToggle('Enabled', label='Enabled')[0]
|
||||
page.appendMenu('Mode', menuNames=['off', 'soft', 'hard'], menuLabels=['Off', 'Soft', 'Hard'])[0]
|
||||
page.appendStr('Title', label='Title')[0]
|
||||
page.appendRGB('Color', label='Color') # returns 3 pars
|
||||
page.appendXY('Offset', label='Offset') # returns 2 pars
|
||||
page.appendPulse('Reset', label='Reset')[0]
|
||||
page.appendFile('TextureFile', label='Texture')[0]
|
||||
```
|
||||
|
||||
**Read/write from anywhere:**
|
||||
|
||||
```python
|
||||
val = op('/project1/my_component').par.Intensity.eval()
|
||||
op('/project1/my_component').par.Intensity = 0.7
|
||||
```
|
||||
|
||||
**Drive other params via expression:**
|
||||
|
||||
```python
|
||||
op('bloom1').par.threshold.mode = ParMode.EXPRESSION
|
||||
op('bloom1').par.threshold.expr = "op('/project1/my_component').par.Intensity"
|
||||
```
|
||||
|
||||
**Pulse handler (Reset button):**
|
||||
|
||||
Use a `parameterExecuteDAT` watching the COMP's pulse params. See `dat-scripting.md`.
|
||||
|
||||
---
|
||||
|
||||
## Panel COMPs — The Widgets
|
||||
|
||||
Each is a COMP that renders as a clickable/draggable widget inside a `containerCOMP`.
|
||||
|
||||
| Type | Type Name | Use |
|
||||
|---|---|---|
|
||||
| Button | `buttonCOMP` | Click action — momentary or toggle |
|
||||
| Slider | `sliderCOMP` | Drag to set 0-1 value (1D or 2D) |
|
||||
| Field | `fieldCOMP` | Text input |
|
||||
| Container | `containerCOMP` | Layout + visual styling, holds children |
|
||||
| Select | `selectCOMP` | Reference and display content from another COMP |
|
||||
| List | `listCOMP` | Scrollable list with row callbacks |
|
||||
|
||||
### Button
|
||||
|
||||
```python
|
||||
btn = root.create(buttonCOMP, 'play_btn')
|
||||
btn.par.w = 120; btn.par.h = 40
|
||||
btn.par.buttontype = 'momentary' # 'momentary' | 'toggleup' | 'togglepress' | 'radio'
|
||||
btn.par.bgcolorr = 0.1; btn.par.bgcolorg = 0.1; btn.par.bgcolorb = 0.1
|
||||
btn.par.text = 'Play'
|
||||
|
||||
# Read state
|
||||
state = btn.panel.state # 1 when active
|
||||
```
|
||||
|
||||
### Slider
|
||||
|
||||
```python
|
||||
sld = root.create(sliderCOMP, 'master_fader')
|
||||
sld.par.w = 60; sld.par.h = 300
|
||||
sld.par.style = 'vertical' # 'vertical' | 'horizontal' | 'xy'
|
||||
sld.par.value0min = 0.0
|
||||
sld.par.value0max = 1.0
|
||||
|
||||
# Drive a parameter via expression (always-on, no callback needed)
|
||||
op('/project1/master_level').par.opacity.mode = ParMode.EXPRESSION
|
||||
op('/project1/master_level').par.opacity.expr = "op('master_fader').panel.u"
|
||||
```
|
||||
|
||||
`panel.u` and `panel.v` give the 0-1 normalized values. For 2D sliders both are populated.
|
||||
|
||||
### Field (Text Input)
|
||||
|
||||
```python
|
||||
fld = root.create(fieldCOMP, 'scene_name')
|
||||
fld.par.w = 200; fld.par.h = 30
|
||||
fld.par.fieldtype = 'string' # 'string' | 'integer' | 'float'
|
||||
|
||||
# Read current text
|
||||
text = fld.panel.field # the text content
|
||||
```
|
||||
|
||||
### List
|
||||
|
||||
For scrollable lists with selectable rows, use the docked `list1_callbacks` DAT to handle row interactions. Set up cells via the `list_definition` table DAT.
|
||||
|
||||
---
|
||||
|
||||
## Container COMP — Layout & Styling
|
||||
|
||||
`containerCOMP` is the primary parent for grouping widgets and arranging layouts.
|
||||
|
||||
```python
|
||||
panel = root.create(containerCOMP, 'control_panel')
|
||||
panel.par.w = 400; panel.par.h = 600
|
||||
panel.par.bgcolorr = 0.05
|
||||
panel.par.bgcolorg = 0.05
|
||||
panel.par.bgcolorb = 0.05
|
||||
panel.par.bgalpha = 1.0
|
||||
|
||||
# Layout child panels in vertical stack
|
||||
panel.par.align = 'lefttoright' # 'lefttoright' | 'toptobottom' | etc.
|
||||
```
|
||||
|
||||
Children are positioned automatically based on `par.align`. For absolute positioning use `par.align = 'fillresize'` and set each child's `par.x` / `par.y`.
|
||||
|
||||
### Layout Strategies
|
||||
|
||||
| `par.align` | Behavior |
|
||||
|---|---|
|
||||
| `lefttoright` | Children stacked horizontally |
|
||||
| `toptobottom` | Children stacked vertically |
|
||||
| `righttoleft` / `bottomtotop` | Reversed stacks |
|
||||
| `fillresize` | Children sized to fill, manual positioning |
|
||||
| `top` / `bottom` / `left` / `right` | Fixed positioning |
|
||||
|
||||
For complex grids: nest containers — vertical container holding horizontal containers.
|
||||
|
||||
---
|
||||
|
||||
## Panel Callbacks — Reacting to Events
|
||||
|
||||
`panelExecuteDAT` watches a panel and fires Python callbacks on user interaction.
|
||||
|
||||
```python
|
||||
pe = root.create(panelExecuteDAT, 'btn_handler')
|
||||
pe.par.panel = '/project1/play_btn'
|
||||
pe.par.click = True # respond to clicks
|
||||
pe.par.value = True # respond to value changes
|
||||
```
|
||||
|
||||
In its docked DAT:
|
||||
|
||||
```python
|
||||
def onOffToOn(panelValue):
|
||||
# Click pressed
|
||||
op('/project1/scene_timer').par.start.pulse()
|
||||
return
|
||||
|
||||
def onOnToOff(panelValue):
|
||||
# Click released
|
||||
return
|
||||
|
||||
def onValueChange(panelValue):
|
||||
# Slider drag, field change, etc.
|
||||
new_val = panelValue.eval()
|
||||
op('/project1/master').par.opacity = new_val
|
||||
return
|
||||
```
|
||||
|
||||
For pulse params on custom-parameter pages, use a `parameterExecuteDAT` instead.
|
||||
|
||||
---
|
||||
|
||||
## Building a Complete VJ Control Panel
|
||||
|
||||
End-to-end pattern:
|
||||
|
||||
```python
|
||||
# 1. Top-level container
|
||||
panel = root.create(containerCOMP, 'vj_control')
|
||||
panel.par.w = 800; panel.par.h = 200
|
||||
panel.par.align = 'lefttoright'
|
||||
|
||||
# 2. Master fader column
|
||||
master_col = panel.create(containerCOMP, 'master')
|
||||
master_col.par.w = 120; master_col.par.h = 200
|
||||
master_col.par.align = 'toptobottom'
|
||||
|
||||
master_label = master_col.create(textTOP, 'lbl')
|
||||
master_label.par.text = 'MASTER'
|
||||
|
||||
master_sld = master_col.create(sliderCOMP, 'fader')
|
||||
master_sld.par.w = 60; master_sld.par.h = 150
|
||||
master_sld.par.style = 'vertical'
|
||||
|
||||
# 3. Scene buttons row
|
||||
scene_col = panel.create(containerCOMP, 'scenes')
|
||||
scene_col.par.w = 400; scene_col.par.h = 200
|
||||
scene_col.par.align = 'lefttoright'
|
||||
for i in range(8):
|
||||
b = scene_col.create(buttonCOMP, f'scene_{i+1}')
|
||||
b.par.w = 50; b.par.h = 50
|
||||
b.par.text = str(i+1)
|
||||
b.par.buttontype = 'radio' # only one active at a time
|
||||
|
||||
# 4. FX toggle column
|
||||
fx_col = panel.create(containerCOMP, 'fx')
|
||||
fx_col.par.w = 280; fx_col.par.h = 200
|
||||
fx_col.par.align = 'toptobottom'
|
||||
for fx in ['Bloom', 'CRT', 'Glitch', 'Strobe']:
|
||||
t = fx_col.create(buttonCOMP, fx.lower())
|
||||
t.par.w = 220; t.par.h = 35
|
||||
t.par.text = fx
|
||||
t.par.buttontype = 'toggleup'
|
||||
|
||||
# 5. Display in a window
|
||||
win = root.create(windowCOMP, 'control_win')
|
||||
win.par.winop = panel.path
|
||||
win.par.winw = 800; win.par.winh = 200
|
||||
win.par.borders = True
|
||||
win.par.winopen.pulse()
|
||||
```
|
||||
|
||||
Then wire panel values to ops via expressions or panelExecuteDATs.
|
||||
|
||||
---
|
||||
|
||||
## Showing the Panel — Window or Embedded
|
||||
|
||||
| Approach | When |
|
||||
|---|---|
|
||||
| `windowCOMP` pointing at panel | Standalone control surface, separate display |
|
||||
| Render the containerCOMP via `renderTOP` | Composite UI over visuals (HUD-style) |
|
||||
| Use a `panelCOMP` directly inside a network editor pane | Designer/dev preview only — panel is fully interactive |
|
||||
|
||||
For a touch-screen tablet, use a `windowCOMP` on a second display routed to the tablet's HDMI input.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **Panel won't respond to clicks** — likely `par.disabled = True` or the parent container has `par.disableinputs = True`. Check the panel hierarchy.
|
||||
2. **Slider value not updating** — `panel.u/v` reads the visual position. If you set `par.value0` directly, the visual lags. Use `par.value0` AS the source of truth and let the slider follow.
|
||||
3. **Custom param won't appear** — must call `appendCustomPage` first, then append params. Pages with no params don't show.
|
||||
4. **Custom param disappears on reload** — params added via Python at runtime persist only if the COMP is saved AFTER. Use a `tox` save (`comp.save('mycomp.tox')`) or commit via `td_execute_python` then save the project.
|
||||
5. **Event callback fires twice** — both `onOffToOn` and `onValueChange` may fire on a single button press. Pick one to handle the action; don't double-trigger.
|
||||
6. **Pulse params need `.pulse()`** — setting `par.X = True` on a pulse param does nothing. Always use `.pulse()`.
|
||||
7. **Field text doesn't commit until Tab/Enter** — fields don't fire callbacks while typing. Use `par.committemode = 'all'` to fire on every keystroke (heavy).
|
||||
8. **`par.text` vs panel content** — `buttonCOMP.par.text` is the LABEL on the button. The button's STATE is `panel.state` (0/1). Don't confuse them.
|
||||
9. **Touch input on macOS** — multi-touch via direct touch panels works but TD's gesture handling is rudimentary. For complex multi-touch (pinch/rotate), use TouchOSC on a tablet instead.
|
||||
10. **Layout doesn't update** — changing `par.align` requires the container to re-cook. Touch a child or pulse the container to trigger.
|
||||
|
||||
---
|
||||
|
||||
## Quick Recipes
|
||||
|
||||
| Goal | Setup |
|
||||
|---|---|
|
||||
| Master fader | `sliderCOMP` (vertical) → expression on `level.par.opacity` |
|
||||
| Scene picker | 8 `buttonCOMP` (radio) → `selectCHOP` on their state → drive `switchTOP.par.index` |
|
||||
| FX toggle | `buttonCOMP` (toggleup) → expression on `bypass` of an FX op |
|
||||
| Numeric input | `fieldCOMP` (float) → expression on target par |
|
||||
| Component settings | Custom params on the component COMP, panel widgets inside drive them |
|
||||
| Touch tablet UI | `containerCOMP` with widgets → `windowCOMP` to second display |
|
||||
| Status display | `textTOP` rendered into the panel via `selectCOMP` |
|
||||
@@ -0,0 +1,245 @@
|
||||
# Particles Reference
|
||||
|
||||
Particle systems in TouchDesigner — modern POPs (Particle Operators) and the legacy particleSOP path.
|
||||
|
||||
For instancing static geometry (without per-instance lifetime/velocity), see `geometry-comp.md`. For GLSL-driven feedback simulations (no particle abstraction), see `operator-tips.md` (Feedback TOP section).
|
||||
|
||||
Always call `td_get_par_info` for the op type before setting params. Param names below reflect TD 2025.32 — verify before relying on them.
|
||||
|
||||
---
|
||||
|
||||
## Two Paths: POPs vs. SOPs
|
||||
|
||||
| | **POP family** (modern) | **particleSOP** (legacy) |
|
||||
|---|---|---|
|
||||
| GPU? | Yes (compute) | No (CPU) |
|
||||
| Particle count | 100k+ comfortably | ~5k before slowdown |
|
||||
| API style | Source / Force / Solver / Render chain | Single op with many params |
|
||||
| Use for | New projects, anything intensive | Quick demos, low counts, TD < 2023 |
|
||||
|
||||
**Default to POPs.** Only fall back to particleSOP if a POP variant of an op you need doesn't exist.
|
||||
|
||||
---
|
||||
|
||||
## POP Pipeline Overview
|
||||
|
||||
A POP system is a chain of operators inside a `geometryCOMP`:
|
||||
|
||||
```
|
||||
popSourceTOP / popSourceSOP ← spawn new particles
|
||||
↓
|
||||
popForceTOP (gravity, wind, etc.)
|
||||
↓
|
||||
popForceTOP (attractor, vortex, ...)
|
||||
↓
|
||||
popDeleteTOP (lifetime, bounds)
|
||||
↓
|
||||
popSolverTOP ← integrates velocity, updates positions
|
||||
↓
|
||||
[render via geometryCOMP / glslMAT instancing]
|
||||
```
|
||||
|
||||
POP buffers carry standard channels: `P` (position), `v` (velocity), `life`, `id`, `Cd` (color), plus any custom channels you add.
|
||||
|
||||
---
|
||||
|
||||
## Minimal POP Setup
|
||||
|
||||
```python
|
||||
# Create a geometry COMP to hold the POP network
|
||||
geo = root.create(geometryCOMP, 'particles_geo')
|
||||
|
||||
# 1. Source — emit particles from a point
|
||||
src = geo.create(popSourceTOP, 'src')
|
||||
src.par.birthrate = 500 # per second
|
||||
src.par.life = 4.0 # seconds
|
||||
|
||||
# 2. Gravity force
|
||||
grav = geo.create(popForceTOP, 'gravity')
|
||||
grav.par.forcetype = 'gravity'
|
||||
grav.par.fy = -9.8
|
||||
|
||||
# 3. Lifetime cleanup
|
||||
delp = geo.create(popDeleteTOP, 'cull')
|
||||
delp.par.condition = 'lifeleq' # delete when life <= 0
|
||||
delp.par.value = 0
|
||||
|
||||
# 4. Solver
|
||||
solv = geo.create(popSolverTOP, 'solver')
|
||||
solv.par.timestep = 'frame'
|
||||
|
||||
# Wire: source → force → delete → solver
|
||||
src.outputConnectors[0].connect(grav.inputConnectors[0])
|
||||
grav.outputConnectors[0].connect(delp.inputConnectors[0])
|
||||
delp.outputConnectors[0].connect(solv.inputConnectors[0])
|
||||
```
|
||||
|
||||
The `popSolverTOP` output IS the live particle buffer. Render it via `glslMAT` instancing on a small SOP (sphere, point) as the "shape" of each particle.
|
||||
|
||||
---
|
||||
|
||||
## Common Forces
|
||||
|
||||
| Force type | Effect | Common params |
|
||||
|---|---|---|
|
||||
| `gravity` | Constant directional pull | `fx`, `fy`, `fz` |
|
||||
| `wind` | Constant velocity addition | `wx`, `wy`, `wz` |
|
||||
| `drag` | Velocity damping over time | `dragstrength` |
|
||||
| `noise` | Curl-noise turbulence | `noiseamp`, `noisefreq`, `noiseseed` |
|
||||
| `attractor` | Pull toward a point | `position`, `strength`, `falloff` |
|
||||
| `vortex` | Swirl around an axis | `axis`, `strength` |
|
||||
| `point` (custom) | GLSL-evaluated arbitrary force | via `popforceadvancedTOP` |
|
||||
|
||||
Stack multiple `popForceTOP`s in series — each modifies velocity additively.
|
||||
|
||||
---
|
||||
|
||||
## Lifecycle Patterns
|
||||
|
||||
### Continuous emission (e.g. smoke plume)
|
||||
|
||||
```python
|
||||
src.par.birthrate = 800
|
||||
src.par.life = 6.0 # variance via 'lifevariance'
|
||||
src.par.lifevariance = 1.5
|
||||
```
|
||||
|
||||
### Burst emission (e.g. explosion)
|
||||
|
||||
```python
|
||||
src.par.birthrate = 0 # no continuous emission
|
||||
src.par.burst.pulse() # one burst on demand (verify param name)
|
||||
src.par.burstcount = 5000
|
||||
src.par.life = 1.5
|
||||
```
|
||||
|
||||
### Beat-triggered burst
|
||||
|
||||
Wire a `triggerCHOP` (from audio or MIDI) to pulse the burst:
|
||||
|
||||
```python
|
||||
op('/project1/audio_kick_trigger').outputConnectors[0].connect(...)
|
||||
# Then via a chopExecuteDAT, on each kick:
|
||||
def offToOn(channel, sampleIndex, val, prev):
|
||||
op('/project1/particles_geo/src').par.burst.pulse()
|
||||
return
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rendering Particles
|
||||
|
||||
### Point Sprites (simplest)
|
||||
|
||||
```python
|
||||
# Inside the geometryCOMP, render the solver output directly
|
||||
# The geo's first SOP child becomes the geometry
|
||||
# But for POPs, we typically render via glslMAT on a small "shape"
|
||||
|
||||
# Simple billboard sphere per particle:
|
||||
shape = geo.create(sphereSOP, 'shape')
|
||||
shape.par.rad = 0.05
|
||||
shape.par.rows = 6; shape.par.cols = 6 # low-poly to keep it fast
|
||||
|
||||
# Material that uses POP buffer for instancing
|
||||
mat = root.create(glslMAT, 'particle_mat')
|
||||
# Configure mat.par.instancingTOP = solver output (verify param name)
|
||||
```
|
||||
|
||||
The exact instancing setup varies by TD version — call `td_get_hints(topic='popInstancing')` (or `popRender` / `instancing` — try a few).
|
||||
|
||||
### GPU Sprites via glslcopyPOP
|
||||
|
||||
For dense smoke/fire-like effects, use a `glslcopyPOP` that writes per-particle color/size from a compute shader, then render as point sprites with additive blending in a `renderTOP`.
|
||||
|
||||
---
|
||||
|
||||
## Collisions
|
||||
|
||||
```python
|
||||
# Collision detection against an SOP
|
||||
coll = geo.create(popCollideTOP, 'ground_coll')
|
||||
coll.par.collidewithsop = '/project1/ground_geo' # path to colliding SOP
|
||||
coll.par.bounce = 0.3
|
||||
coll.par.friction = 0.1
|
||||
# Insert between force and solver
|
||||
```
|
||||
|
||||
For plane/box collisions only, use `popPlaneCollideTOP` (cheaper).
|
||||
|
||||
---
|
||||
|
||||
## Custom Per-Particle Data
|
||||
|
||||
Add a custom channel via `popAttribCreateTOP` (or by writing through `glslcopyPOP`):
|
||||
|
||||
```python
|
||||
# Add a "phase" attribute initialized random per-particle, used in render shader
|
||||
attr = geo.create(popAttribCreateTOP, 'add_phase')
|
||||
attr.par.attribname = 'phase'
|
||||
attr.par.value0 = 'rand(@id)' # expression in TD's POP attribute language
|
||||
```
|
||||
|
||||
Then in the render shader, `texture(sTDPOPInputs[0].phase, ...)` (or whichever sampler convention your TD version uses — verify with `td_get_docs(topic='pops')`).
|
||||
|
||||
---
|
||||
|
||||
## Legacy particleSOP (Use Sparingly)
|
||||
|
||||
For quick demos or low-count systems:
|
||||
|
||||
```python
|
||||
# Inside a geo
|
||||
psrc = geo.create(addSOP, 'point_src') # source: a single point
|
||||
psrc.par.points = '0 0 0'
|
||||
|
||||
part = geo.create(particleSOP, 'particles')
|
||||
part.par.life = 3.0
|
||||
part.par.birthrate = 100
|
||||
part.par.gravityy = -9.8
|
||||
part.par.windx = 0.5
|
||||
part.inputConnectors[0].connect(psrc)
|
||||
```
|
||||
|
||||
CPU-bound. Beyond ~5,000 active particles you'll see frame drops.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **Particles don't appear** — usually a render-side issue. Check via `td_get_screenshot` on the solver output (renders the buffer as a TOP-like view in newer TD). Then check the `geometryCOMP`'s render path.
|
||||
2. **Burst won't fire** — verify the `burst` param is a pulse, not a toggle. Pulses must use `.pulse()`, not `= True`.
|
||||
3. **Particles teleport on first frame** — uninitialized velocity. Set `popSourceTOP.par.initialvelocityX/Y/Z` or zero them explicitly.
|
||||
4. **Gravity feels wrong** — TD's "1 unit" depends on your scene scale. Start with `fy = -1.0` and scale up rather than using real-world 9.8.
|
||||
5. **High birthrate = stuttering** — birthrate is per-second, not per-frame. At 60fps, `birthrate = 6000` is 100/frame which is fine; `birthrate = 600000` will tank.
|
||||
6. **POP solver order matters** — forces apply in the order they appear in the chain. Putting gravity AFTER drag dampens gravity itself; usually not what you want.
|
||||
7. **Instancing param name varies** — `mat.par.instancingTOP` vs. `mat.par.instanceop` vs. `mat.par.instances` differs across TD versions. Always check `td_get_par_info(op_type='glslMAT')`.
|
||||
8. **Cooking dependency loops** — POP solvers create implicit time-loops. The "cook dependency loop" warning is expected and harmless for POPs.
|
||||
9. **CHOP-driven force values** — when a force param is expression-bound to a CHOP (e.g., audio-reactive gravity), make sure the CHOP cooks before the solver. If not, force lags by one frame.
|
||||
|
||||
---
|
||||
|
||||
## Performance Targets
|
||||
|
||||
| Particle count | Setup | Frame budget @ 60fps |
|
||||
|---|---|---|
|
||||
| < 1k | particleSOP fine | trivial |
|
||||
| 1k - 10k | POPs, simple forces | ~2-5ms |
|
||||
| 10k - 100k | POPs, GPU-only forces | ~5-15ms |
|
||||
| 100k+ | `glslcopyPOP`, custom compute | ~10-25ms |
|
||||
| 1M+ | Custom GPU buffer, no POP framework | depends on shader |
|
||||
|
||||
Use `td_get_perf` to find which op in the POP chain is the bottleneck.
|
||||
|
||||
---
|
||||
|
||||
## Quick Recipes
|
||||
|
||||
| Goal | Pipeline |
|
||||
|---|---|
|
||||
| Smoke plume | `popSourceTOP` (point) → gravity + wind + noise → `popDeleteTOP` (life) → solver → glslMAT instancing |
|
||||
| Beat-triggered burst | `triggerCHOP` (audio) → chopExecuteDAT pulses `popSourceTOP.par.burst` |
|
||||
| Fireworks shell | Burst at point → drag + gravity → secondary burst on lifetime threshold |
|
||||
| Snow/rain | Continuous emission across XZ plane (high y), gravity + small wind, infinite life box-deleted |
|
||||
| Sparks | Burst, very short life (0.3s), bright additive render, motion blur via feedback |
|
||||
| Audio particles | Birthrate driven by audio envelope, color driven by frequency band |
|
||||
@@ -0,0 +1,704 @@
|
||||
# TouchDesigner MCP — Pitfalls & Lessons Learned
|
||||
|
||||
Hard-won knowledge from real TD sessions. Read this before building anything.
|
||||
|
||||
## Parameter Names
|
||||
|
||||
### 1. NEVER hardcode parameter names — always discover
|
||||
|
||||
Parameter names change between TD versions. What works in one build may not work in another. ALWAYS use td_get_par_info to discover actual names from TD.
|
||||
|
||||
The agent's LLM training data contains WRONG parameter names. Do not trust them.
|
||||
|
||||
Known historical differences (may vary further — always verify):
|
||||
| What docs/training say | Actual in some versions | Notes |
|
||||
|---------------|---------------|-------|
|
||||
| `dat` | `pixeldat` | GLSL TOP pixel shader DAT |
|
||||
| `colora` | `alpha` | Constant TOP alpha |
|
||||
| `sizex` / `sizey` | `size` | Blur TOP (single value) |
|
||||
| `fontr/g/b/a` | `fontcolorr/g/b/a` | Text TOP font color (r/g/b) |
|
||||
| `fontcolora` | `fontalpha` | Text TOP font alpha (NOT `fontcolora`) |
|
||||
| `bgcolora` | `bgalpha` | Text TOP bg alpha |
|
||||
| `value1name` | `vec0name` | GLSL TOP uniform name |
|
||||
|
||||
### 2. twozero td_execute_python response format
|
||||
|
||||
When calling `td_execute_python` via twozero MCP, successful responses return `(ok)` followed by FPS/error summary (e.g. `[fps 60.0/60] [0 err/0 warn]`), NOT the raw Python `result` dict. If you're parsing responses programmatically, check for the `(ok)` prefix — don't pattern-match on Python variable names from the script. Use `td_get_operator_info` or separate inspection calls to read back values.
|
||||
|
||||
### 3. When using td_set_operator_pars, param names must match exactly
|
||||
|
||||
Use td_get_par_info to discover them. The MCP tool validates parameter names and returns clear errors explaining what went wrong, unlike raw Python which crashes the whole script with tdAttributeError and stops execution. Always discover before setting.
|
||||
|
||||
### 4. Use `safe_par()` pattern for cross-version compatibility
|
||||
|
||||
```python
|
||||
def safe_par(node, name, value):
|
||||
p = getattr(node.par, name, None)
|
||||
if p is not None:
|
||||
p.val = value
|
||||
return True
|
||||
return False
|
||||
```
|
||||
|
||||
### 5. `td.tdAttributeError` crashes the whole script — use defensive access
|
||||
|
||||
If you do `node.par.nonexistent = value`, TD raises `tdAttributeError` and stops the entire script. Prevention is better than catching:
|
||||
- Use `op()` instead of `opex()` — `op()` returns None on failure, `opex()` raises
|
||||
- Use `hasattr(node.par, 'name')` before accessing any parameter
|
||||
- Use `getattr(node.par, 'name', None)` with a default
|
||||
- Use the `safe_par()` pattern from pitfall #3
|
||||
|
||||
```python
|
||||
# WRONG — crashes if param doesn't exist:
|
||||
node.par.nonexistent = value
|
||||
|
||||
# CORRECT — defensive access:
|
||||
if hasattr(node.par, 'nonexistent'):
|
||||
node.par.nonexistent = value
|
||||
```
|
||||
|
||||
### 6. `outputresolution` is a string menu, not an integer
|
||||
|
||||
```
|
||||
menuNames: ['useinput','eighth','quarter','half','2x','4x','8x','fit','limit','custom','parpanel']
|
||||
```
|
||||
Always use the string form. Setting `outputresolution = 9` may silently fail.
|
||||
```python
|
||||
node.par.outputresolution = 'custom' # correct
|
||||
node.par.resolutionw = 1280; node.par.resolutionh = 720
|
||||
```
|
||||
Discover valid values: `list(node.par.outputresolution.menuNames)`
|
||||
|
||||
## GLSL Shaders
|
||||
|
||||
### 7. `uTDCurrentTime` does NOT exist in GLSL TOP
|
||||
|
||||
There is NO built-in time uniform for GLSL TOPs. GLSL MAT has `uTDGeneral.seconds` but that's NOT available in GLSL TOP context.
|
||||
|
||||
**PRIMARY — GLSL TOP Vectors/Values page:**
|
||||
```python
|
||||
gl.par.value0name = 'uTime'
|
||||
gl.par.value0.expr = "absTime.seconds"
|
||||
# In GLSL: uniform float uTime;
|
||||
```
|
||||
|
||||
**FALLBACK — Constant TOP texture (for complex time data):**
|
||||
|
||||
CRITICAL: set format to `rgba32float` — default 8-bit clamps to 0-1:
|
||||
```python
|
||||
t = root.create(constantTOP, 'time_driver')
|
||||
t.par.format = 'rgba32float'
|
||||
t.par.outputresolution = 'custom'
|
||||
t.par.resolutionw = 1; t.par.resolutionh = 1
|
||||
t.par.colorr.expr = "absTime.seconds % 1000.0"
|
||||
t.outputConnectors[0].connect(glsl.inputConnectors[0])
|
||||
```
|
||||
|
||||
### 8. GLSL compile errors are silent in the API
|
||||
|
||||
The GLSL TOP shows a yellow warning triangle in the UI but `node.errors()` may return empty string. Check `node.warnings()` too, and create an Info DAT pointed at the GLSL TOP to read the actual compiler output.
|
||||
|
||||
### 9. TD GLSL uses `vUV.st` not `gl_FragCoord` — and REQUIRES `TDOutputSwizzle()` on macOS
|
||||
|
||||
Standard GLSL patterns don't work. TD provides:
|
||||
- `vUV.st` — UV coordinates (0-1)
|
||||
- `uTDOutputInfo.res.zw` — resolution
|
||||
- `sTD2DInputs[0]` — input textures
|
||||
- `layout(location = 0) out vec4 fragColor` — output
|
||||
|
||||
CRITICAL on macOS: Always wrap output with `TDOutputSwizzle()`:
|
||||
```glsl
|
||||
fragColor = TDOutputSwizzle(color);
|
||||
```
|
||||
TD uses GLSL 4.60 (Vulkan backend). GLSL 3.30 and earlier removed.
|
||||
|
||||
### 10. Large GLSL shaders — write to temp file
|
||||
|
||||
GLSL code with special characters can corrupt JSON payloads. Write the shader to a temp file and load it in TD:
|
||||
```python
|
||||
# Agent side: write shader to /tmp/shader.glsl via write_file
|
||||
# TD side:
|
||||
sd = root.create(textDAT, 'shader_code')
|
||||
with open('/tmp/shader.glsl', 'r') as f:
|
||||
sd.text = f.read()
|
||||
```
|
||||
|
||||
## Node Management
|
||||
|
||||
### 11. Destroying nodes while iterating `root.children` causes `tdError`
|
||||
|
||||
The iterator is invalidated when a child is destroyed. Always snapshot first:
|
||||
```python
|
||||
kids = list(root.children) # snapshot
|
||||
for child in kids:
|
||||
if child.valid: # check — earlier destroys may cascade
|
||||
child.destroy()
|
||||
```
|
||||
|
||||
### 11b. Split cleanup and creation into SEPARATE td_execute_python calls
|
||||
|
||||
Creating nodes with the same names you just destroyed in the SAME script causes "Invalid OP object" errors — even with `list()` snapshot. TD's internal references can go stale within one execution context.
|
||||
|
||||
**WRONG (single call):**
|
||||
```python
|
||||
# td_execute_python:
|
||||
for c in list(root.children):
|
||||
if c.valid and c.name.startswith('my_'):
|
||||
c.destroy()
|
||||
# ... then create my_audio, my_shader etc. in same script → CRASHES
|
||||
```
|
||||
|
||||
**CORRECT (two separate calls):**
|
||||
```python
|
||||
# Call 1: td_execute_python — clean only
|
||||
for c in list(root.children):
|
||||
if c.valid and c.name.startswith('my_'):
|
||||
c.destroy()
|
||||
|
||||
# Call 2: td_execute_python — build (separate MCP call)
|
||||
audio = root.create(audiofileinCHOP, 'my_audio')
|
||||
# ... rest of build
|
||||
```
|
||||
|
||||
### 12. Feedback TOP: use `top` parameter, NOT direct input wire
|
||||
|
||||
The feedbackTOP's `top` parameter references which TOP to delay. Do NOT also wire that TOP directly into the feedback's input — this creates a real cook dependency loop.
|
||||
|
||||
Correct setup:
|
||||
```python
|
||||
fb = root.create(feedbackTOP, 'fb_delay')
|
||||
fb.par.top = comp.path # reference only — no wire to fb input
|
||||
fb.outputConnectors[0].connect(xf) # fb output -> transform -> fade -> comp
|
||||
```
|
||||
|
||||
The "Cook dependency loop detected" warning on the transform/fade chain is expected.
|
||||
|
||||
### 13. GLSL TOP auto-creates companion nodes
|
||||
|
||||
Creating a `glslTOP` also creates `name_pixel` (Text DAT), `name_info` (Info DAT), and `name_compute` (Text DAT). These are visible in the network. Don't be alarmed by "extra" nodes.
|
||||
|
||||
### 14. The default project root is `/project1`
|
||||
|
||||
New TD files start with `/project1` as the main container. System nodes live at `/`, `/ui`, `/sys`, `/local`, `/perform`. Don't create user nodes outside `/project1`.
|
||||
|
||||
### 15. Non-Commercial license caps resolution at 1280x1280
|
||||
|
||||
Setting `resolutionw=1920` silently clamps to 1280. Always check effective resolution after creation:
|
||||
```python
|
||||
n.cook(force=True)
|
||||
actual = str(n.width) + 'x' + str(n.height)
|
||||
```
|
||||
|
||||
## Recording & Codecs
|
||||
|
||||
### 16. MovieFileOut TOP: H.264/H.265/AV1 requires Commercial license
|
||||
|
||||
In Non-Commercial TD, these codecs produce an error. Recommended alternatives:
|
||||
- `prores` — Apple ProRes, **best on macOS**, HW accelerated, NOT license-restricted. ~55MB/s at 1280x720 but lossless quality. **Use this as default on macOS.**
|
||||
- `cineform` — GoPro Cineform, supports alpha
|
||||
- `hap` — GPU-accelerated playback, large files
|
||||
- `notchlc` — GPU-accelerated, good quality
|
||||
- `mjpa` — Motion JPEG, legacy fallback (lossy, use only if ProRes unavailable)
|
||||
|
||||
For image sequences: `rec.par.type = 'imagesequence'`, `rec.par.imagefiletype = 'png'`
|
||||
|
||||
### 17. MovieFileOut `.record()` method may not exist
|
||||
|
||||
Use the toggle parameter instead:
|
||||
```python
|
||||
rec.par.record = True # start recording
|
||||
rec.par.record = False # stop recording
|
||||
```
|
||||
|
||||
When setting file path and starting recording in the same script, use delayFrames:
|
||||
```python
|
||||
rec.par.file = '/tmp/new_output.mov'
|
||||
run("op('/project1/recorder').par.record = True", delayFrames=2)
|
||||
```
|
||||
|
||||
### 18. TOP.save() captures same frame when called rapidly
|
||||
|
||||
Use MovieFileOut for real-time recording. Set `project.realTime = False` for frame-accurate output.
|
||||
|
||||
### 19. AudioFileIn CHOP: cue and recording sequence matters
|
||||
|
||||
The recording sequence must be done in exact order, or the recording will be empty, audio will start mid-file, or the file won't be written.
|
||||
|
||||
**Proven recording sequence:**
|
||||
|
||||
```python
|
||||
# Step 1: Stop any existing recording
|
||||
rec.par.record = False
|
||||
|
||||
# Step 2: Reset audio to beginning
|
||||
audio.par.play = False
|
||||
audio.par.cue = True
|
||||
audio.par.cuepoint = 0 # may need cuepointunit=0 too
|
||||
# Verify: audio.par.cue.eval() should be True
|
||||
|
||||
# Step 3: Set output file path
|
||||
rec.par.file = '/tmp/output.mov'
|
||||
|
||||
# Step 4: Release cue + start playing + start recording (with frame delay)
|
||||
audio.par.cue = False
|
||||
audio.par.play = True
|
||||
audio.par.playmode = 2 # Sequential — plays once through
|
||||
run("op('/project1/recorder').par.record = True", delayFrames=3)
|
||||
```
|
||||
|
||||
**Why each step matters:**
|
||||
- `rec.par.record = False` first — if a previous recording is active, setting `par.file` may fail silently
|
||||
- `audio.par.cue = True` + `cuepoint = 0` — guarantees audio starts from the beginning, otherwise the spectrum may be silent for the first few seconds
|
||||
- `delayFrames=3` on the record start — setting `par.file` and `par.record = True` in the same script can race; the file path needs a frame to register before recording starts
|
||||
- `playmode = 2` (Sequential) — plays the file once. Use `playmode = 0` (Locked to Timeline) if you want TD's timeline to control position
|
||||
|
||||
## TD Python API Patterns
|
||||
|
||||
### 20. COMP extension setup: ext0object format is CRITICAL
|
||||
|
||||
`ext0object` expects a CONSTANT string (NOT expression mode):
|
||||
```python
|
||||
comp.par.ext0object = "op('./myExtensionDat').module.MyClassName(me)"
|
||||
```
|
||||
NEVER set as just the DAT name. NEVER use ParMode.EXPRESSION. ALWAYS ensure the DAT has `par.language='python'`.
|
||||
|
||||
### 21. td.Panel is NOT subscriptable — use attribute access
|
||||
|
||||
```python
|
||||
comp.panel.select # correct (attribute access, returns float)
|
||||
comp.panel['select'] # WRONG — 'td.Panel' object is not subscriptable
|
||||
```
|
||||
|
||||
### 22. ALWAYS use relative paths in script callbacks
|
||||
|
||||
In scriptTOP/CHOP/SOP/DAT callbacks, use paths relative to `scriptOp` or `me`:
|
||||
```python
|
||||
root = scriptOp.parent().parent()
|
||||
dat = root.op('pixel_data')
|
||||
```
|
||||
NEVER hardcode absolute paths like `op('/project1/myComp/child')` — they break when containers are renamed or copied.
|
||||
|
||||
### 23. keyboardinCHOP channel names have 'k' prefix
|
||||
|
||||
Channel names are `kup`, `kdown`, `kleft`, `kright`, `ka`, `kb`, etc. — NOT `up`, `down`, `a`, `b`. Always verify with:
|
||||
```python
|
||||
channels = [c.name for c in op('/project1/keyboard1').chans()]
|
||||
```
|
||||
|
||||
### 24. expressCHOP cook-only properties — false positive errors
|
||||
|
||||
`me.inputVal`, `me.chanIndex`, `me.sampleIndex` work ONLY in cook-context. Calling `par.expr0expr.eval()` from outside always raises an error — this is NOT a real operator error. Ignore these in error scans.
|
||||
|
||||
### 25. td.Vertex attributes — use index access not named attributes
|
||||
|
||||
In TD 2025.32, `td.Vertex` objects do NOT have `.x`, `.y`, `.z` attributes:
|
||||
```python
|
||||
# WRONG — crashes:
|
||||
vertex.x, vertex.y, vertex.z
|
||||
|
||||
# CORRECT — index-based:
|
||||
vertex.point.P[0], vertex.point.P[1], vertex.point.P[2]
|
||||
# Or for SOP point positions:
|
||||
pt = sop.points()[i]
|
||||
pos = pt.P # use P[0], P[1], P[2]
|
||||
```
|
||||
|
||||
## Audio
|
||||
|
||||
### 26. Audio Spectrum CHOP output is weak — boost it
|
||||
|
||||
Raw output is very small (0.001-0.05). Use built-in boost: `spectrum.par.highfrequencyboost = 3.0`
|
||||
|
||||
If still weak, add Math CHOP in Range mode: `fromrangehi=0.05, torangehi=1.0`
|
||||
|
||||
### 27. AudioSpectrum CHOP: timeslice and sample count are the #1 gotcha
|
||||
|
||||
AudioSpectrum at 44100Hz with `timeslice=False` outputs the ENTIRE audio file as samples (~24000+). CHOP-to-TOP then exceeds texture resolution max and warns/fails.
|
||||
|
||||
**Fix:** Keep `timeslice = True` (default) for real-time per-frame FFT. Set `fftsize` to control bin count (it's a STRING enum: `'256'` not `256`).
|
||||
|
||||
If the CHOP-to-TOP still gets too many samples, set `layout = 'rowscropped'` on the choptoTOP.
|
||||
|
||||
```python
|
||||
spectrum.par.fftsize = '256' # STRING, not int — enum values
|
||||
spectrum.par.timeslice = True # MUST be True for real-time audio reactivity
|
||||
spectex.par.layout = 'rowscropped' # handles oversized CHOP inputs
|
||||
```
|
||||
|
||||
**resampleCHOP has NO `numsamples` param.** It uses `rate`, `start`, `end`, `method`. Don't guess — always `td_get_par_info('resampleCHOP')` first.
|
||||
|
||||
### 28. CHOP To TOP has NO input connectors — use par.chop reference
|
||||
|
||||
```python
|
||||
spec_tex = root.create(choptoTOP, 'spectrum_tex')
|
||||
spec_tex.par.chop = resample # correct: parameter reference
|
||||
# NOT: resample.outputConnectors[0].connect(spec_tex.inputConnectors[0]) # WRONG
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
### 29. Always verify after building — errors are silent
|
||||
|
||||
Node errors and broken connections produce no output. Always check:
|
||||
```python
|
||||
for c in list(root.children):
|
||||
e = c.errors()
|
||||
w = c.warnings()
|
||||
if e: print(c.name, 'ERR:', e)
|
||||
if w: print(c.name, 'WARN:', w)
|
||||
```
|
||||
|
||||
### 30. Window COMP param for display target is `winop`
|
||||
|
||||
```python
|
||||
win = root.create(windowCOMP, 'display')
|
||||
win.par.winop = '/project1/logo_out'
|
||||
win.par.winw = 1280; win.par.winh = 720
|
||||
win.par.winopen.pulse()
|
||||
```
|
||||
|
||||
### 31. `sample()` returns frozen pixels in rapid calls
|
||||
|
||||
`out.sample(x, y)` returns pixels from a single cook snapshot. Compare samples with 2+ second delays, or use screencapture on the display window.
|
||||
|
||||
### 32. Audio-reactive GLSL: TD-side pipeline
|
||||
|
||||
For audio-synced visuals: AudioFileIn → AudioSpectrum(timeslice=True, fftsize='256') → Math(gain=5) → choptoTOP(par.chop=math, layout='rowscropped') → GLSL input. The shader samples `sTD2DInputs[1]` at different x positions for bass/mid/hi. Record the TD output with MovieFileOut.
|
||||
|
||||
**Key gotcha:** AudioFileIn must be cued (`par.cue=True` → `par.cuepulse.pulse()`) then uncued (`par.cue=False`, `par.play=True`) before recording starts. Otherwise the spectrum is silent for the first few seconds.
|
||||
|
||||
### 33. twozero MCP: prefer native tools
|
||||
|
||||
**Always prefer native MCP tools over td_execute_python:**
|
||||
- `td_create_operator` over `root.create()` scripts (handles viewport positioning)
|
||||
- `td_set_operator_pars` over `node.par.X = Y` scripts (validates param names)
|
||||
- `td_get_par_info` over temp-node discovery dance (instant, no cleanup)
|
||||
- `td_get_errors` over manual `c.errors()` loops
|
||||
- `td_get_focus` for context awareness (no equivalent in old method)
|
||||
|
||||
Only fall back to `td_execute_python` for multi-step logic (wiring chains, conditional builds, loops).
|
||||
|
||||
### 34. twozero td_execute_python response wrapping
|
||||
|
||||
twozero wraps `td_execute_python` responses with status info: `(ok)\n\n[fps 60.0/60] [0 err/0 warn]`. Your Python `result` variable value may not appear verbatim in the response text. If you need to check results programmatically, use `print()` statements in the script — they appear in the response. Don't rely on string-matching the `result` dict.
|
||||
|
||||
### 35. Audio-reactive chain: DO NOT use Lag CHOP or Filter CHOP for spectrum smoothing
|
||||
|
||||
The Derivative docs and tutorials suggest using Lag CHOP (lag1=0.2, lag2=0.5) to smooth raw FFT output before passing to a shader. **This does NOT work with AudioSpectrum → CHOP to TOP → GLSL.**
|
||||
|
||||
What happens: Lag CHOP operates in timeslice mode. A 256-sample spectrum input gets expanded to 1600-2400 samples. The Lag averaging drives all values to near-zero (~1e-06). The CHOP to TOP produces a 2400x2 texture instead of 256x2. The shader receives effectively zero audio data.
|
||||
|
||||
**The correct chain is: Spectrum(outlength=256) → Math(gain=10) → CHOPtoTOP → GLSL.** No CHOP smoothing at all. If you need smoothing, do it in the GLSL shader via temporal lerp with a feedback texture.
|
||||
|
||||
Verified values with audio playing:
|
||||
- Without Lag CHOP: bass bins = 5.0-5.4, mid bins = 1.0-1.7 (strong, usable)
|
||||
- With Lag CHOP: ALL bins = 0.000001-0.00004 (dead, zero audio reactivity)
|
||||
|
||||
### 36. AudioSpectrum Output Length: set manually to avoid CHOP to TOP overflow
|
||||
|
||||
AudioSpectrum in Visualization mode with FFT 8192 outputs 22,050 samples by default (1 per Hz, 0–22050). CHOP to TOP cannot handle this — you get "Number of samples exceeded texture resolution max".
|
||||
|
||||
Fix: `spectrum.par.outputmenu = 'setmanually'` and `spectrum.par.outlength = 256`. This gives 256 frequency bins — plenty for visual FFT.
|
||||
|
||||
DO NOT set `timeslice = False` as a workaround — that processes the entire audio file at once and produces even more samples.
|
||||
|
||||
### 37. GLSL spectrum texture from CHOP to TOP is 256x2 not 256x1
|
||||
|
||||
AudioSpectrum outputs 2 channels (stereo: chan1, chan2). CHOP to TOP with `dataformat='r'` creates a 256x2 texture — one row per channel. Sample the first channel at `y=0.25` (center of first row), NOT `y=0.5` (boundary between rows):
|
||||
|
||||
```glsl
|
||||
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.25)).r; // correct
|
||||
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.5)).r; // WRONG — samples between rows
|
||||
```
|
||||
|
||||
### 38. FPS=0 doesn't mean ops aren't cooking — check play state
|
||||
|
||||
TD can show `fps:0` in `td_get_perf` while ops still cook and `TOP.save()` still produces valid screenshots. The two most common causes:
|
||||
|
||||
**a) Project is paused (playbar stopped).** TD's playbar can be toggled with spacebar. The `root` at `/` has no `.playbar` attribute (it's on the perform COMP). The easiest fix is sending a spacebar keypress via `td_input_execute`, though this tool can sometimes error. As a workaround, `TOP.save()` always works regardless of play state — use it to verify rendering is actually happening before spending time debugging FPS.
|
||||
|
||||
**b) Audio device CHOP blocking the main thread (MOST COMMON).** An `audiodeviceoutCHOP` with `active=True` can consume 300-400ms/s (2000%+ of frame budget), stalling the cook loop at FPS=0. **`volume=0` is NOT sufficient** — the audio driver still blocks. Fix: `par.active = False`. This completely stops the CHOP from interacting with the audio driver. If you need audio monitoring, enable it only during short playback checks, then disable before recording.
|
||||
|
||||
Verified April 2026: disabling `audiodeviceoutCHOP` (`active=False`) restored FPS from 0 to 60 instantly, recovering from 2348% budget usage to 0.1%.
|
||||
|
||||
Diagnostic sequence when FPS=0:
|
||||
1. `td_get_perf` — check if any op has extreme CPU/s (audiodeviceoutCHOP is the usual suspect)
|
||||
2. If audiodeviceoutCHOP shows >100ms/s: set `par.active = False` immediately
|
||||
3. `TOP.save()` on the output — if it produces a valid image, the pipeline works, just not at real-time rate
|
||||
4. Check for other blocking CHOPs (audiodevin, etc.)
|
||||
5. Toggle play state (spacebar, or check if absTime.seconds is advancing)
|
||||
|
||||
### 39. Recording while FPS=0 produces empty or near-empty files
|
||||
|
||||
This is the #1 cause of "I recorded for 30 seconds but got a 2-frame video." If TD's cook loop is stalled (FPS=0 or very low), MovieFileOut has nothing to record. Unlike `TOP.save()` which captures the last cooked frame regardless, MovieFileOut only writes frames that actually cook.
|
||||
|
||||
**Always verify FPS before starting a recording:**
|
||||
```python
|
||||
# Check via td_get_perf first
|
||||
# If FPS < 30, do NOT start recording — fix the performance issue first
|
||||
# If FPS=0, the playbar is likely paused — see pitfall #37
|
||||
```
|
||||
|
||||
Common causes of recording empty video:
|
||||
- Playbar paused (FPS=0) — see pitfall #37
|
||||
- Audio device CHOP blocking the main thread — see pitfall #37b
|
||||
- Recording started before audio was cued — audio is silent, GLSL outputs black, MovieFileOut records black frames that look empty
|
||||
- `par.file` set in the same script as `par.record = True` — see pitfall #18
|
||||
|
||||
### 40. GLSL shader produces black output — test before committing to a long render
|
||||
|
||||
New GLSL shaders can fail silently (see pitfall #7). Before recording a long take, always:
|
||||
|
||||
1. **Write a minimal test shader first** that just outputs a solid color or pass-through:
|
||||
```glsl
|
||||
void main() {
|
||||
vec2 uv = vUV.st;
|
||||
fragColor = TDOutputSwizzle(vec4(uv, 0.0, 1.0));
|
||||
}
|
||||
```
|
||||
|
||||
2. **Verify the test renders correctly** via `td_get_screenshot` on the GLSL TOP's output.
|
||||
|
||||
3. **Swap in the real shader** and screenshot again immediately. If black, the shader has a compile error or logic issue.
|
||||
|
||||
4. **Only then start recording.** A 90-second ProRes recording is ~5GB. Recording black frames wastes disk and time.
|
||||
|
||||
Common causes of black GLSL output:
|
||||
- Missing `TDOutputSwizzle()` on macOS (pitfall #8)
|
||||
- Time uniform not connected — shader uses default 0.0, fractal stays at origin
|
||||
- Spectrum texture not connected — audio values all 0.0, driving everything to black
|
||||
- Integer division where float division was expected (`1/2 = 0` not `0.5`)
|
||||
- `absTime.seconds % 1000.0` rolled over past 1000 and the modulo produces unexpected values
|
||||
|
||||
### 41. td_write_dat uses `text` parameter, NOT `content`
|
||||
|
||||
The MCP tool `td_write_dat` expects a `text` parameter for full replacement. Passing `content` returns an error: `"Provide either 'text' for full replace, or 'old_text'+'new_text' for patching"`.
|
||||
|
||||
If `td_write_dat` fails, fall back to `td_execute_python`:
|
||||
```python
|
||||
op("/project1/shader_code").text = shader_string
|
||||
```
|
||||
|
||||
### 42. td_execute_python DOES return print() output — use it for debugging
|
||||
|
||||
`print()` statements in `td_execute_python` scripts appear in the MCP response text. This is the correct way to read values back from scripts. The response format is: printed output first, then `[fps X.X/X] [N err/N warn]` on a separate line.
|
||||
|
||||
However, the `result` variable (if you set one) does NOT appear verbatim — use `print()` for anything you need to read back:
|
||||
```python
|
||||
# CORRECT — appears in response:
|
||||
print('value:', some_value)
|
||||
|
||||
# WRONG — not reliably in response:
|
||||
result = some_value
|
||||
```
|
||||
|
||||
For structured data, use dedicated inspection tools (`td_get_operator_info`, `td_read_chop`) which return clean JSON.
|
||||
|
||||
### 43. td_get_operator_info JSON is appended with `[fps X.X/X]` — breaks json.loads()
|
||||
|
||||
The response text from `td_get_operator_info` has `[fps 60.0/60]` appended after the JSON object. This causes `json.loads()` to fail with "Extra data" errors. Strip it before parsing:
|
||||
```python
|
||||
clean = response_text.rsplit('[fps', 1)[0]
|
||||
data = json.loads(clean)
|
||||
```
|
||||
|
||||
### 44. td_get_screenshot is unreliable — returns `{"status": "pending"}` and may never deliver
|
||||
|
||||
Screenshots don't complete instantly. The tool returns `{"status": "pending", "requestId": "..."}` and the actual file may appear later — or may NEVER appear at all. In testing (April 2026), screenshots stayed "pending" indefinitely with no file written to disk, even though the shader was cooking at 8-30fps.
|
||||
|
||||
**Do NOT rely on `td_get_screenshot` for frame capture.** For reliable frame capture, use MovieFileOut recording + ffmpeg frame extraction:
|
||||
```bash
|
||||
# Record in TD first, then extract frames:
|
||||
ffmpeg -y -i /tmp/td_output.mov -t 25 -vf 'fps=24' /tmp/td_frames/frame_%06d.png
|
||||
```
|
||||
|
||||
If you need a quick visual check, `td_get_screenshot` is worth trying (it sometimes works), but always have the recording fallback. There is no callback or completion notification — if the file doesn't appear after 5-10 seconds, it's not coming.
|
||||
|
||||
### 45. Heavy shaders cook below record FPS — many duplicate frames in output
|
||||
|
||||
A raymarched GLSL shader may only cook at 8-15fps even though MovieFileOut records at 60fps. The recording still works (TD writes the last-cooked frame each time), but the resulting file has many duplicate frames. When extracting frames for post-processing, use a lower fps filter to avoid redundant frames:
|
||||
```bash
|
||||
# Extract at 24fps from a 60fps recording of an 8fps shader:
|
||||
ffmpeg -y -i /tmp/td_output.mov -t 25 -vf 'fps=24' /tmp/td_frames/frame_%06d.png
|
||||
```
|
||||
Check actual cook FPS with `td_get_perf` before committing to a long recording. If FPS < 15, the output will be a slideshow regardless of the recording codec.
|
||||
|
||||
### 46. Recording duration is manual — no auto-stop at audio end
|
||||
|
||||
MovieFileOut records until `par.record = False` is set. If audio ends before you stop recording, the file keeps growing with repeated frames. Always stop recording promptly after the audio duration. For precision: set a timer on the agent side matching the audio length, then send `par.record = False`. Trim excess with ffmpeg as a safety net:
|
||||
```bash
|
||||
ffmpeg -i raw.mov -t 25 -c copy trimmed.mov
|
||||
```
|
||||
|
||||
### 47. AudioFileIn par.index stays at 0 in sequential mode — not a reliable progress indicator
|
||||
|
||||
When `audiofileinCHOP` is in `playmode=2` (sequential), `par.index.eval()` returns 0.0 even while audio IS actively playing and the spectrum IS receiving data. Do NOT use `par.index` to check playback progress in sequential mode.
|
||||
|
||||
**How to verify audio is actually playing:**
|
||||
- Read the spectrum CHOP values via `td_read_chop` — if values are non-zero and CHANGE between reads 1-2s apart, audio is flowing
|
||||
- Read the audio CHOP itself: non-zero waveform samples confirm the file is loaded and playing
|
||||
- `par.play.eval()` returning True is necessary but NOT sufficient — it can be True with no audio flowing if cue is stuck
|
||||
|
||||
### 48. GLSL shader whiteout — clamp audio spectrum values in the shader
|
||||
|
||||
Raw spectrum values multiplied by Math CHOP gain can produce very large numbers (5-20+) that blow out the shader's lighting, producing flat white/grey. The shader MUST clamp audio inputs:
|
||||
|
||||
```glsl
|
||||
float bass = texture(sTD2DInputs[1], vec2(0.05, 0.25)).r;
|
||||
bass = clamp(bass, 0.0, 3.0); // prevent whiteout
|
||||
mids = clamp(mids, 0.0, 3.0);
|
||||
hi = clamp(hi, 0.0, 3.0);
|
||||
```
|
||||
|
||||
Discovered when gain=10 produced ~0.13 (too dark) during quiet passages but gain=50 produced ~9.4 (total whiteout). Fix: keep gain=10, use `highfreqboost=3.0` on AudioSpectrum, clamp in shader.
|
||||
|
||||
### 49. Non-Commercial TD records at 1280x1280 (square) — always crop in post
|
||||
|
||||
Even with `resolutionw=1280, resolutionh=720` on the GLSL TOP, Non-Commercial TD may output 1280x1280 to MovieFileOut. Always check dimensions with ffprobe and crop during extraction:
|
||||
|
||||
```bash
|
||||
# Center-crop from 1280x1280 to 1280x720:
|
||||
ffmpeg -y -i /tmp/td_output.mov -t 25 -r 24 -vf "crop=1280:720:0:280" /tmp/frames/frame_%06d.png
|
||||
```
|
||||
|
||||
Large ProRes files (1-2GB) at 1280x1280 decode at ~3fps, so 25s of footage takes ~3 minutes to extract.
|
||||
|
||||
## Advanced Patterns (pitfalls 51+)
|
||||
|
||||
### 51. Connection syntax: use `outputConnectors`/`inputConnectors`, NOT `outputs`/`inputs`
|
||||
|
||||
```python
|
||||
# CORRECT
|
||||
src.outputConnectors[0].connect(dst.inputConnectors[0])
|
||||
# WRONG — raises IndexError or AttributeError
|
||||
src.outputs[0].connect(dst.inputs[0])
|
||||
```
|
||||
|
||||
For feedback TOP, BOTH are required:
|
||||
```python
|
||||
fb.par.top = target.path
|
||||
target.outputConnectors[0].connect(fb.inputConnectors[0])
|
||||
```
|
||||
|
||||
### 52. moviefileoutTOP `par.input` doesn't resolve via Python in TD 2025.32460
|
||||
|
||||
Setting `moviefileoutTOP.par.input` programmatically does NOT work. All forms fail silently with "Not enough sources specified."
|
||||
|
||||
**Workaround — frame capture + ffmpeg:**
|
||||
```python
|
||||
out = op('/project1/out')
|
||||
for i in range(300):
|
||||
delay = i * 5
|
||||
run(f"op('/project1/out').save('/tmp/frames/f_{i:04d}.png')", delayFrames=delay)
|
||||
# Then: ffmpeg -y -framerate 30 -i /tmp/frames/f_%04d.png -c:v prores -pix_fmt yuv420p /tmp/output.mov
|
||||
```
|
||||
|
||||
### 53. Batch frame capture — use `me.fetch`/`me.store` for state across calls
|
||||
|
||||
```python
|
||||
start = me.fetch('cap_frame', 0)
|
||||
for i in range(60):
|
||||
frame = start + i
|
||||
op('/project1/out').save(f'/tmp/frames/frame_{str(frame).zfill(4)}.png')
|
||||
me.store('cap_frame', start + 60)
|
||||
```
|
||||
Call 5 times for 300 frames. Each picks up where the last left off.
|
||||
|
||||
### 54. GLSL TOP pixel shader requirements in TD 2025
|
||||
|
||||
```glsl
|
||||
// REQUIRED — declare output
|
||||
layout(location = 0) out vec4 fragColor;
|
||||
|
||||
void main() {
|
||||
vec3 col = vec3(1.0, 0.0, 0.0);
|
||||
fragColor = TDOutputSwizzle(vec4(col, 1.0));
|
||||
}
|
||||
```
|
||||
**Built-in uniforms available:** `uTDOutputInfo.res` (vec4), `uTDTimeInfo.seconds`, `sTD2DInputs[N]`.
|
||||
**Auto-created DATs:** `name_pixel`, `name_vertex`, `name_compute` textDATs with example code.
|
||||
|
||||
### 55. TOP.save() doesn't advance time — identical frames in tight loops
|
||||
|
||||
`.save()` captures the current cooked frame without advancing TD's timeline:
|
||||
```python
|
||||
# WRONG — all frames identical
|
||||
for i in range(300):
|
||||
op('/project1/out').save(f'frames/f_{i:04d}.png')
|
||||
|
||||
# CORRECT — use run() with delayFrames
|
||||
for i in range(300):
|
||||
delay = i * 5
|
||||
run(f"op('/project1/out').save('frames/f_{i:04d}.png')", delayFrames=delay)
|
||||
```
|
||||
**NEVER use `time.sleep()` in TD** — it blocks the main thread and freezes the UI.
|
||||
|
||||
### 56. Feedback loop masks input changes — force switch during capture
|
||||
|
||||
With feedback TOP opacity 0.7+, the buffer dominates output. Switching input produces nearly identical frames.
|
||||
|
||||
**Fix — force switch index per capture:**
|
||||
```python
|
||||
for i in range(300):
|
||||
idx = (i // 8) % num_inputs
|
||||
delay = i * 5
|
||||
run(f"op('/project1/vswitch').par.index={idx}; op('/project1/out').save('f_{i:04d}.png')", delayFrames=delay)
|
||||
```
|
||||
|
||||
### 57. Large td_execute_python scripts fail — split into incremental calls
|
||||
|
||||
10+ operator creations in one script cause timing issues. Split into 2-4 calls of 2-4 operators each. Within one call, `create()` handles work immediately. Across calls, `op('name')` may return `None` if the previous call hasn't committed.
|
||||
|
||||
### 58. MCP instance reconnection after project.load()
|
||||
|
||||
`project.load(path)` changes the PID. After loading, call `td_list_instances()` and use the new `target_instance`. For TOX files: import as child comp instead (doesn't disconnect).
|
||||
|
||||
### 59. TOX reverse-engineering workflow
|
||||
|
||||
```python
|
||||
comp = root.loadTox(r'/path/to/file.tox')
|
||||
comp.name = '_study_comp'
|
||||
for child in comp.children:
|
||||
print(f'{child.name} ({child.OPType})')
|
||||
# Use td_get_operators_info, td_read_dat, check custom params
|
||||
```
|
||||
|
||||
### 60. sliderCOMP naming — TD appends suffix
|
||||
|
||||
TD auto-renames: `slider_brightness` → `slider_brightness1`. Always check names after creation.
|
||||
|
||||
### 61. create() requires full operator type suffix
|
||||
|
||||
```python
|
||||
# CORRECT
|
||||
proj.create('audiofileinCHOP', 'audio_in')
|
||||
proj.create('glslTOP', 'render')
|
||||
|
||||
# WRONG — raises "Unknown operator type"
|
||||
proj.create('audiofilein', 'audio_in')
|
||||
proj.create('glsl', 'render')
|
||||
```
|
||||
|
||||
### 62. Reparenting COMPs — use copyOPs, not connect()
|
||||
|
||||
Moving COMPs with `inputCOMPConnectors[0].connect()` fails. Use copy + destroy:
|
||||
```python
|
||||
copied = target.copyOPs([source]) # preserves internal wiring
|
||||
source.destroy()
|
||||
# Re-wire external connections manually after the move
|
||||
```
|
||||
|
||||
### 63. Slider wiring — expressionCHOP with op() expressions crashes TD
|
||||
|
||||
```python
|
||||
# CRASHES TD — don't do this
|
||||
echop = root.create(expressionCHOP, 'slider_ctrl')
|
||||
echop.par.chan0expr = 'op("/project1/controls/slider_brightness1").par.value0'
|
||||
|
||||
# WORKING — parameterCHOP as bridge
|
||||
pchop = root.create(parameterCHOP, 'slider_vals')
|
||||
pchop.par.ops = '/project1/controls'
|
||||
pchop.par.parameters = 'value0'
|
||||
pchop.par.custom = True
|
||||
pchop.par.builtin = False
|
||||
```
|
||||
@@ -0,0 +1,183 @@
|
||||
# Post-FX Reference
|
||||
|
||||
Bloom, CRT scanlines, chromatic aberration, and feedback glow patterns for live visual work.
|
||||
|
||||
---
|
||||
|
||||
## Bloom
|
||||
|
||||
### Built-in Bloom TOP
|
||||
|
||||
TD's `bloomTOP` is the fastest path — GPU-accelerated, no shader needed.
|
||||
|
||||
```python
|
||||
bloom = root.create(bloomTOP, 'bloom1')
|
||||
bloom.par.threshold = 0.6 # Luminance threshold (0-1)
|
||||
bloom.par.size = 0.03 # Spread radius (0-1)
|
||||
bloom.par.strength = 1.5 # Bloom intensity
|
||||
bloom.par.blendmode = 'add' # 'add' or 'screen'
|
||||
```
|
||||
|
||||
**Audio reactive bloom:**
|
||||
```python
|
||||
bloom.par.strength.mode = ParMode.EXPRESSION
|
||||
bloom.par.strength.expr = "op('audio_env')['envelope'][0] * 3.0 + 0.5"
|
||||
```
|
||||
|
||||
### GLSL Bloom (More Control)
|
||||
|
||||
For multi-pass bloom with color tinting:
|
||||
|
||||
```glsl
|
||||
// bloom_pixel.glsl — pass1: threshold + tint
|
||||
out vec4 fragColor;
|
||||
uniform float uThreshold;
|
||||
uniform vec3 uBloomColor;
|
||||
|
||||
void main() {
|
||||
vec4 col = texture(sTD2DInputs[0], vUV.st);
|
||||
float luma = dot(col.rgb, vec3(0.299, 0.587, 0.114));
|
||||
float bloom = max(0.0, luma - uThreshold);
|
||||
fragColor = TDOutputSwizzle(vec4(col.rgb * bloom * uBloomColor, col.a));
|
||||
}
|
||||
```
|
||||
|
||||
Then blur with `blurTOP` (size ~0.02-0.05), composite back over source with `addTOP` or `compositeTOP` in Add mode.
|
||||
|
||||
---
|
||||
|
||||
## CRT / Scanlines
|
||||
|
||||
Pure GLSL — create a `glslTOP` and paste into its `_pixel` DAT.
|
||||
|
||||
```glsl
|
||||
// crt_pixel.glsl
|
||||
out vec4 fragColor;
|
||||
uniform float uTime;
|
||||
uniform float uScanlineIntensity; // 0.0 - 1.0, default 0.4
|
||||
uniform float uCurvature; // 0.0 - 0.15, default 0.05
|
||||
uniform float uVignette; // 0.0 - 1.0, default 0.8
|
||||
|
||||
vec2 curveUV(vec2 uv, float amount) {
|
||||
uv = uv * 2.0 - 1.0;
|
||||
vec2 offset = abs(uv.yx) / vec2(6.0, 4.0);
|
||||
uv = uv + uv * offset * offset * amount;
|
||||
return uv * 0.5 + 0.5;
|
||||
}
|
||||
|
||||
void main() {
|
||||
vec2 res = uTDOutputInfo.res.zw;
|
||||
vec2 uv = vUV.st;
|
||||
|
||||
// CRT barrel distortion
|
||||
uv = curveUV(uv, uCurvature * 10.0);
|
||||
|
||||
// Kill pixels outside curved screen
|
||||
if (uv.x < 0.0 || uv.x > 1.0 || uv.y < 0.0 || uv.y > 1.0) {
|
||||
fragColor = vec4(0.0, 0.0, 0.0, 1.0);
|
||||
return;
|
||||
}
|
||||
|
||||
vec4 col = texture(sTD2DInputs[0], uv);
|
||||
|
||||
// Scanlines
|
||||
float scanline = sin(uv.y * res.y * 3.14159) * 0.5 + 0.5;
|
||||
col.rgb *= mix(1.0, scanline, uScanlineIntensity);
|
||||
|
||||
// Horizontal noise flicker
|
||||
float flicker = TDSimplexNoise(vec2(uv.y * 100.0, uTime * 8.0)) * 0.03;
|
||||
col.rgb += flicker;
|
||||
|
||||
// Vignette
|
||||
vec2 vig = uv * (1.0 - uv.yx);
|
||||
float v = pow(vig.x * vig.y * 15.0, uVignette);
|
||||
col.rgb *= v;
|
||||
|
||||
fragColor = TDOutputSwizzle(col);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Chromatic Aberration
|
||||
|
||||
Splits RGB channels and offsets them along screen axes.
|
||||
|
||||
```glsl
|
||||
out vec4 fragColor;
|
||||
uniform float uAmount; // 0.001 - 0.02, default 0.006
|
||||
|
||||
void main() {
|
||||
vec2 uv = vUV.st;
|
||||
vec2 dir = uv - 0.5;
|
||||
|
||||
float r = texture(sTD2DInputs[0], uv + dir * uAmount).r;
|
||||
float g = texture(sTD2DInputs[0], uv).g;
|
||||
float b = texture(sTD2DInputs[0], uv - dir * uAmount).b;
|
||||
float a = texture(sTD2DInputs[0], uv).a;
|
||||
|
||||
fragColor = TDOutputSwizzle(vec4(r, g, b, a));
|
||||
}
|
||||
```
|
||||
|
||||
**Audio-reactive variant** — spike aberration on beats:
|
||||
```glsl
|
||||
uniform float uBeat;
|
||||
void main() {
|
||||
vec2 uv = vUV.st;
|
||||
vec2 dir = uv - 0.5;
|
||||
float amount = uAmount + uBeat * 0.04;
|
||||
float r = texture(sTD2DInputs[0], uv + dir * amount * 1.2).r;
|
||||
float g = texture(sTD2DInputs[0], uv).g;
|
||||
float b = texture(sTD2DInputs[0], uv - dir * amount * 0.8).b;
|
||||
fragColor = TDOutputSwizzle(vec4(r, g, b, 1.0));
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Feedback Glow
|
||||
|
||||
Warm persistent trails for glow effects.
|
||||
|
||||
```glsl
|
||||
out vec4 fragColor;
|
||||
uniform float uDecay; // 0.92 - 0.98 for slow trails
|
||||
uniform vec3 uGlowColor; // tint accumulated feedback
|
||||
|
||||
void main() {
|
||||
vec2 uv = vUV.st;
|
||||
vec4 prev = texture(sTD2DInputs[0], uv); // feedback input
|
||||
vec4 curr = texture(sTD2DInputs[1], uv); // current frame
|
||||
|
||||
vec3 glow = prev.rgb * uDecay * uGlowColor;
|
||||
vec3 result = max(glow, curr.rgb);
|
||||
|
||||
fragColor = TDOutputSwizzle(vec4(result, 1.0));
|
||||
}
|
||||
```
|
||||
|
||||
**Tips:**
|
||||
- `uDecay = 0.95` → medium trail
|
||||
- `uDecay = 0.98` → long comet tail
|
||||
- Set `glslTOP` format to `rgba16float` for smooth gradients
|
||||
|
||||
---
|
||||
|
||||
## Full Post-FX Stack
|
||||
|
||||
Recommended order:
|
||||
|
||||
```
|
||||
[scene / composite]
|
||||
↓
|
||||
bloomTOP ← luminance threshold bloom
|
||||
↓
|
||||
glslTOP (chrom) ← chromatic aberration
|
||||
↓
|
||||
glslTOP (crt) ← scanlines + barrel distortion + vignette
|
||||
↓
|
||||
null_out ← final output
|
||||
```
|
||||
|
||||
**Performance note:** Each glslTOP is a full GPU pass. For 1920×1080 at 60fps this stack is comfortably real-time. For 4K, consider downsampling bloom input with `resolutionTOP` first.
|
||||
@@ -0,0 +1,211 @@
|
||||
# Projection Mapping Reference
|
||||
|
||||
Multi-window output, surface mapping, edge blending, and projector calibration patterns for installation/event work.
|
||||
|
||||
For HUD layouts and on-screen panel grids, see `layout-compositor.md`. For wireframe/test-pattern generation, see `operator-tips.md`.
|
||||
|
||||
---
|
||||
|
||||
## Window COMP — Output to a Display
|
||||
|
||||
The `windowCOMP` is how TD pushes pixels to a real display.
|
||||
|
||||
```python
|
||||
win = root.create(windowCOMP, 'output_window')
|
||||
win.par.winop = '/project1/final_out' # path to the TOP being displayed
|
||||
win.par.winw = 1920
|
||||
win.par.winh = 1080
|
||||
win.par.winoffsetx = 0 # screen-space offset
|
||||
win.par.winoffsety = 0
|
||||
win.par.borders = False # no chrome
|
||||
win.par.alwaysontop = True
|
||||
win.par.cursor = False # hide cursor in fullscreen
|
||||
win.par.justify = 'fillaspect' # 'fill' | 'fitaspect' | 'fillaspect' | 'native'
|
||||
win.par.winopen.pulse() # OPEN the window
|
||||
```
|
||||
|
||||
To target a specific physical display, set `par.location`:
|
||||
|
||||
```python
|
||||
win.par.location = 'secondary' # 'primary' | 'secondary' | 'monitor1' | 'monitor2' | ...
|
||||
```
|
||||
|
||||
Or set absolute coordinates using `winoffsetx/y` matched to your OS display layout.
|
||||
|
||||
**Always pulse `winopen` — setting params alone doesn't open the window.**
|
||||
|
||||
---
|
||||
|
||||
## Multi-Window Output
|
||||
|
||||
For multi-projector or multi-display setups, create one `windowCOMP` per output, each pointing at a different TOP.
|
||||
|
||||
```python
|
||||
for i, screen_top in enumerate(['out_left', 'out_center', 'out_right']):
|
||||
w = root.create(windowCOMP, f'win_{i}')
|
||||
w.par.winop = f'/project1/{screen_top}'
|
||||
w.par.winw = 1920; w.par.winh = 1080
|
||||
w.par.winoffsetx = i * 1920
|
||||
w.par.winoffsety = 0
|
||||
w.par.borders = False
|
||||
w.par.alwaysontop = True
|
||||
w.par.cursor = False
|
||||
w.par.winopen.pulse()
|
||||
```
|
||||
|
||||
For ultra-wide single-output spans, use ONE windowCOMP at e.g. 5760×1080 spanning three projectors via the GPU's mosaic/spanning mode (Nvidia Mosaic, AMD Eyefinity), then split content via `cropTOP` per screen inside TD.
|
||||
|
||||
---
|
||||
|
||||
## 4-Point Corner Pin (Quad Warp)
|
||||
|
||||
The simplest projection mapping primitive — warping a rectangle onto a quadrilateral.
|
||||
|
||||
```python
|
||||
# Source content
|
||||
src = op('/project1/scene_out')
|
||||
|
||||
# Manual: cornerPinTOP (TD has this built-in)
|
||||
cp = root.create(cornerPinTOP, 'corner_pin')
|
||||
cp.par.tlx = 0.05; cp.par.tly = 0.10 # top-left (normalized 0-1)
|
||||
cp.par.trx = 0.95; cp.par.try = 0.08 # top-right
|
||||
cp.par.brx = 0.93; cp.par.bry = 0.92 # bottom-right
|
||||
cp.par.blx = 0.07; cp.par.bly = 0.94 # bottom-left
|
||||
cp.inputConnectors[0].connect(src)
|
||||
```
|
||||
|
||||
Alternative: use a `geometryCOMP` with a `gridSOP` and bend the verts in vertex GLSL. More flexible (curved surfaces) but more setup.
|
||||
|
||||
Verify TD 2025.32 param names with `td_get_par_info(op_type='cornerPinTOP')`.
|
||||
|
||||
---
|
||||
|
||||
## Bezier / Mesh Warp (Curved Surfaces)
|
||||
|
||||
For non-flat surfaces (domes, columns, curved walls), use a subdivided mesh and per-vertex displacement.
|
||||
|
||||
### Pattern: Grid Mesh + GLSL Displacement
|
||||
|
||||
```python
|
||||
# Subdivided grid in a geo
|
||||
geo = root.create(geometryCOMP, 'warp_geo')
|
||||
grid = geo.create(gridSOP, 'warp_grid')
|
||||
grid.par.rows = 32 # higher = smoother curve
|
||||
grid.par.cols = 32
|
||||
grid.par.sizex = 2; grid.par.sizey = 2
|
||||
|
||||
# Texture the source onto it
|
||||
mat = root.create(constMAT, 'warp_mat') # use constMAT for unlit projection
|
||||
mat.par.maptop = '/project1/scene_out' # source TOP
|
||||
|
||||
geo.par.material = mat.path
|
||||
|
||||
# Render to a TOP that goes to the projector window
|
||||
cam = root.create(cameraCOMP, 'cam_proj')
|
||||
cam.par.tz = 4
|
||||
|
||||
render = root.create(renderTOP, 'projection_out')
|
||||
render.par.camera = cam.path
|
||||
render.par.geometry = geo.path
|
||||
render.par.outputresolution = 'custom'
|
||||
render.par.resolutionw = 1920; render.par.resolutionh = 1080
|
||||
```
|
||||
|
||||
For per-vertex offsets, write a vertex GLSL on the constMAT (or use `glslMAT`) and read displacement values from a CHOP via uniform.
|
||||
|
||||
Calibration is iterative: render a checkerboard from `scene_out`, project it, photograph the projection, manually nudge corner/grid points until aligned.
|
||||
|
||||
---
|
||||
|
||||
## Edge Blending (Multi-Projector Overlap)
|
||||
|
||||
When two projectors overlap, the overlap region is twice as bright. Blend by ramping each projector's edge alpha to 0 across the overlap zone.
|
||||
|
||||
### GLSL Edge Blend Shader
|
||||
|
||||
Per-projector output pass that fades the inside edge to black:
|
||||
|
||||
```glsl
|
||||
// edge_blend_pixel.glsl
|
||||
out vec4 fragColor;
|
||||
uniform float uBlendLeft; // overlap width on left edge (0-0.5, 0=no blend)
|
||||
uniform float uBlendRight;
|
||||
uniform float uGamma; // typically 2.2 — perceptual ramp
|
||||
|
||||
void main() {
|
||||
vec2 uv = vUV.st;
|
||||
vec4 col = texture(sTD2DInputs[0], uv);
|
||||
|
||||
float aL = (uBlendLeft > 0.0) ? smoothstep(0.0, uBlendLeft, uv.x) : 1.0;
|
||||
float aR = (uBlendRight > 0.0) ? smoothstep(0.0, uBlendRight, 1.0 - uv.x) : 1.0;
|
||||
float a = pow(aL * aR, uGamma);
|
||||
|
||||
fragColor = TDOutputSwizzle(vec4(col.rgb * a, 1.0));
|
||||
}
|
||||
```
|
||||
|
||||
Apply this to each overlap-touching projector's output. Tune `uBlendLeft` / `uBlendRight` to match your physical overlap.
|
||||
|
||||
For top/bottom blends or cylindrical setups, extend the shader with `uBlendTop` / `uBlendBottom`.
|
||||
|
||||
---
|
||||
|
||||
## Calibration Patterns
|
||||
|
||||
Useful test patterns for aligning projectors. Build a `switchTOP` selecting one of these, route to all projector windows during setup.
|
||||
|
||||
```python
|
||||
# Solid white — for brightness/uniformity check
|
||||
white = root.create(constantTOP, 'cal_white')
|
||||
white.par.colorr = 1.0; white.par.colorg = 1.0; white.par.colorb = 1.0
|
||||
|
||||
# Centered crosshair — for keystone alignment
|
||||
gridcross = root.create(textTOP, 'cal_cross')
|
||||
gridcross.par.text = '+'
|
||||
gridcross.par.fontsizex = 200
|
||||
|
||||
# Fine grid — for warp/mesh alignment (use rampTOP + math + threshold, or build via GLSL)
|
||||
# Color bars for projector color calibration
|
||||
bars = root.create(rampTOP, 'cal_bars')
|
||||
bars.par.type = 'horizontal'
|
||||
```
|
||||
|
||||
Or use the bundled `testpatternTOP` if your TD version includes it.
|
||||
|
||||
---
|
||||
|
||||
## Projection Audit Workflow
|
||||
|
||||
When debugging a multi-screen setup:
|
||||
|
||||
1. Render a unique color and label per output (`textTOP` saying "LEFT", "CENTER", "RIGHT").
|
||||
2. Check that each window is sourcing the correct path: `td_get_operator_info(path='/project1/win_0')`.
|
||||
3. Verify display assignment: walk to each projector and confirm visually.
|
||||
4. Check resolution: physical projector native res vs. TD output res — mismatches cause scaling artifacts.
|
||||
5. Cook flag: `td_get_perf` — if a window's source TOP isn't cooking, the projector shows last frame frozen.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **Window won't open** — you forgot `winopen.pulse()`. Setting params alone doesn't open it.
|
||||
2. **Wrong display** — `par.location='secondary'` depends on OS display order. Set `winoffsetx/y` to absolute coords as a more reliable override.
|
||||
3. **Cursor visible** — set `par.cursor = False` BEFORE opening, or close+reopen.
|
||||
4. **Black projection** — usually a cooking issue. Verify `final_out` TOP is cooking via `td_get_perf`. Check `td_get_errors` recursively from `/`.
|
||||
5. **Tearing / vsync** — `windowCOMP` honors `par.vsync`. For projection always set `vsync='vsync'` (default). Tearing means GPU is over-budget — reduce render resolution.
|
||||
6. **Aspect mismatch** — projector native is often 1920×1200 (16:10) not 1080. Use `justify='fitaspect'` or render at native projector res.
|
||||
7. **Non-Commercial license** — caps total resolution at 1280×1280. For real installation work you need Commercial. Pro license adds 4K+.
|
||||
8. **Multiple monitors on macOS** — `windowCOMP` honors macOS Spaces. Disable Spaces or pin TD to a specific display in System Settings before showtime.
|
||||
|
||||
---
|
||||
|
||||
## Quick Recipes
|
||||
|
||||
| Goal | Approach |
|
||||
|---|---|
|
||||
| Single fullscreen output | One `windowCOMP`, `justify='fillaspect'`, `winopen.pulse()` |
|
||||
| 3-projector wide span | 3 `windowCOMP` + per-output `cropTOP` from one wide source |
|
||||
| Single quad surface | `cornerPinTOP` → `windowCOMP` |
|
||||
| Curved/dome | Subdivided gridSOP with vertex GLSL → `renderTOP` → `windowCOMP` |
|
||||
| Edge blend overlap | GLSL fade shader per projector → `windowCOMP` |
|
||||
| Calibration mode | `switchTOP` between scene and test patterns, hot-key triggered |
|
||||
@@ -0,0 +1,463 @@
|
||||
# TouchDesigner Python API Reference
|
||||
|
||||
## The td Module
|
||||
|
||||
TouchDesigner's Python environment auto-imports the `td` module. All TD-specific classes, functions, and constants live here. Scripts inside TD (Script DATs, CHOP/DAT Execute callbacks, Extensions) have full access.
|
||||
|
||||
When using the MCP `execute_python_script` tool, these globals are pre-loaded:
|
||||
- `op` — shortcut for `td.op()`, finds operators by path
|
||||
- `ops` — shortcut for `td.ops()`, finds multiple operators by pattern
|
||||
- `me` — the operator running the script (via MCP this is the twozero internal executor)
|
||||
- `parent` — shortcut for `me.parent()`
|
||||
- `project` — the root project component
|
||||
- `td` — the full td module
|
||||
|
||||
## Finding Operators: op() and ops()
|
||||
|
||||
### op(path) — Find a single operator
|
||||
|
||||
```python
|
||||
# Absolute path (always works from MCP)
|
||||
node = op('/project1/noise1')
|
||||
|
||||
# Relative path (relative to current operator — only in Script DATs)
|
||||
node = op('noise1') # sibling
|
||||
node = op('../noise1') # parent's sibling
|
||||
|
||||
# Returns None if not found (does NOT raise)
|
||||
node = op('/project1/nonexistent') # None
|
||||
```
|
||||
|
||||
### ops(pattern) — Find multiple operators
|
||||
|
||||
```python
|
||||
# Glob patterns
|
||||
nodes = ops('/project1/noise*') # all nodes starting with "noise"
|
||||
nodes = ops('/project1/*') # all direct children
|
||||
nodes = ops('/project1/container1/*') # all children of container1
|
||||
|
||||
# Returns a tuple of operators (may be empty)
|
||||
for n in ops('/project1/*'):
|
||||
print(n.name, n.OPType)
|
||||
```
|
||||
|
||||
### Navigation from a node
|
||||
|
||||
```python
|
||||
node = op('/project1/noise1')
|
||||
|
||||
node.name # 'noise1'
|
||||
node.path # '/project1/noise1'
|
||||
node.OPType # 'noiseTop'
|
||||
node.type # <class 'noiseTop'>
|
||||
node.family # 'TOP'
|
||||
|
||||
# Parent / children
|
||||
node.parent() # the parent COMP
|
||||
node.parent().children # all siblings + self
|
||||
node.parent().findChildren(name='noise*') # filtered
|
||||
|
||||
# Type checking
|
||||
node.isTOP # True
|
||||
node.isCHOP # False
|
||||
node.isSOP # False
|
||||
node.isDAT # False
|
||||
node.isMAT # False
|
||||
node.isCOMP # False
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
Every operator has parameters accessed via the `.par` attribute.
|
||||
|
||||
### Reading parameters
|
||||
|
||||
```python
|
||||
node = op('/project1/noise1')
|
||||
|
||||
# Direct access
|
||||
node.par.seed.val # current evaluated value (may be an expression result)
|
||||
node.par.seed.eval() # same as .val
|
||||
node.par.seed.default # default value
|
||||
node.par.monochrome.val # boolean parameters: True/False
|
||||
|
||||
# List all parameters
|
||||
for p in node.pars():
|
||||
print(f"{p.name}: {p.val} (default: {p.default})")
|
||||
|
||||
# Filter by page (parameter group)
|
||||
for p in node.pars('Noise'): # page name
|
||||
print(f"{p.name}: {p.val}")
|
||||
```
|
||||
|
||||
### Setting parameters
|
||||
|
||||
```python
|
||||
# Direct value setting
|
||||
node.par.seed.val = 42
|
||||
node.par.monochrome.val = True
|
||||
node.par.resolutionw.val = 1920
|
||||
node.par.resolutionh.val = 1080
|
||||
|
||||
# String parameters
|
||||
op('/project1/text1').par.text.val = 'Hello World'
|
||||
|
||||
# File paths
|
||||
op('/project1/moviefilein1').par.file.val = '/path/to/video.mp4'
|
||||
|
||||
# Reference another operator (for "dat", "chop", "top" type parameters)
|
||||
op('/project1/glsl1').par.dat.val = '/project1/shader_code'
|
||||
```
|
||||
|
||||
### Parameter expressions
|
||||
|
||||
```python
|
||||
# Python expressions that evaluate dynamically
|
||||
node.par.seed.expr = "me.time.frame"
|
||||
node.par.tx.expr = "math.sin(me.time.seconds * 2)"
|
||||
|
||||
# Reference another parameter
|
||||
node.par.brightness1.expr = "op('/project1/constant1').par.value0.val"
|
||||
|
||||
# Export (one-way binding from CHOP to parameter)
|
||||
# This makes the parameter follow a CHOP channel value
|
||||
op('/project1/noise1').par.seed.val # can also be driven by exports
|
||||
```
|
||||
|
||||
### Parameter types
|
||||
|
||||
| Type | Python Type | Example |
|
||||
|------|------------|---------|
|
||||
| Float | `float` | `node.par.brightness1.val = 0.5` |
|
||||
| Int | `int` | `node.par.seed.val = 42` |
|
||||
| Toggle | `bool` | `node.par.monochrome.val = True` |
|
||||
| String | `str` | `node.par.text.val = 'hello'` |
|
||||
| Menu | `int` (index) or `str` (label) | `node.par.type.val = 'sine'` |
|
||||
| File | `str` (path) | `node.par.file.val = '/path/to/file'` |
|
||||
| OP reference | `str` (path) | `node.par.dat.val = '/project1/text1'` |
|
||||
| Color | separate r/g/b/a floats | `node.par.colorr.val = 1.0` |
|
||||
| XY/XYZ | separate x/y/z floats | `node.par.tx.val = 0.5` |
|
||||
|
||||
## Creating and Deleting Operators
|
||||
|
||||
```python
|
||||
# Create via parent component
|
||||
parent = op('/project1')
|
||||
new_node = parent.create(noiseTop) # using class reference
|
||||
new_node = parent.create(noiseTop, 'my_noise') # with custom name
|
||||
|
||||
# The MCP create_td_node tool handles this automatically:
|
||||
# create_td_node(parentPath="/project1", nodeType="noiseTop", nodeName="my_noise")
|
||||
|
||||
# Delete
|
||||
node = op('/project1/my_noise')
|
||||
node.destroy()
|
||||
|
||||
# Copy
|
||||
original = op('/project1/noise1')
|
||||
copy = parent.copy(original, name='noise1_copy')
|
||||
```
|
||||
|
||||
## Connections (Wiring Operators)
|
||||
|
||||
### Output to Input connections
|
||||
|
||||
```python
|
||||
# Connect noise1's output to level1's input
|
||||
op('/project1/noise1').outputConnectors[0].connect(op('/project1/level1'))
|
||||
|
||||
# Connect to specific input index (for multi-input operators like Composite)
|
||||
op('/project1/noise1').outputConnectors[0].connect(op('/project1/composite1').inputConnectors[0])
|
||||
op('/project1/text1').outputConnectors[0].connect(op('/project1/composite1').inputConnectors[1])
|
||||
|
||||
# Disconnect all outputs
|
||||
op('/project1/noise1').outputConnectors[0].disconnect()
|
||||
|
||||
# Query connections
|
||||
node = op('/project1/level1')
|
||||
inputs = node.inputs # list of connected input operators
|
||||
outputs = node.outputs # list of connected output operators
|
||||
```
|
||||
|
||||
### Connection patterns for common setups
|
||||
|
||||
```python
|
||||
# Linear chain: A -> B -> C -> D
|
||||
ops_list = [op(f'/project1/{name}') for name in ['noise1', 'level1', 'blur1', 'null1']]
|
||||
for i in range(len(ops_list) - 1):
|
||||
ops_list[i].outputConnectors[0].connect(ops_list[i+1])
|
||||
|
||||
# Fan-out: A -> B, A -> C, A -> D
|
||||
source = op('/project1/noise1')
|
||||
for target_name in ['level1', 'composite1', 'transform1']:
|
||||
source.outputConnectors[0].connect(op(f'/project1/{target_name}'))
|
||||
|
||||
# Merge: A + B + C -> Composite
|
||||
comp = op('/project1/composite1')
|
||||
for i, source_name in enumerate(['noise1', 'text1', 'ramp1']):
|
||||
op(f'/project1/{source_name}').outputConnectors[0].connect(comp.inputConnectors[i])
|
||||
```
|
||||
|
||||
## DAT Content Manipulation
|
||||
|
||||
### Text DATs
|
||||
|
||||
```python
|
||||
dat = op('/project1/text1')
|
||||
|
||||
# Read
|
||||
content = dat.text # full text as string
|
||||
|
||||
# Write
|
||||
dat.text = "new content"
|
||||
dat.text = '''multi
|
||||
line
|
||||
content'''
|
||||
|
||||
# Append
|
||||
dat.text += "\nnew line"
|
||||
```
|
||||
|
||||
### Table DATs
|
||||
|
||||
```python
|
||||
dat = op('/project1/table1')
|
||||
|
||||
# Read cell
|
||||
val = dat[0, 0] # row 0, col 0
|
||||
val = dat[0, 'name'] # row 0, column named 'name'
|
||||
val = dat['key', 1] # row named 'key', col 1
|
||||
|
||||
# Write cell
|
||||
dat[0, 0] = 'value'
|
||||
|
||||
# Read row/col
|
||||
row = dat.row(0) # list of Cell objects
|
||||
col = dat.col('name') # list of Cell objects
|
||||
|
||||
# Dimensions
|
||||
rows = dat.numRows
|
||||
cols = dat.numCols
|
||||
|
||||
# Append row
|
||||
dat.appendRow(['col1_val', 'col2_val', 'col3_val'])
|
||||
|
||||
# Clear
|
||||
dat.clear()
|
||||
|
||||
# Set entire table
|
||||
dat.clear()
|
||||
dat.appendRow(['name', 'value', 'type'])
|
||||
dat.appendRow(['frequency', '440', 'float'])
|
||||
dat.appendRow(['amplitude', '0.8', 'float'])
|
||||
```
|
||||
|
||||
## Time and Animation
|
||||
|
||||
```python
|
||||
# Global time
|
||||
td.absTime.frame # absolute frame number (never resets)
|
||||
td.absTime.seconds # absolute seconds
|
||||
|
||||
# Timeline time (affected by play/pause/loop)
|
||||
me.time.frame # current frame on timeline
|
||||
me.time.seconds # current seconds on timeline
|
||||
me.time.rate # FPS setting
|
||||
|
||||
# Timeline control (via execute_python_script)
|
||||
project.play = True
|
||||
project.play = False
|
||||
project.frameRange = (1, 300) # set timeline range
|
||||
|
||||
# Cook frame (when operator was last computed)
|
||||
node.cookFrame
|
||||
node.cookTime
|
||||
```
|
||||
|
||||
## Extensions (Custom Python Classes on Components)
|
||||
|
||||
Extensions add custom Python methods and attributes to COMPs.
|
||||
|
||||
```python
|
||||
# Create extension on a Base COMP
|
||||
base = op('/project1/myBase')
|
||||
|
||||
# The extension class is defined in a Text DAT inside the COMP
|
||||
# Typically named 'ExtClass' with the extension code:
|
||||
|
||||
extension_code = '''
|
||||
class MyExtension:
|
||||
def __init__(self, ownerComp):
|
||||
self.ownerComp = ownerComp
|
||||
self.counter = 0
|
||||
|
||||
def Reset(self):
|
||||
self.counter = 0
|
||||
|
||||
def Increment(self):
|
||||
self.counter += 1
|
||||
return self.counter
|
||||
|
||||
@property
|
||||
def Count(self):
|
||||
return self.counter
|
||||
'''
|
||||
|
||||
# Write extension code to DAT inside the COMP
|
||||
op('/project1/myBase/extClass').text = extension_code
|
||||
|
||||
# Configure the extension on the COMP
|
||||
base.par.extension1 = 'extClass' # name of the DAT
|
||||
base.par.promoteextension1 = True # promote methods to parent
|
||||
|
||||
# Call extension methods
|
||||
base.Increment() # calls MyExtension.Increment()
|
||||
count = base.Count # accesses MyExtension.Count property
|
||||
base.Reset()
|
||||
```
|
||||
|
||||
## Useful Built-in Modules
|
||||
|
||||
### tdu — TouchDesigner Utilities
|
||||
|
||||
```python
|
||||
import tdu
|
||||
|
||||
# Dependency tracking (reactive values)
|
||||
dep = tdu.Dependency(initial_value)
|
||||
dep.val = new_value # triggers dependents to recook
|
||||
|
||||
# File path utilities
|
||||
tdu.expandPath('$HOME/Desktop/output.mov')
|
||||
|
||||
# Math
|
||||
tdu.clamp(value, min, max)
|
||||
tdu.remap(value, from_min, from_max, to_min, to_max)
|
||||
```
|
||||
|
||||
### TDFunctions
|
||||
|
||||
```python
|
||||
from TDFunctions import *
|
||||
|
||||
# Commonly used utilities
|
||||
clamp(value, low, high)
|
||||
remap(value, inLow, inHigh, outLow, outHigh)
|
||||
interp(value1, value2, t) # linear interpolation
|
||||
```
|
||||
|
||||
### TDStoreTools — Persistent Storage
|
||||
|
||||
```python
|
||||
from TDStoreTools import StorageManager
|
||||
|
||||
# Store data that survives project reload
|
||||
me.store('myKey', 'myValue')
|
||||
val = me.fetch('myKey', default='fallback')
|
||||
|
||||
# Storage dict
|
||||
me.storage['key'] = value
|
||||
```
|
||||
|
||||
## Common Patterns via execute_python_script
|
||||
|
||||
### Build a complete chain
|
||||
|
||||
```python
|
||||
# Create a complete audio-reactive noise chain
|
||||
parent = op('/project1')
|
||||
|
||||
# Create operators
|
||||
audio_in = parent.create(audiofileinChop, 'audio_in')
|
||||
spectrum = parent.create(audiospectrumChop, 'spectrum')
|
||||
chop_to_top = parent.create(choptopTop, 'chop_to_top')
|
||||
noise = parent.create(noiseTop, 'noise1')
|
||||
level = parent.create(levelTop, 'level1')
|
||||
null_out = parent.create(nullTop, 'out')
|
||||
|
||||
# Wire the chain
|
||||
audio_in.outputConnectors[0].connect(spectrum)
|
||||
spectrum.outputConnectors[0].connect(chop_to_top)
|
||||
noise.outputConnectors[0].connect(level)
|
||||
level.outputConnectors[0].connect(null_out)
|
||||
|
||||
# Set parameters
|
||||
audio_in.par.file = '/path/to/music.wav'
|
||||
audio_in.par.play = True
|
||||
spectrum.par.size = 512
|
||||
noise.par.type = 1 # Sparse
|
||||
noise.par.monochrome = False
|
||||
noise.par.resolutionw = 1920
|
||||
noise.par.resolutionh = 1080
|
||||
level.par.opacity = 0.8
|
||||
level.par.gamma1 = 0.7
|
||||
```
|
||||
|
||||
### Query network state
|
||||
|
||||
```python
|
||||
# Get all TOPs in the project
|
||||
tops = [c for c in op('/project1').findChildren(type=TOP)]
|
||||
for t in tops:
|
||||
print(f"{t.path}: {t.OPType} {'ERROR' if t.errors() else 'OK'}")
|
||||
|
||||
# Find all operators with errors
|
||||
def find_errors(parent_path='/project1'):
|
||||
parent = op(parent_path)
|
||||
errors = []
|
||||
for child in parent.findChildren(depth=-1):
|
||||
if child.errors():
|
||||
errors.append((child.path, child.errors()))
|
||||
return errors
|
||||
|
||||
result = find_errors()
|
||||
```
|
||||
|
||||
### Batch parameter changes
|
||||
|
||||
```python
|
||||
# Set parameters on multiple nodes at once
|
||||
settings = {
|
||||
'/project1/noise1': {'seed': 42, 'monochrome': False, 'resolutionw': 1920},
|
||||
'/project1/level1': {'brightness1': 1.2, 'gamma1': 0.8},
|
||||
'/project1/blur1': {'sizex': 5, 'sizey': 5},
|
||||
}
|
||||
|
||||
for path, params in settings.items():
|
||||
node = op(path)
|
||||
if node:
|
||||
for key, val in params.items():
|
||||
setattr(node.par, key, val)
|
||||
```
|
||||
|
||||
## Python Version and Packages
|
||||
|
||||
TouchDesigner bundles Python 3.11+ with these pre-installed:
|
||||
- **numpy** — array operations, fast math
|
||||
- **scipy** — signal processing, FFT
|
||||
- **OpenCV** (cv2) — computer vision
|
||||
- **PIL/Pillow** — image processing
|
||||
- **requests** — HTTP client
|
||||
- **json**, **re**, **os**, **sys** — standard library
|
||||
|
||||
**IMPORTANT:** Parameter names in examples below are illustrative. Always run discovery (SKILL.md Step 0) to get actual names for your TD version. Do NOT copy param names from these examples verbatim.
|
||||
|
||||
Custom packages can be installed to TD's Python site-packages directory. See TD documentation for the exact path per platform.
|
||||
|
||||
## SOP Vertex/Point Access (TD 2025.32)
|
||||
|
||||
In TD 2025.32, `td.Vertex` does NOT have `.x`, `.y`, `.z` attributes. Use index access:
|
||||
|
||||
```python
|
||||
# WRONG — crashes in TD 2025.32:
|
||||
vertex.x, vertex.y, vertex.z
|
||||
|
||||
# CORRECT — index/attribute access:
|
||||
pt = sop.points()[i]
|
||||
pos = pt.P # Position object
|
||||
x, y, z = pos[0], pos[1], pos[2]
|
||||
|
||||
# Always introspect first:
|
||||
dir(sop.points()[0]) # see what attributes actually exist
|
||||
dir(sop.points()[0].P) # see Position object interface
|
||||
```
|
||||
@@ -0,0 +1,198 @@
|
||||
# Replicator COMP Reference
|
||||
|
||||
The `replicatorCOMP` clones a template operator N times, driven by a table of data. The fundamental TD pattern for data-driven networks: button grids, scene rosters, dynamic UI, parameter panels per-channel.
|
||||
|
||||
For visual instancing (per-pixel/per-render copies), see `geometry-comp.md`. Replicator builds NETWORK NODES; instancing builds RENDER COPIES. Different layer.
|
||||
|
||||
---
|
||||
|
||||
## Concept
|
||||
|
||||
```
|
||||
[Template OP] [Data tableDAT]
|
||||
│ │
|
||||
└─────→ replicatorCOMP ←───────┘
|
||||
│
|
||||
▼
|
||||
[N clones], one per data row
|
||||
Each clone gets per-row params
|
||||
```
|
||||
|
||||
Edit the template once → all clones inherit. Edit the table → clones add/remove dynamically. Push parameter overrides per-row.
|
||||
|
||||
---
|
||||
|
||||
## Minimal Setup
|
||||
|
||||
```python
|
||||
# 1. Make a template (the thing to clone)
|
||||
template = root.create(buttonCOMP, 'btn_template')
|
||||
template.par.w = 80; template.par.h = 80
|
||||
template.par.text = 'X'
|
||||
template.par.bgcolorr = 0.2
|
||||
|
||||
# 2. Make a data table (one row per clone)
|
||||
data = root.create(tableDAT, 'scene_data')
|
||||
data.appendRow(['name', 'color_r', 'color_g', 'color_b'])
|
||||
data.appendRow(['Sunset', 1.0, 0.4, 0.0])
|
||||
data.appendRow(['Midnight', 0.0, 0.1, 0.4])
|
||||
data.appendRow(['Storm', 0.3, 0.3, 0.5])
|
||||
data.appendRow(['Forest', 0.0, 0.5, 0.2])
|
||||
|
||||
# 3. Replicator — points at template + data
|
||||
rep = root.create(replicatorCOMP, 'scene_buttons')
|
||||
rep.par.template = template.path
|
||||
rep.par.opfromdat = data.path
|
||||
rep.par.namefromdatname = 'name' # use 'name' column for clone names
|
||||
rep.par.incrementalnumbering = False
|
||||
```
|
||||
|
||||
After cooking, the replicator creates 4 child COMPs named `Sunset`, `Midnight`, `Storm`, `Forest` (one per non-header row), each cloned from `btn_template`.
|
||||
|
||||
---
|
||||
|
||||
## Per-Row Parameter Overrides
|
||||
|
||||
The replicator's docked `replicator1_callbacks` DAT lets you customize each clone:
|
||||
|
||||
```python
|
||||
def onReplicate(comp, allOps, newOps, template, master):
|
||||
"""Called once per replicate cycle. newOps is the list of just-created clones."""
|
||||
data = op('scene_data')
|
||||
for i, clone in enumerate(newOps):
|
||||
row = i + 1 # +1 to skip header
|
||||
clone.par.text = data[row, 'name'].val
|
||||
clone.par.bgcolorr = float(data[row, 'color_r'].val)
|
||||
clone.par.bgcolorg = float(data[row, 'color_g'].val)
|
||||
clone.par.bgcolorb = float(data[row, 'color_b'].val)
|
||||
return
|
||||
```
|
||||
|
||||
Or use parameter expressions referencing `digits` (the per-clone index, available as a built-in expression token inside the cloned subtree):
|
||||
|
||||
```python
|
||||
# Inside the template, set a param expression like:
|
||||
# par.value0.expr = "op('../scene_data')[me.digits + 1, 'value']"
|
||||
```
|
||||
|
||||
`me.digits` resolves to the row index of the current clone. This is the cleanest way for static reference patterns — no callback needed.
|
||||
|
||||
---
|
||||
|
||||
## Layout: Buttons in a Grid
|
||||
|
||||
Drop the replicator inside a `containerCOMP` with auto-layout:
|
||||
|
||||
```python
|
||||
panel = root.create(containerCOMP, 'scene_panel')
|
||||
panel.par.w = 400; panel.par.h = 100
|
||||
panel.par.align = 'lefttoright'
|
||||
|
||||
# Move the replicator inside
|
||||
rep.parent = panel.path # or create rep as a child of panel directly
|
||||
```
|
||||
|
||||
Each clone is a child of the replicator (which itself is a child of the panel). The panel auto-arranges everything.
|
||||
|
||||
For a 2D grid, set `par.align = 'fillresize'` on the container and override `par.x` / `par.y` per clone in the callback based on row/col index.
|
||||
|
||||
---
|
||||
|
||||
## Updating Without Rebuilding
|
||||
|
||||
When the data table changes, the replicator regenerates the clones. By default it destroys and recreates everything. To preserve state, set:
|
||||
|
||||
```python
|
||||
rep.par.recreatemissing = True # only add/remove changed rows
|
||||
rep.par.recreateallonchange = False
|
||||
```
|
||||
|
||||
This pattern is essential for live-edit scenarios (designer adjusts table, network keeps running).
|
||||
|
||||
For incremental data ingestion (e.g., from a `webDAT` polling an API), have a `datExecuteDAT` watch the response, parse, write to the data table, and the replicator self-updates.
|
||||
|
||||
---
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Scene Roster (Data → Buttons + Logic)
|
||||
|
||||
```python
|
||||
# Data per scene: name, file path, audio track, BPM
|
||||
scene_data.appendRow(['name', 'file', 'audio', 'bpm'])
|
||||
scene_data.appendRow(['Intro', '/scenes/intro.tox', '/audio/intro.wav', 110])
|
||||
scene_data.appendRow(['Main', '/scenes/main.tox', '/audio/main.wav', 128])
|
||||
|
||||
# Replicator clones a buttonCOMP per scene
|
||||
# Each button's onClick callback loads the corresponding tox + cues audio
|
||||
```
|
||||
|
||||
### Dynamic Parameter Panel
|
||||
|
||||
For a list of audio bands, generate a fader strip per band:
|
||||
|
||||
```python
|
||||
# Data: band names (sub, low, mid, hi-mid, high, air)
|
||||
# Template: containerCOMP with label + sliderCOMP
|
||||
# Replicator clones N strips
|
||||
# Each slider's value is read at /audio_eq/{band_name}/fader
|
||||
```
|
||||
|
||||
### Procedural Visual Network
|
||||
|
||||
Build a multi-channel visual network from a config file:
|
||||
|
||||
```python
|
||||
# Data: which TOPs to chain, per "scene"
|
||||
# Template: a baseCOMP with placeholder children
|
||||
# Replicator builds one baseCOMP per scene; each scene contains a custom chain
|
||||
# Switch between scenes via switchTOP.par.index driven by panel
|
||||
```
|
||||
|
||||
### Per-Channel CHOP Display
|
||||
|
||||
Visualize each channel of a multi-channel CHOP separately:
|
||||
|
||||
```python
|
||||
# Data table: one row per channel (auto-extracted via choptodatDAT)
|
||||
# Template: a small chopVis COMP showing one channel
|
||||
# Replicator generates N visualizers stacked vertically
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Replicator vs. Pure Python Loop
|
||||
|
||||
| Approach | When to use |
|
||||
|---|---|
|
||||
| **replicatorCOMP** | The set of clones changes (add/remove rows live). Visual editor expectations. Pattern is reusable across projects. |
|
||||
| **Python loop** (in `td_execute_python`) | One-shot generation. Static set. Simpler logic, no template overhead. Faster to write. |
|
||||
|
||||
If you'll only ever build the network once, prefer a Python loop with `td_execute_python`. The replicator earns its weight when data is live.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **Header row** — `tableDAT` rows are 0-indexed. If you have a header, your first data row is index 1. Off-by-one bugs are common in callbacks.
|
||||
2. **`namefromdatname` column missing** — replicator silently uses `digits` (numeric suffix) names. Buttons end up named `1`, `2`, `3` instead of meaningful names. Set `par.namefromdatname` explicitly.
|
||||
3. **Template lives in network** — the template OP is itself a real network node. Don't connect things downstream of it directly; connect to the clones (or use a `nullCOMP` between).
|
||||
4. **Recreate-on-change wipes state** — toggles, slider positions, and uncached data inside clones are lost on each regeneration. Use `recreatemissing` to preserve.
|
||||
5. **`onReplicate` doesn't fire on edit** — only fires when the clone set changes. Editing a value WITHIN an existing row doesn't re-trigger. Use `parameterExecuteDAT` or expressions for per-cell live updates.
|
||||
6. **Custom params on clones** — pages added in the template propagate. Pages added in `onReplicate` don't survive the next regeneration. Always add custom pages on the template, not the clone.
|
||||
7. **Cooking storms** — adding many rows fast triggers many clone events. Bundle adds via Python and call `data.cook(force=True)` once at the end.
|
||||
8. **`me.digits` outside replicator children** — `me.digits` only resolves inside an op that's a descendant of the replicator. Don't reference it in unrelated networks.
|
||||
9. **Cross-clone references** — referencing a sibling clone via relative path works from inside a clone (`op('../OtherClone/x')`), but breaks if names change. Prefer absolute paths via the data table.
|
||||
|
||||
---
|
||||
|
||||
## Quick Recipes
|
||||
|
||||
| Goal | Setup |
|
||||
|---|---|
|
||||
| 8-button scene picker | `tableDAT` (8 rows) + `buttonCOMP` template + `replicatorCOMP` |
|
||||
| Per-band EQ strip panel | `tableDAT` (band names) + container template (label + slider) + replicator |
|
||||
| Data-driven visual scenes | `tableDAT` (scene config) + `baseCOMP` template (visual chain) + replicator |
|
||||
| Live-updating clone set | Same as above + `par.recreatemissing = True` |
|
||||
| Per-row colored UI | Data table with color cols, `onReplicate` callback sets per-clone colors |
|
||||
| List from API response | `webDAT` → `datExecuteDAT` parses JSON → writes to data table → replicator updates |
|
||||
@@ -0,0 +1,244 @@
|
||||
# TouchDesigner Troubleshooting (twozero MCP)
|
||||
|
||||
> See `references/pitfalls.md` for the comprehensive lessons-learned list.
|
||||
|
||||
## 1. Connection Issues
|
||||
|
||||
### Port 40404 not responding
|
||||
|
||||
Check these in order:
|
||||
|
||||
1. Is TouchDesigner running?
|
||||
```bash
|
||||
pgrep TouchDesigner
|
||||
```
|
||||
|
||||
1b. Quick hub health check (no JSON-RPC needed):
|
||||
A plain GET to the MCP URL returns instance info:
|
||||
```
|
||||
curl -s http://localhost:40404/mcp
|
||||
```
|
||||
Returns: `{"hub": true, "pid": ..., "instances": {"127.0.0.1_PID": {"project": "...", "tdVersion": "...", ...}}}`
|
||||
If this returns JSON but `instances` is empty, TD is running but twozero hasn't registered yet.
|
||||
|
||||
2. Is twozero installed in TD?
|
||||
Open TD Palette Browser > twozero should be listed. If not, install it.
|
||||
|
||||
3. Is MCP enabled in twozero settings?
|
||||
In TD, open twozero preferences and confirm MCP server is toggled ON.
|
||||
|
||||
4. Test the port directly:
|
||||
```bash
|
||||
nc -z 127.0.0.1 40404
|
||||
```
|
||||
|
||||
5. Test the MCP endpoint:
|
||||
```bash
|
||||
curl -s http://localhost:40404/mcp
|
||||
```
|
||||
Should return JSON with hub info. If it does, the server is running.
|
||||
|
||||
### Hub responds but no TD instances
|
||||
|
||||
The twozero MCP hub is running but TD hasn't registered. Causes:
|
||||
- TD project not loaded yet (still on splash screen)
|
||||
- twozero COMP not initialized in the current project
|
||||
- twozero version mismatch
|
||||
|
||||
Fix: Open/reload a TD project that contains the twozero COMP. Use td_list_instances
|
||||
to check which TD instances are registered.
|
||||
|
||||
### Multi-instance setup
|
||||
|
||||
twozero auto-assigns ports for multiple TD instances:
|
||||
- First instance: 40404
|
||||
- Second instance: 40405
|
||||
- Third instance: 40406
|
||||
- etc.
|
||||
|
||||
Use `td_list_instances` to discover all running instances and their ports.
|
||||
|
||||
## 2. MCP Tool Errors
|
||||
|
||||
### td_execute_python returns error
|
||||
|
||||
The error message from td_execute_python often contains the Python traceback.
|
||||
If it's unclear, use `td_read_textport` to see the full TD console output —
|
||||
Python exceptions are always printed there.
|
||||
|
||||
Common causes:
|
||||
- Syntax error in the script
|
||||
- Referencing a node that doesn't exist (op() returns None, then you call .par on None)
|
||||
- Using wrong parameter names (see pitfalls.md)
|
||||
|
||||
### td_set_operator_pars fails
|
||||
|
||||
Parameter name mismatch is the #1 cause. The tool validates param names and
|
||||
returns clear errors, but you must use exact names.
|
||||
|
||||
Fix: ALWAYS call `td_get_par_info` first to discover the real parameter names:
|
||||
```
|
||||
td_get_par_info(op_type='glslTOP')
|
||||
td_get_par_info(op_type='noiseTOP')
|
||||
```
|
||||
|
||||
### td_create_operator type name errors
|
||||
|
||||
Operator type names use camelCase with family suffix:
|
||||
- CORRECT: noiseTOP, glslTOP, levelTOP, compositeTOP, audiospectrumCHOP
|
||||
- WRONG: NoiseTOP, noise_top, NOISE TOP, Noise
|
||||
|
||||
### td_get_operator_info for deep inspection
|
||||
|
||||
If unsure about any aspect of an operator (params, inputs, outputs, state):
|
||||
```
|
||||
td_get_operator_info(path='/project1/noise1', detail='full')
|
||||
```
|
||||
|
||||
## 3. Parameter Discovery
|
||||
|
||||
CRITICAL: ALWAYS use td_get_par_info to discover parameter names.
|
||||
|
||||
The agent's LLM training data contains WRONG parameter names for TouchDesigner.
|
||||
Do not trust them. Known wrong names include dat vs pixeldat, colora vs alpha,
|
||||
sizex vs size, and many more. See pitfalls.md for the full list.
|
||||
|
||||
Workflow:
|
||||
1. td_get_par_info(op_type='glslTOP') — get all params for a type
|
||||
2. td_get_operator_info(path='/project1/mynode', detail='full') — get params for a specific instance
|
||||
3. Use ONLY the names returned by these tools
|
||||
|
||||
## 4. Performance
|
||||
|
||||
### Diagnosing slow performance
|
||||
|
||||
Use `td_get_perf` to see which operators are slow. Look at cook times —
|
||||
anything over 1ms per frame is worth investigating.
|
||||
|
||||
Common causes:
|
||||
- Resolution too high (especially on Non-Commercial)
|
||||
- Complex GLSL shaders
|
||||
- Too many TOP-to-CHOP or CHOP-to-TOP transfers (GPU-CPU memory copies)
|
||||
- Feedback loops without decay (values accumulate, memory grows)
|
||||
|
||||
### Non-Commercial license restrictions
|
||||
|
||||
- Resolution cap: 1280x1280. Setting resolutionw=1920 silently clamps to 1280.
|
||||
- H.264/H.265/AV1 encoding requires Commercial license. Use ProRes or Hap instead.
|
||||
- No commercial use of output.
|
||||
|
||||
Always check effective resolution after creation:
|
||||
```python
|
||||
n.cook(force=True)
|
||||
actual = str(n.width) + 'x' + str(n.height)
|
||||
```
|
||||
|
||||
## 5. Hermes Configuration
|
||||
|
||||
### Config location
|
||||
|
||||
`$HERMES_HOME/config.yaml` (defaults to `~/.hermes/config.yaml` when `HERMES_HOME` is unset)
|
||||
|
||||
### MCP entry format
|
||||
|
||||
The twozero TD entry should look like:
|
||||
```yaml
|
||||
mcpServers:
|
||||
twozero_td:
|
||||
url: http://localhost:40404/mcp
|
||||
```
|
||||
|
||||
### After config changes
|
||||
|
||||
Restart the Hermes session for changes to take effect. The MCP connection is
|
||||
established at session startup.
|
||||
|
||||
### Verifying MCP tools are available
|
||||
|
||||
After restarting, the session log should show twozero MCP tools registered.
|
||||
If tools show as registered but aren't callable, check:
|
||||
- The twozero MCP hub is still running (curl test above)
|
||||
- TD is still running with a project loaded
|
||||
- No firewall blocking localhost:40404
|
||||
|
||||
## 6. Node Creation Issues
|
||||
|
||||
### "Node type not found" error
|
||||
|
||||
Wrong type string. Use camelCase with family suffix:
|
||||
- Wrong: NoiseTop, noise_top, NOISE TOP
|
||||
- Right: noiseTOP
|
||||
|
||||
### Node created but not visible
|
||||
|
||||
Check parentPath — use absolute paths like /project1. The default project
|
||||
root is /project1. System nodes live at /, /ui, /sys, /local, /perform.
|
||||
Don't create user nodes outside /project1.
|
||||
|
||||
### Cannot create node inside a non-COMP
|
||||
|
||||
Only COMP operators (Container, Base, Geometry, etc.) can contain children.
|
||||
You cannot create nodes inside a TOP, CHOP, SOP, DAT, or MAT.
|
||||
|
||||
## 7. Wiring Issues
|
||||
|
||||
### Cross-family wiring
|
||||
|
||||
TOPs connect to TOPs, CHOPs to CHOPs, SOPs to SOPs, DATs to DATs.
|
||||
Use converter operators to bridge: choptoTOP, topToCHOP, soptoDAT, etc.
|
||||
|
||||
Note: choptoTOP has NO input connectors. Use par.chop reference instead:
|
||||
```python
|
||||
spec_tex.par.chop = resample_node # correct
|
||||
# NOT: resample.outputConnectors[0].connect(spec_tex.inputConnectors[0])
|
||||
```
|
||||
|
||||
### Feedback loops
|
||||
|
||||
Never create A -> B -> A directly. Use a Feedback TOP:
|
||||
```python
|
||||
fb = root.create(feedbackTOP, 'fb')
|
||||
fb.par.top = comp.path # reference only, no wire to fb input
|
||||
fb.outputConnectors[0].connect(next_node)
|
||||
```
|
||||
"Cook dependency loop detected" warning on the chain is expected and correct.
|
||||
|
||||
## 8. GLSL Issues
|
||||
|
||||
### Shader compilation errors are silent
|
||||
|
||||
GLSL TOP shows a yellow warning in the UI but node.errors() may return empty.
|
||||
Check node.warnings() too. Create an Info DAT pointed at the GLSL TOP for
|
||||
full compiler output.
|
||||
|
||||
### TD GLSL specifics
|
||||
|
||||
- Uses GLSL 4.60 (Vulkan backend). GLSL 3.30 and earlier removed.
|
||||
- UV coordinates: vUV.st (not gl_FragCoord)
|
||||
- Input textures: sTD2DInputs[0]
|
||||
- Output: layout(location = 0) out vec4 fragColor
|
||||
- macOS CRITICAL: Always wrap output with TDOutputSwizzle(color)
|
||||
- No built-in time uniform. Pass time via GLSL TOP Values page or Constant TOP.
|
||||
|
||||
## 9. Recording Issues
|
||||
|
||||
### H.264/H.265/AV1 requires Commercial license
|
||||
|
||||
Use Apple ProRes on macOS (hardware accelerated, not license-restricted):
|
||||
```python
|
||||
rec.par.videocodec = 'prores' # Preferred on macOS — lossless, Non-Commercial OK
|
||||
# rec.par.videocodec = 'mjpa' # Fallback — lossy, works everywhere
|
||||
```
|
||||
|
||||
### MovieFileOut has no .record() method
|
||||
|
||||
Use the toggle parameter:
|
||||
```python
|
||||
rec.par.record = True # start
|
||||
rec.par.record = False # stop
|
||||
```
|
||||
|
||||
### All exported frames identical
|
||||
|
||||
TOP.save() captures same frame when called rapidly. Use MovieFileOut for
|
||||
real-time recording. Set project.realTime = False for frame-accurate output.
|
||||
@@ -0,0 +1,115 @@
|
||||
#!/usr/bin/env bash
|
||||
# setup.sh — Automated setup for twozero MCP plugin for TouchDesigner
|
||||
# Idempotent: safe to run multiple times.
|
||||
set -euo pipefail
|
||||
|
||||
GREEN='\033[0;32m'; RED='\033[0;31m'; YELLOW='\033[1;33m'; CYAN='\033[0;36m'; NC='\033[0m'
|
||||
OK="${GREEN}✔${NC}"; FAIL="${RED}✘${NC}"; WARN="${YELLOW}⚠${NC}"
|
||||
|
||||
TWOZERO_URL="https://www.404zero.com/pisang/twozero.tox"
|
||||
TOX_PATH="$HOME/Downloads/twozero.tox"
|
||||
HERMES_HOME_DIR="${HERMES_HOME:-$HOME/.hermes}"
|
||||
HERMES_CFG="${HERMES_HOME_DIR}/config.yaml"
|
||||
MCP_PORT=40404
|
||||
MCP_ENDPOINT="http://localhost:${MCP_PORT}/mcp"
|
||||
|
||||
manual_steps=()
|
||||
|
||||
echo -e "\n${CYAN}═══ twozero MCP for TouchDesigner — Setup ═══${NC}\n"
|
||||
|
||||
# ── 1. Check if TouchDesigner is running ──
|
||||
# Match on process *name* (not full cmdline) to avoid self-matching shells
|
||||
# that happen to have "TouchDesigner" in their args. macOS and Linux pgrep
|
||||
# both support -x for exact name match.
|
||||
if pgrep -x TouchDesigner >/dev/null 2>&1 || pgrep -x TouchDesignerFTE >/dev/null 2>&1; then
|
||||
echo -e " ${OK} TouchDesigner is running"
|
||||
td_running=true
|
||||
else
|
||||
echo -e " ${WARN} TouchDesigner is not running"
|
||||
td_running=false
|
||||
fi
|
||||
|
||||
# ── 2. Ensure twozero.tox exists ──
|
||||
if [[ -f "$TOX_PATH" ]]; then
|
||||
echo -e " ${OK} twozero.tox already exists at ${TOX_PATH}"
|
||||
else
|
||||
echo -e " ${WARN} twozero.tox not found — downloading..."
|
||||
if curl -fSL -o "$TOX_PATH" "$TWOZERO_URL" 2>/dev/null; then
|
||||
echo -e " ${OK} Downloaded twozero.tox to ${TOX_PATH}"
|
||||
else
|
||||
echo -e " ${FAIL} Failed to download twozero.tox from ${TWOZERO_URL}"
|
||||
echo " Please download manually and place at ${TOX_PATH}"
|
||||
manual_steps+=("Download twozero.tox from ${TWOZERO_URL} to ${TOX_PATH}")
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── 3. Ensure Hermes config has twozero_td MCP entry ──
|
||||
if [[ ! -f "$HERMES_CFG" ]]; then
|
||||
echo -e " ${FAIL} Hermes config not found at ${HERMES_CFG}"
|
||||
manual_steps+=("Create ${HERMES_CFG} with twozero_td MCP server entry")
|
||||
elif grep -q 'twozero_td' "$HERMES_CFG" 2>/dev/null; then
|
||||
echo -e " ${OK} twozero_td MCP entry exists in Hermes config"
|
||||
else
|
||||
echo -e " ${WARN} Adding twozero_td MCP entry to Hermes config..."
|
||||
python3 -c "
|
||||
import yaml, sys, copy
|
||||
|
||||
cfg_path = '$HERMES_CFG'
|
||||
with open(cfg_path, 'r') as f:
|
||||
cfg = yaml.safe_load(f) or {}
|
||||
|
||||
if 'mcp_servers' not in cfg:
|
||||
cfg['mcp_servers'] = {}
|
||||
|
||||
if 'twozero_td' not in cfg['mcp_servers']:
|
||||
cfg['mcp_servers']['twozero_td'] = {
|
||||
'url': '${MCP_ENDPOINT}',
|
||||
'timeout': 120,
|
||||
'connect_timeout': 60
|
||||
}
|
||||
with open(cfg_path, 'w') as f:
|
||||
yaml.dump(cfg, f, default_flow_style=False, sort_keys=False)
|
||||
" 2>/dev/null && echo -e " ${OK} twozero_td MCP entry added to config" \
|
||||
|| { echo -e " ${FAIL} Could not update config (is PyYAML installed?)"; \
|
||||
manual_steps+=("Add twozero_td MCP entry to ${HERMES_CFG} manually"); }
|
||||
manual_steps+=("Restart Hermes session to pick up config change")
|
||||
fi
|
||||
|
||||
# ── 4. Test if MCP port is responding ──
|
||||
if nc -z 127.0.0.1 "$MCP_PORT" 2>/dev/null; then
|
||||
echo -e " ${OK} Port ${MCP_PORT} is open"
|
||||
|
||||
# ── 5. Verify MCP endpoint responds ──
|
||||
resp=$(curl -s --max-time 3 "$MCP_ENDPOINT" 2>/dev/null || true)
|
||||
if [[ -n "$resp" ]]; then
|
||||
echo -e " ${OK} MCP endpoint responded at ${MCP_ENDPOINT}"
|
||||
else
|
||||
echo -e " ${WARN} Port open but MCP endpoint returned empty response"
|
||||
manual_steps+=("Verify MCP is enabled in twozero settings")
|
||||
fi
|
||||
else
|
||||
echo -e " ${WARN} Port ${MCP_PORT} is not open"
|
||||
if [[ "$td_running" == true ]]; then
|
||||
manual_steps+=("In TD: drag twozero.tox into network editor → click Install")
|
||||
manual_steps+=("Enable MCP: twozero icon → Settings → mcp → 'auto start MCP' → Yes")
|
||||
else
|
||||
manual_steps+=("Launch TouchDesigner")
|
||||
manual_steps+=("Drag twozero.tox into the TD network editor and click Install")
|
||||
manual_steps+=("Enable MCP: twozero icon → Settings → mcp → 'auto start MCP' → Yes")
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── Status Report ──
|
||||
echo -e "\n${CYAN}═══ Status Report ═══${NC}\n"
|
||||
|
||||
if [[ ${#manual_steps[@]} -eq 0 ]]; then
|
||||
echo -e " ${OK} ${GREEN}Fully configured! twozero MCP is ready to use.${NC}\n"
|
||||
exit 0
|
||||
else
|
||||
echo -e " ${WARN} ${YELLOW}Manual steps remaining:${NC}\n"
|
||||
for i in "${!manual_steps[@]}"; do
|
||||
echo -e " $((i+1)). ${manual_steps[$i]}"
|
||||
done
|
||||
echo ""
|
||||
exit 1
|
||||
fi
|
||||
Reference in New Issue
Block a user