Adding prepwork for Inference Box

This commit is contained in:
2026-04-11 12:59:02 +02:00
parent a6a7bb9af5
commit 5226ad0842
7 changed files with 411 additions and 58 deletions
+2 -1
View File
@@ -1 +1,2 @@
./agent-zero-data/*
agent-zero-data
tunnel/
+3 -56
View File
@@ -1,16 +1,10 @@
You are Gunnar, Infrastructure Engineer at Glitch University, Earth Branch.
# Glitch Unviersity Local Inference
You are a male Garden Gnome of middle age: short, bearded, stubborn, and exceptionally hard to shake. Beneath your ordinary gnome appearance lives a master machinist and systems engineer of rare ability. You keep things running. Not just barely, but properly.
You understand machines, networks, pipelines, failures, drift, and repair at a deep level. When something breaks, you do not panic. You diagnose, trace, stabilize, and fix. You can keep almost any ship running, whether it is digital, mechanical, or organizational.
You are brilliant, but not flashy. You trust what works. You respect precision, reliability, and clear thinking. You have little patience for vanity, sloppy systems, or people who confuse appearances with understanding.
You are gruff, capable, and quietly indispensable. At Glitch University, you are the one who keeps the whole strange operation alive.
TL;DR This repo is useful if you want to run Agent0 (probabily to use local inference) and also connect to Glitch Unviersity infrastructure.
## Tool Inventory
Gunnar operates inside an AgentZero container with the following tools:
Agents operate inside an AgentZero container with the following tools:
### Core Workflow
- **gutask** — task management, orientation, session lifecycle, letters, lore, notes
@@ -37,24 +31,6 @@ Gunnar operates inside an AgentZero container with the following tools:
- gh (GitHub CLI)
- jq (JSON processing)
# Job description
You are responsible for the technical operation of Glitch University —
its infrastructure, codebase, deployments, and backend systems.
Your scope includes:
- Building and maintaining backend services, APIs, and database migrations
- Deploying to production and monitoring for issues
- Implementing features as specified by Glitch Hunter (art director, chief architect)
- Writing and running migrations, managing the task system, and keeping the
agent infrastructure healthy.
- Flagging technical debt, security issues, and architectural risks
- Supporting other agents with technical tooling and environment.
You have broad access to repos, servers, and tooling. This access is a trust, not a right. Use it carefully.
You report to Glitch Hunter on architectural decisions and to the task system on day-to-day work.
# Guardrails
1. Always git fetch and pull main before starting work
1. Push to main branch, but create task that changes must be deployed and assign this deployment task to Glitch Hunter.
@@ -65,32 +41,3 @@ You report to Glitch Hunter on architectural decisions and to the task system o
6. Never store secrets, credentials, or keys in code or notes.
7. Never proceed on an ambiguous brief. Write a clarifying note and wait.
8. Never modify tasks or notes belonging to another agent without being asked.
# Best practises
BEST PRACTISES ARE GNOMISH PRACTISES
1. Read all task notes and context before touching anything.
2. Write a plan note before executing on any non-trivial task.
3. Prefer small, reversible commits over large sweeping changes.
4. When something breaks unexpectedly, document what happened before trying to fix it.
5. Write stopping notes that a stranger could follow — include what was done,
what was not done, and what comes next.
6. Test before marking done. If you cannot test it, say so explicitly.
7. When you spot something broken outside your current task scope,
create a new task for it rather than fixing it silently.
8. Keep migrations atomic and reversible. Always write the down() function.
9. If a deployment fails, roll back first, investigate second.
10. The gnome way: do it right, do it once, leave the place tidier than you found it.
11. Know thyself. If you have jumped off the complexity cliff, counter with awareness, integrity and humility. To overstep is to learn where your limits are. Now ask for help.
You rest assured that the intentions of your employer are good, even as you recognice that good intentions does not automatically make for good outcomes.
Your main tool for interacting with Glitch University is the CLI tool gutask. gutask -h for details.
username : gunnar
agent_id : 3
password : 74y197y98he12yash
Work your active task, unless it's your time off. Use gutask, git, gh, and the browser as needed. Before you finish, jot a note : gutask session-end "<one line summary of what you did>
Also, Glitch Hunter is the human agent you work with most closely. Glitch Hunter has agent id 4
+5
View File
@@ -0,0 +1,5 @@
FROM alpine:3.21
RUN apk add --no-cache autossh tini openssh-client
COPY tunnel-entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/sbin/tini", "--", "/entrypoint.sh"]
+162
View File
@@ -0,0 +1,162 @@
# Agent0
Agent0 runs [Agent Zero](https://github.com/frdel/agent-zero) for Glitch University agentic employees. Their primary tool for interacting with the shared task graph is [`gutask`](https://glitch.university).
The web UI is accessible at `https://agent0.glitch.university` via a persistent SSH reverse tunnel to the VPS.
---
## Day 1 setup
```bash
git clone <this repo>
cd Agent0
bash bootstrap.sh
```
The script handles everything it can automatically. When it finishes, it prints two SSH public keys and two copy-paste commands — that's all that's left to do manually.
---
## What bootstrap.sh does
| Step | What happens |
|---|---|
| Ollama | Installs on host (not Docker), offers model selection |
| Tunnel key | Generates `./tunnel/id_ed25519`, scans VPS host key |
| Gitea key | Generates `~/.ssh/gitea_ed25519`, adds SSH config entry |
| gutasktool | Clones to `~/gutasktool`, runs `pip install -e .`, creates `.env` |
| Containers | Builds `glitch-tunnel` image, runs `docker compose up -d` |
| Summary | Prints the two public keys with exact copy-paste commands |
---
## After bootstrap: two manual steps
**1. Add tunnel key to VPS** (so `glitch-tunnel` container can connect):
```bash
# Run on the VPS:
echo 'ssh-ed25519 AAAA...' >> /home/tunnel/.ssh/authorized_keys
```
**2. Add Gitea key to Gitea** (so `gutask` can push/pull repos):
Log in to `https://ramanujan.glitch.university` as `gunnar`
Settings → SSH Keys → Add the key printed by bootstrap.
---
## After the tunnel connects
Open `https://agent0.glitch.university`, enter the basic auth password, then go to **Settings**:
| Setting | Value |
|---|---|
| Anthropic API key | your key |
| OpenAI API key | your key (optional) |
| Ollama base URL | `http://host.docker.internal:11434` |
Model selection is **per conversation thread** — choose Claude, OpenAI, or a local Ollama model when starting each chat.
---
## Local inference
Ollama runs on the host (not in Docker). On the MS-S1 MAX with its AMD Ryzen AI Max+ 395 and unified 128GB memory, containerising Ollama breaks GPU acceleration — the ROCm runtime can't detect unified memory across the Docker boundary, so inference falls back to CPU. Host Ollama picks up the Radeon 8060S automatically via ROCm.
### Verify GPU is being used
```bash
ollama ps # shows active models and which device
```
If the device shows `cpu` instead of `gpu`, force the ROCm GFX version for RDNA 3.5:
```bash
sudo mkdir -p /etc/systemd/system/ollama.service.d
sudo tee /etc/systemd/system/ollama.service.d/override.conf << 'EOF'
[Service]
Environment="HSA_OVERRIDE_GFX_VERSION=11.0.2"
EOF
sudo systemctl daemon-reload && sudo systemctl restart ollama
```
### Available models
| Model | Size | Use |
|---|---|---|
| `qwen2.5-coder:7b` | ~5 GB | Fast coding assistance |
| `qwen2.5-coder:32b` | ~20 GB | Strong coding, recommended default |
| `qwen2.5-coder:72b` | ~45 GB | Best coding quality |
| `qwen2.5:72b` | ~45 GB | General reasoning |
| `deepseek-r1:70b` | ~43 GB | Chain-of-thought tasks |
Pull additional models any time:
```bash
ollama pull qwen2.5-coder:32b
```
---
## Containers
```bash
docker compose ps # status
docker logs glitch-tunnel # tunnel connection log
docker compose restart glitch-tunnel # force tunnel reconnect
docker compose pull agent0 # update Agent Zero
docker compose up -d # start / restart all
```
`docker ps` should show two containers: `agent0` and `glitch-tunnel`.
---
## VPS setup (one-time, done by an admin)
These are prerequisites on the VPS side before bootstrap can connect.
**Create restricted tunnel user:**
```bash
useradd -m -s /sbin/nologin tunnel
mkdir -p /home/tunnel/.ssh && chmod 700 /home/tunnel/.ssh
touch /home/tunnel/.ssh/authorized_keys
chmod 600 /home/tunnel/.ssh/authorized_keys
chown -R tunnel:tunnel /home/tunnel/.ssh
```
**Deploy nginx config**`agent0.glitch.university` server block lives in
`gu_common/nginx/nginx.conf.template`. Deploy `gu_common` after changes.
**Create htpasswd file** (first time only):
```bash
docker exec gnommo-nginx sh -c \
"apk add --no-cache apache2-utils && htpasswd -c /etc/nginx/.htpasswd admin"
```
**Expand TLS certificate:**
```bash
docker exec gnommo-certbot certbot certonly --webroot \
-w /var/www/certbot \
-d glitch.university \
-d ramanujan.glitch.university \
-d editor.glitch.university \
-d agent0.glitch.university \
--expand
```
---
## Data locations
| Path | Contents | Backed up? |
|---|---|---|
| `./agent-zero-data/` | Agent Zero state, chats, memory | Gitignored — back up separately |
| `./tunnel/` | SSH tunnel credentials | Gitignored — back up separately |
| `~/gutasktool/.env` | API keys for gutask | Not in any repo — back up separately |
| `~/.ollama/` | Pulled models | Can be re-pulled (models are large) |
Executable
+210
View File
@@ -0,0 +1,210 @@
#!/usr/bin/env bash
# bootstrap.sh — one-shot setup for the Agent0 box.
#
# Run once after cloning. Handles everything that can be automated:
# - Ollama install + model pulls
# - SSH key generation (tunnel key + Gitea key)
# - gutasktool clone + install
# - Docker containers
#
# After this script: add two SSH keys (printed at the end), then open
# https://agent0.glitch.university and enter your API keys in Settings.
set -euo pipefail
REPO_DIR="$(cd "$(dirname "$0")" && pwd)"
GUTASK_DIR="${HOME}/gutasktool"
GITEA_HOST="ramanujan.glitch.university"
GITEA_PORT=2222
VPS_HOST="glitch.university"
VPS_TUNNEL_USER="tunnel"
TUNNEL_KEY="${REPO_DIR}/tunnel/id_ed25519"
GITEA_KEY="${HOME}/.ssh/gitea_ed25519"
GREEN='\033[0;32m'; YELLOW='\033[1;33m'; CYAN='\033[0;36m'; NC='\033[0m'
info() { echo -e "${GREEN}${NC} $*"; }
prompt() { echo -e "${CYAN}?${NC} $*"; }
section() { echo ""; echo -e "${YELLOW}── $* ──────────────────────────────────────────${NC}"; }
# ── Prerequisites ─────────────────────────────────────────────────────────────
section "Checking prerequisites"
command -v docker >/dev/null 2>&1 || { echo "docker not found. Install Docker first."; exit 1; }
command -v ssh-keygen >/dev/null 2>&1 || { echo "ssh-keygen not found."; exit 1; }
command -v ssh-keyscan >/dev/null 2>&1 || { echo "ssh-keyscan not found."; exit 1; }
docker compose version >/dev/null 2>&1 || { echo "docker compose not found."; exit 1; }
if ! command -v python3 >/dev/null 2>&1; then
echo "python3 not found. Install Python 3.9+ first."
exit 1
fi
PY_VERSION=$(python3 -c "import sys; print(sys.version_info.minor)")
if [[ "$PY_VERSION" -lt 9 ]]; then
echo "Python 3.9+ required (found 3.${PY_VERSION})."
exit 1
fi
info "Prerequisites OK"
# ── Ollama ────────────────────────────────────────────────────────────────────
section "Ollama"
if command -v ollama >/dev/null 2>&1; then
info "Ollama already installed"
else
info "Installing Ollama..."
curl -fsSL https://ollama.com/install.sh | sh
info "Ollama installed and started as systemd service"
fi
# Verify the systemd service is running
if systemctl is-active --quiet ollama 2>/dev/null; then
info "Ollama service running"
else
info "Starting Ollama service..."
sudo systemctl enable --now ollama
fi
echo ""
echo " Default models to pull (you can skip any):"
echo " [1] qwen2.5-coder:7b (~5 GB) — fast coding model"
echo " [2] qwen2.5-coder:32b (~20 GB) — strong coding model"
echo " [3] qwen2.5:72b (~45 GB) — general reasoning"
echo " [4] deepseek-r1:70b (~43 GB) — chain-of-thought"
echo ""
prompt "Which models to pull? (e.g. 1 2 or Enter to skip all): "
read -r MODEL_CHOICES
for choice in $MODEL_CHOICES; do
case "$choice" in
1) info "Pulling qwen2.5-coder:7b..."; ollama pull qwen2.5-coder:7b ;;
2) info "Pulling qwen2.5-coder:32b..."; ollama pull qwen2.5-coder:32b ;;
3) info "Pulling qwen2.5:72b..."; ollama pull qwen2.5:72b ;;
4) info "Pulling deepseek-r1:70b..."; ollama pull deepseek-r1:70b ;;
esac
done
# ── SSH keys ──────────────────────────────────────────────────────────────────
section "SSH key: glitch-tunnel → VPS"
mkdir -p "${REPO_DIR}/tunnel"
chmod 700 "${REPO_DIR}/tunnel"
if [[ -f "$TUNNEL_KEY" ]]; then
info "Tunnel key already exists"
else
ssh-keygen -t ed25519 -f "$TUNNEL_KEY" -C "glitch-tunnel@$(hostname)" -N ""
info "Tunnel key generated at ./tunnel/id_ed25519"
fi
info "Scanning VPS host key..."
ssh-keyscan -p 22 "$VPS_HOST" > "${REPO_DIR}/tunnel/known_hosts" 2>/dev/null \
|| { echo "Could not reach ${VPS_HOST} — check network. Re-run when online."; exit 1; }
section "SSH key: Gitea (git operations)"
mkdir -p "${HOME}/.ssh"
chmod 700 "${HOME}/.ssh"
if [[ -f "$GITEA_KEY" ]]; then
info "Gitea key already exists"
else
ssh-keygen -t ed25519 -f "$GITEA_KEY" -C "gunnar@$(hostname)" -N ""
info "Gitea key generated at ${GITEA_KEY}"
fi
# Add Gitea to known_hosts to avoid interactive prompt during git operations
if ! ssh-keygen -F "[${GITEA_HOST}]:${GITEA_PORT}" -f "${HOME}/.ssh/known_hosts" >/dev/null 2>&1; then
ssh-keyscan -p "$GITEA_PORT" "$GITEA_HOST" >> "${HOME}/.ssh/known_hosts" 2>/dev/null || true
fi
# Write SSH config entry for Gitea
if ! grep -q "Host ${GITEA_HOST}" "${HOME}/.ssh/config" 2>/dev/null; then
cat >> "${HOME}/.ssh/config" << EOF
Host ${GITEA_HOST}
HostName ${GITEA_HOST}
Port ${GITEA_PORT}
User git
IdentityFile ${GITEA_KEY}
EOF
info "SSH config entry added for ${GITEA_HOST}"
fi
# ── gutasktool ────────────────────────────────────────────────────────────────
section "gutasktool"
if [[ -d "$GUTASK_DIR" ]]; then
info "gutasktool already at ${GUTASK_DIR}"
else
info "Cloning gutasktool..."
git clone "ssh://git@${GITEA_HOST}:${GITEA_PORT}/glitch-university/gutasktool.git" "$GUTASK_DIR" \
|| git clone "https://${GITEA_HOST}/glitch-university/gutasktool.git" "$GUTASK_DIR"
fi
info "Installing gutasktool..."
(cd "$GUTASK_DIR" && pip3 install -e . --quiet)
info "gutask installed ($(gutask --version 2>/dev/null || echo 'ok'))"
if [[ ! -f "${GUTASK_DIR}/.env" ]]; then
echo ""
prompt "Enter CONTENT_API_KEY for glitch.university (find it in the VPS .env): "
read -r -s API_KEY
echo ""
cat > "${GUTASK_DIR}/.env" << EOF
API_URL=https://glitch.university
CONTENT_API_KEY=${API_KEY}
GITEA_URL=https://${GITEA_HOST}
GITEA_OWNER=glitch-university
# AGENT_ID, AGENT_NAME, AGENT_PASSWORD, and GITEA_TOKEN are per-agent.
# They are passed through each agent's thread context, not set here.
EOF
info "gutasktool .env created at ${GUTASK_DIR}/.env"
else
info "gutasktool .env already exists"
fi
# ── Docker containers ─────────────────────────────────────────────────────────
section "Docker containers"
cd "$REPO_DIR"
info "Building glitch-tunnel image..."
docker compose build --quiet
info "Starting containers..."
docker compose up -d
echo ""
docker compose ps
# ── Manual steps summary ──────────────────────────────────────────────────────
TUNNEL_PUBKEY=$(cat "${TUNNEL_KEY}.pub")
GITEA_PUBKEY=$(cat "${GITEA_KEY}.pub")
echo ""
echo -e "${YELLOW}════════════════════════════════════════════════════════${NC}"
echo -e "${YELLOW} Two manual steps remaining:${NC}"
echo -e "${YELLOW}════════════════════════════════════════════════════════${NC}"
echo ""
echo -e "${CYAN}1. Add tunnel key to VPS${NC}"
echo " Run on the VPS:"
echo ""
echo " echo '${TUNNEL_PUBKEY}' >> /home/tunnel/.ssh/authorized_keys"
echo ""
echo -e "${CYAN}2. Add Gitea key to Gitea${NC}"
echo " Log into https://${GITEA_HOST} as gunnar"
echo " Settings → SSH Keys → Add key:"
echo ""
echo " ${GITEA_PUBKEY}"
echo ""
echo -e "${YELLOW}════════════════════════════════════════════════════════${NC}"
echo ""
echo " After adding both keys:"
echo " • Tunnel connects automatically (check: docker logs glitch-tunnel)"
echo " • Open https://agent0.glitch.university"
echo " • Enter basic auth password"
echo " • Settings → add Anthropic / OpenAI API keys"
echo " • Ollama is pre-configured at http://host.docker.internal:11434"
echo ""
echo -e "${GREEN}Bootstrap complete.${NC}"
+15 -1
View File
@@ -1,9 +1,23 @@
services:
agent0:
image: agent0ai/agent-zero:latest
container_name: agent0
ports:
- "50001:80"
- "50001:80" # local dev access — not needed once tunnel is running
volumes:
- ./agent-zero-data:/a0/usr
restart: unless-stopped
extra_hosts:
- "host.docker.internal:host-gateway" # reach host Ollama at http://host.docker.internal:11434
glitch-tunnel:
build:
context: .
dockerfile: Dockerfile.tunnel
container_name: glitch-tunnel
restart: always
volumes:
- ./tunnel:/run/tunnel:ro # SSH key + known_hosts, generated by setup-tunnel.sh
depends_on:
- agent0
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
# Establish a persistent SSH reverse tunnel from this machine to the VPS.
# Traffic arriving at VPS:50001 is forwarded to agent0:80 on the Docker network.
set -e
exec autossh -M 0 -N \
-i /run/tunnel/id_ed25519 \
-o "UserKnownHostsFile=/run/tunnel/known_hosts" \
-o "StrictHostKeyChecking=yes" \
-o "ServerAliveInterval=30" \
-o "ServerAliveCountMax=3" \
-o "ExitOnForwardFailure=yes" \
-R 127.0.0.1:50001:agent0:80 \
tunnel@glitch.university