diff --git a/README.md b/README.md index 34bcb44..ca65e0c 100644 --- a/README.md +++ b/README.md @@ -105,7 +105,8 @@ published image there and runs `scripts/smoke-test.sh`. The old container is onl you can roll back. `--help` lists the options (owner, port, bind address, dry run). `--tools-dir /mnt/disks/ssd/codecraft` puts everything agents install or download – the agent home (`/data/home`: toolchains, pip/npm/cargo packages, caches), the users' homes and `/tmp` – on -a separate disk; the data folder keeps only the database, configuration and backups. +a separate disk; the data folder keeps only the database, configuration and backups. Step by step, +with rollback and troubleshooting: [docs/MOVING.md](docs/MOVING.md). ## How it works diff --git a/docs/MOVING.md b/docs/MOVING.md new file mode 100644 index 0000000..e49e4de --- /dev/null +++ b/docs/MOVING.md @@ -0,0 +1,128 @@ +# Moving CodeCraft to another host (Unraid) + +This guide moves a running installation – Docker volume `codecraft-data` on the current host – +to another Docker host that keeps its data in **bind mounts**, for example Unraid. The script +`scripts/move-server.sh` does the work; this page explains what goes where, what to prepare and +how to check and roll back. + +## What lives where + +| Part | Contents | Today (Docker volume) | On the new host | +|---|---|---|---| +| **Data** | database `codecraft.db`, `config.toml` (incl. API keys), daily backups | `codecraft-data:/data` | `/data` → `/data` | +| **Agent installs** | the agent's home `/data/home`: toolchains (Rust, …), pip/pipx/npm/cargo packages, downloaded programs (`~/opt`), caches | inside the volume | `/home` → `/data/home` | +| **Users' homes** | homes of sandboxed users `/data/users//home` (their installs, git settings) | inside the volume | `/users` → `/data/users` | +| **Temporary files** | `/tmp` of agents, builds, downloads | container layer | `/tmp` → `/tmp` | +| **Workspace** | your projects | `WORKSPACE` from `.env` | `--workspace` path → `/workspace` | +| **Image** | the program | registry image | registry image | + +The image never changes: the container runs without root (no `apt`, no `sudo`), so agents can +only install into their home directory. With `--tools-dir` everything agents install or +download lands on a separate disk (e.g. an SSD); the data folder keeps only the database, +configuration and backups. If the tools disk is lost, agents simply install what they need +again – nothing important is on it. (Backups contain the database and configuration, not the +agent home.) + +Sizes on 2026-11-09 for orientation: agent home 2.2 GB (of which downloaded programs 1.3 GB, +Rust toolchain 0.8 GB), database/config/backups about 1.2 GB. + +## Before you start + +On the **new host** (Unraid): + +- **SSH with a key** from the current host, without a password prompt. Check from the current + host: `ssh root@unraid true` (no output, no question = fine). +- **docker**, **rsync** and **tar** available (all three are on Unraid by default). +- **Docker Compose** is optional: without it the script starts the container with the generated + `start.sh` (Unraid: the "Docker Compose Manager" plugin adds Compose). +- The **tools disk** is mounted, e.g. an SSD via Unassigned Devices under `/mnt/disks/ssd`. +- **Registry access**: the image comes from `git.familie-zink.org/zink/codecraft`. If the pull is + refused, run `docker login git.familie-zink.org` on Unraid once. + +Decide: + +| Question | Option | Example | +|---|---|---| +| Folder for data, compose file and scripts | target `root@host:/dir` | `/mnt/user/appdata/codecraft` | +| Folder for agent installs and `/tmp` | `--tools-dir` | `/mnt/disks/ssd/codecraft` | +| Your projects | `--workspace` | `/mnt/user/projects` | +| Owner of files (the container runs as this user) | `--uid`, `--gid` | Unraid default `99`/`100` (nobody:users); today `1000`/`1000` | +| Port and address | `--port`, `--bind` | `8787`; `0.0.0.0` if the reverse proxy runs on another machine or in another Docker network, `127.0.0.1` if it runs on the same host with host networking | + +## Moving + +On the **current host**, in the repository folder (where `.env` is): + +```sh +# 1. Look at the steps first (changes nothing) +scripts/move-server.sh root@unraid:/mnt/user/appdata/codecraft \ + --workspace /mnt/user/projects --tools-dir /mnt/disks/ssd/codecraft \ + --uid 99 --gid 100 --bind 0.0.0.0 --dry-run + +# 2. Move (same command without --dry-run) +scripts/move-server.sh root@unraid:/mnt/user/appdata/codecraft \ + --workspace /mnt/user/projects --tools-dir /mnt/disks/ssd/codecraft \ + --uid 99 --gid 100 --bind 0.0.0.0 +``` + +What happens: + +1. **Checks**: docker/tar/rsync on the target; the target data folder (and `home/`, `users/` in + the tools folder) must be empty – the script never overwrites. +2. **Workspace, first pass** with rsync while CodeCraft keeps running (the long part). +3. **Stops CodeCraft here.** Running agents and terminals end. The container is only stopped, + not removed. +4. **Data volume** is streamed with tar over SSH into `/data`; with `--tools-dir` the agent + home and the users' homes are then moved to the tools folder. +5. **Workspace, second pass**: only what changed since step 2. +6. **Owner** of data, tools and workspace set to `--uid:--gid` (needs root on the target). +7. **Writes** `/docker-compose.yml`, `/.env` (your `.env` with the new paths, port and + owner), `/start.sh` (the same as `docker run`, for hosts without Compose) and + `/docker/seccomp-sandbox.json`. +8. **Starts** the image `CODECRAFT_VERSION` from `.env` (Compose, else `start.sh`). +9. **Smoke test** on the target (`scripts/smoke-test.sh`): health, version, web UI, terminal + client download. + +Afterwards: + +- **Reverse proxy**: point the proxy host for `code.familie-zink.org` (Nginx Proxy Manager) to the + new host and port; keep WebSocket support on. +- **Check in the web UI**: sign in, open a chat, check the status LEDs (providers, MCP servers), + let an agent run `git status` in a project (credentials, identity), open a terminal tab. + Models or MCP servers on the old host's LAN address or `localhost` need new addresses. +- **Clean up** on the old host when everything works: `docker rm codecraft`, later + `docker volume rm codecraft-data`. + +## Rollback + +Nothing is deleted on the old host. On Unraid stop the new container (`docker stop codecraft`), +on the old host start the old one (`docker start codecraft`) and point the reverse proxy back. +Changes made on the new host in the meantime (chats, settings) are not in the old installation. + +## Running it on the new host + +In `` on Unraid: + +| Task | With Compose | Without Compose | +|---|---|---| +| Update | set `CODECRAFT_VERSION` in `.env`, then `docker compose pull && docker compose up -d` | set `CODECRAFT_VERSION` in `.env`, `docker pull …:`, then `./start.sh` | +| Restart | `docker compose restart` | `docker restart codecraft` | +| Logs | `docker compose logs -f` | `docker logs -f codecraft` | +| Check | `bash smoke-test.sh http://127.0.0.1:8787 ` | same | + +Releases are published from the development machine with `scripts/release.sh`; the new host +only pulls them. + +## Troubleshooting + +- **`data on the target is not empty`**: an earlier attempt left files. Move `/data` (and + `home/`, `users/` in the tools folder) away and run the script again. +- **Image pull refused**: `docker login git.familie-zink.org` on the target, then + `cd && docker compose up -d` (or `./start.sh`). +- **Smoke test fails**: `docker logs codecraft` on the target. The old installation is stopped but + intact – `docker start codecraft` on the old host brings it back. +- **`could not chown`**: the SSH user is not root; set the owner by hand + (`chown -R 99:100 /data `). +- **Permission denied inside the container**: data, tools and workspace must belong to the + `UID`/`GID` in `/.env`. +- **Port already in use**: change `PORT` in `/.env` and start again.