Skip to content

development.md

Latest commit

 

History

History
493 lines (355 loc) · 16.9 KB

development.md

File metadata and controls

493 lines (355 loc) · 16.9 KB

Development Guide

Quick Reference

Task Command
One-shot notebook setup cargo xtask dev
Start dev server cargo xtask notebook
Standalone Vite cargo xtask vite
Attach to Vite cargo xtask notebook --attach
Full debug build cargo xtask build
Rust-only rebuild cargo xtask build --rust-only
Run bundled binary cargo xtask run
Run with notebook cargo xtask run path/to/notebook.ipynb
Build release .app cargo xtask build-app
Build release DMG cargo xtask build-dmg
nteract-dev MCP server cargo xtask run-mcp
MCP config JSON cargo xtask run-mcp --print-config
Direct runt mcp (no proxy) ./target/debug/runt mcp
Lint (check mode) cargo xtask lint
Lint (auto-fix) cargo xtask lint --fix

Build Cache (sccache)

Install sccache to share compiled artifacts across worktrees. Without it, each worktree rebuilds ~788 crates from scratch.

brew install sccache   # macOS

The xtask commands auto-detect sccache and set RUSTC_WRAPPER when it's available — no configuration needed. You'll see "Using sccache for compilation cache" in the build output when it's active.

Windows Target Checks From macOS

Use cargo-xwin when validating Windows MSVC compile errors from macOS. A direct cargo check --target x86_64-pc-windows-msvc does not provide the Windows SDK or MSVC CRT headers that C dependencies need.

Install the local tools:

rustup target add x86_64-pc-windows-msvc
cargo install cargo-xwin
brew install llvm nasm

Then run the Windows check through cargo-xwin:

cargo xwin check -p runtimed --lib --target x86_64-pc-windows-msvc

The first run downloads the MSVC CRT and Windows SDK into cargo-xwin's cache. nasm is required by aws-lc-sys, and Homebrew's LLVM provides the clang-cl/lld-link toolchain used by the cross build.

Choosing a Workflow

cargo xtask dev — One Command Setup + Dev

Best for first-time local setup or when you want the daemon and notebook app to come up together.

cargo xtask dev

This command:

  • runs pnpm install when your workspace dependencies are missing or stale
  • runs cargo xtask build unless you pass --skip-build
  • starts the per-worktree dev daemon
  • waits for the daemon to be reachable
  • launches the notebook app in dev mode

For faster repeat launches:

cargo xtask dev --skip-install --skip-build

cargo xtask notebook — Hot Reload

Best for UI/React development. Uses Vite dev server on port 5174. Changes to React components hot-reload instantly.

cargo xtask notebook

cargo xtask vite + notebook --attach — Multi-Window Testing

When testing with multiple notebook windows, closing the first Tauri window normally kills the Vite server. To avoid this:

# Terminal 1: Start Vite standalone (stays running)
cargo xtask vite

# Terminal 2+: Attach Tauri to existing Vite
cargo xtask notebook --attach

Now you can close and reopen Tauri windows without losing Vite. This is useful for:

  • Testing realtime collaboration
  • Testing widgets across windows
  • Avoiding confusion when one window close breaks others

cargo xtask build / run — Debug Build

Best for:

  • Testing Rust changes
  • Multiple worktrees (avoids port 5174 conflicts)
  • Running the standalone binary

Builds a debug binary with frontend assets bundled in.

# Full build (frontend + rust)
cargo xtask build

# Run the bundled binary
cargo xtask run

# Run with a specific notebook
cargo xtask run path/to/notebook.ipynb

cargo xtask build also emits JavaScript source maps for the bundled debug UI, including inline maps for the isolated renderer iframe bundle, so native webview devtools can step through .tsx sources.

cargo xtask build --rust-only — Fast Rust Iteration

When you're only changing Rust code (not the frontend), skip the frontend rebuild:

# First time: full build
cargo xtask build

# Subsequent rebuilds: rust only (much faster)
cargo xtask build --rust-only
cargo xtask run

This is ideal for daemon development — build the frontend once, then iterate on Rust with fast rebuilds.

cargo xtask build-app / build-dmg — Release Builds

Mostly handled by CI for preview releases. Use locally only when testing:

  • App bundle structure
  • File associations
  • Icons

Build Order

The UI must be built before Rust because:

  • crates/notebook embeds assets from apps/notebook/dist/ via Tauri

The xtask commands handle this automatically. If building manually:

pnpm build          # Build notebook UI (isolated-renderer built inline)
cargo build         # Build Rust

Note: If you've changed crates/runtimed-wasm/, rebuild it explicitly with cargo xtask wasm (or the equivalent wasm-pack build ... command) before pnpm build. Normal cargo xtask build only verifies that the committed WASM artifact exists; it does not regenerate it for you.

Test Notebooks

Test notebooks live in crates/notebook/fixtures/audit-test/ and sample notebooks in crates/notebook/resources/sample-notebooks/.

cargo xtask build
./target/debug/notebook crates/notebook/fixtures/audit-test/1-vanilla.ipynb

Daemon Development

The notebook app connects to a background daemon (runtimed) that manages prewarmed environments and notebook document sync. Important: The daemon is a separate process. When you change code in crates/runtimed/, the running daemon still uses the old binary until you reinstall it.

Development Mode (Per-Worktree Isolation)

In production, the Tauri app auto-installs and manages the system daemon. In development, you control the daemon yourself, which gives you:

  • Isolated state per worktree (no conflicts when testing across branches)
  • Your code changes take effect immediately on daemon restart
  • No interference with the system daemon

With nteract-dev (preferred for agents):

If you have the repo-local nteract-dev MCP entry configured, the daemon is managed for you:

  • up — idempotent "get me to a working state". Sweeps zombie Vite processes, ensures daemon is running, ensures the MCP child is healthy. Args: vite=true also starts Vite (health-probed), rebuild=true rebuilds the daemon binary + Python bindings first, mode="debug"|"release" switches build mode.
  • down — stop the managed Vite dev server. Pass daemon=true to also stop the daemon.
  • status — read-only report (child, daemon, managed processes, build mode, version).
  • logs — tail daemon logs.
  • vite_logs — tail the Vite dev server log file.

No env vars or extra terminals needed. nteract-dev handles per-worktree isolation automatically.

Two-terminal workflow (without nteract-dev):

# Terminal 1: Start the dev daemon (stays running)
cargo xtask dev-daemon

# Terminal 2: Run the notebook app
cargo xtask notebook         # Hot-reload mode
# or
cargo xtask dev              # One-shot setup + daemon + app
# or
cargo xtask build            # Full build once
cargo xtask build --rust-only && cargo xtask run  # Fast iteration

The app detects dev mode and connects to the per-worktree daemon instead of installing/starting the system daemon.

Worktree isolation: When using cargo xtask dev, cargo xtask notebook, cargo xtask dev-daemon, or cargo xtask run-mcp, xtask automatically enables dev mode and passes the current git worktree as RUNTIMED_WORKSPACE_PATH to the subprocesses it starts. Conductor users get the same behavior via CONDUCTOR_WORKSPACE_PATH.

Non-Conductor users: No extra environment is needed for xtask commands:

# Terminal 1
cargo xtask dev-daemon

# Terminal 2
cargo xtask notebook

Useful commands:

./target/debug/runt daemon status           # Shows dev mode, worktree path, version
./target/debug/runt dev worktrees           # List all running dev daemons (requires RUNTIMED_DEV=1)
./target/debug/runt daemon logs -f          # Tail logs (uses correct log path in dev mode)

Per-worktree state is stored in <cache>/{cache_namespace}/worktrees/{hash}/ (macOS: ~/Library/Caches/, Linux: ~/.cache/). Source builds default to runt-nightly; set RUNT_BUILD_CHANNEL=stable only when you intentionally need the stable flow.

For AI agents: If the repo-local nteract-dev MCP entry is available, prefer its up / down / status / logs / vite_logs tools — they handle env vars and daemon lifecycle automatically. Keep nteract as the name for the global/system-installed MCP server. When using raw ./target/debug/runt commands outside an xtask-managed subprocess, set the env vars manually:

export RUNTIMED_DEV=1
export RUNTIMED_WORKSPACE_PATH="$(pwd)"
./target/debug/runt daemon status

Testing Against System Daemon (Production Mode)

When you need to test the full production flow (daemon auto-install, upgrades, etc.):

# Make sure dev mode is NOT set
unset RUNTIMED_DEV
unset RUNTIMED_WORKSPACE_PATH

# On macOS: install the nteract Nightly .app and let SMAppService manage the daemon.
# On Linux / headless: rebuild and reinstall the full nightly stack from source:
./scripts/install-nightly

# Run the app (it will connect to the system daemon)
cargo xtask notebook

Note: ./scripts/install-nightly refuses on macOS by default (the app bundle manages the daemon itself). Pass --on-macos if you really need to install out-of-bundle binaries, and --replace-installed-app if an app bundle is already present.

Daemon logs

# View recent logs
runt daemon logs -n 100

# Watch logs in real-time
runt daemon logs -f

# Filter for specific topics
runt daemon logs -f | grep -i "kernel\|auto-detect"

Common gotchas

If your daemon code changes aren't taking effect:

  1. In dev mode: Did you restart cargo xtask dev-daemon?
  2. In production mode (macOS): Re-install the nteract Nightly .app (it auto-updates the bundled daemon).
  3. In production mode (Linux / headless): Did you run ./scripts/install-nightly?
  4. Check which daemon is running: runt daemon status

If the app says "Dev daemon not running":

  • You're in dev mode but haven't started the dev daemon
  • Run cargo xtask dev-daemon in another terminal first

See contributing/runtimed.md for full daemon development docs.

MCP Server Development

The nteract MCP server lets AI agents (Claude, Zed, etc.) interact with notebooks via the daemon. There are two ways to run it locally.

Python environment

The Python workspace root is the repo root — pyproject.toml and .venv both live there, not inside python/.

Venv Location Purpose
Workspace venv .venv (repo root) MCP server, uv run nteract, maturin develop target
Test venv python/runtimed/.venv Isolated pytest runs for the runtimed bindings

Sync the workspace (installs nteract + runtimed + deps into .venv):

uv sync            # from repo root

Build the native runtimed extension (from crates/runtimed-py):

cd crates/runtimed-py
VIRTUAL_ENV=../../.venv maturin develop

The VIRTUAL_ENV override ensures the .so is installed into the workspace venv (where the MCP server runs), not into python/runtimed/.venv (the test-only venv).

Run the MCP server manually:

# Using the built runt binary (preferred)
./target/debug/runt mcp

# Or via the Python wrapper (finds and launches runt mcp)
uv run nteract

nteract-dev (recommended)

nteract-dev (the dev MCP server, built from crates/mcp-supervisor/) proxies runt mcp and adds dev tools on top. It handles daemon lifecycle, auto-restart on crash, and hot-reload on file changes — one command, everything works:

cargo xtask run-mcp

This:

  1. Starts the dev daemon if not running
  2. Builds runt and spawns runt mcp as a child process
  3. Proxies all tool calls + adds the dev tools (up, down, status, logs, vite_logs)
  4. Watches source files and hot-reloads on changes

For your MCP client config (Zed, Claude Desktop, Codex, etc.):

cargo xtask run-mcp --print-config

Use nteract-dev as the server name for this source tree so it stays distinct from any global/system nteract MCP entry.

Codex app/CLI can read a project-scoped .codex/config.toml. This repo includes one that mirrors the same setup:

[mcp_servers.nteract-dev]
command = "cargo"
args = ["run", "-p", "mcp-supervisor"]
cwd = "."
startup_timeout_sec = 120

[mcp_servers.nteract-dev.env]
RUNTIMED_DEV = "1"

cwd = "." matters for Codex app/CLI: it forces mcp-supervisor to start from the repo root, which is enough to keep the server pinned to the worktree even when Codex was launched outside a direnv-managed shell. The longer startup timeout avoids dropping nteract-dev during the initial cargo build.

Or configure .zed/settings.json directly (gitignored):

{
  "context_servers": {
    "nteract-dev": {
      "command": "./target/debug/mcp-supervisor",
      "args": [],
      "env": { "RUNTIMED_DEV": "1" }
    }
  }
}

In clients that namespace tools by server name, this keeps repo-local notebook tools separate from the global install while still exposing the same notebook APIs plus the extra dev tools.

nteract-dev tools

These tools are always available, even when the child runt mcp is down:

Tool Purpose
up Idempotent dev-env bring-up. Sweeps zombie Vite processes, ensures daemon + child healthy. Args: vite=true, rebuild=true, mode="debug"|"release"
down Stop the managed Vite dev server. daemon=true also stops the daemon.
status Read-only report: child, daemon, managed processes, build mode, version.
logs Tail the daemon log file.
vite_logs Tail the Vite dev server log file.

Hot reload

nteract-dev watches crates/runt-mcp/src/, crates/runtimed-client/src/, python/nteract/src/, python/runtimed/src/, crates/runtimed-py/src/, and crates/runtimed/src/:

  • crates/runt-mcp/src/ → cargo build -p runt, then child restart
  • crates/runtimed-client/src/ → cargo build -p runt + maturin develop, then child restart
  • Python changes → child restarts automatically
  • Daemon / bindings Rust changes → maturin develop runs first, then child restarts
  • Behavior changes take effect immediately on the next tool call
  • New/removed tools may take a moment for the client to discover

Direct mode (no proxy)

If you don't need auto-restart or file watching, run runt mcp directly from the dev build:

# Terminal 1: start the dev daemon
cargo xtask dev-daemon

# Terminal 2: launch MCP server (Rust-native, no Python)
./target/debug/runt mcp

How it works

nteract-dev is the dev-only MCP server for this source tree. It exposes the dev tools (up, down, status, logs, vite_logs) itself, then proxies the regular notebook tools from a child runt mcp process. That child is the Rust-native MCP implementation from crates/runt-mcp/, not a Python MCP server.

The Python workspace packages still matter for local development: python/runtimed/ provides the PyO3 bindings, and python/nteract/ is a convenience wrapper that finds and launches runt mcp (still shipped, not the recommended path). Both are workspace members of the repo-root pyproject.toml, so uv sync installs them into .venv at the repo root.

Before You Commit

CI rejects PRs that fail formatting. Run this before every commit:

cargo xtask lint --fix

This formats Rust, lints/formats TypeScript/JavaScript with vp (Vite+ unified toolchain), and lints/formats Python with ruff.

Use conventional commits for commit messages and PR titles:

feat(kernel): add environment source labels
fix(runtimed): handle missing daemon socket
docs(agents): enforce conventional commit format

Zed Editor Integration

The repo includes .zed/tasks.json with pre-configured tasks. xtask-managed tasks derive the correct worktree environment automatically. Use task: spawn (cmd-shift-t) to run them:

Task What it does
Dev Daemon cargo xtask dev-daemon; xtask derives the worktree env
Dev App cargo xtask notebook; xtask derives the worktree env and auto-assigned Vite port
Daemon Status ./target/debug/runt daemon status pointed at the worktree daemon
Daemon Logs ./target/debug/runt daemon logs -f with live tail
Format cargo xtask lint --fix (Rust + JS/TS via vp + Python ruff)
Setup pnpm install && cargo xtask build for first-time setup

The tasks use $ZED_WORKTREE_ROOT for RUNTIMED_WORKSPACE_PATH, giving each Zed worktree its own isolated daemon — no conflicts when working across branches.

For agents in Zed: The Zed task env vars aren't available in agent terminal sessions. Set them explicitly:

export RUNTIMED_DEV=1
export RUNTIMED_WORKSPACE_PATH="/path/to/your/worktree"