---
name: add-llama-cpp-8eec98df2db6
description: Install and verify a local llama.cpp server for optional Deus local-generation experiments. Keeps Ollama as the required default for embeddings and judge work.
disable-model-invocation: true
---

# Add llama.cpp

This skill installs `llama.cpp`, runs `llama-server` as a local host service, and wires the local endpoint into Deus only when the current checkout already supports the optional `llama_cpp` provider.

Use this when the user wants a faster or cheaper local text-generation path for experiments, benchmarks, or future backend work.

Slash command: `/add-llama-cpp`.

**Important boundaries:**

- This does **not** replace Ollama for memory embeddings or the default judge. Ollama remains required unless the repo deliberately changes those surfaces.
- This skill is **macOS-first** for installation and service management. On Linux or Windows, continue only if `llama-server` is already installed or the user explicitly wants a manual install path.
- If the current checkout does not yet contain the optional Deus-side `llama_cpp` integration, complete the host install anyway and tell the user the runtime wiring is a separate source task.

## Phase 1: Pre-flight

### Check current state

```bash
command -v llama-server >/dev/null 2>&1 && llama-server --version || echo "llama.cpp not installed"
curl -fsS http://127.0.0.1:8080/health 2>/dev/null || echo "llama-server not responding on 127.0.0.1:8080"
test -f evolution/generative/providers/llama_cpp.py && echo "DEUS_LLAMA_CPP_PROVIDER=true" || echo "DEUS_LLAMA_CPP_PROVIDER=false"
test -f setup/llama-cpp.ts && echo "DEUS_LLAMA_CPP_SETUP=true" || echo "DEUS_LLAMA_CPP_SETUP=false"
```

### Ask scope

AskUserQuestion: Do you want host install only, or host install plus optional Deus wiring if this checkout supports it?
1. **Host install only** - install `llama.cpp`, run `llama-server`, and verify the local endpoint
2. **Host install + Deus wiring** - also configure repo env vars and run Deus-side verification when the checkout supports it

If the user chooses option 2 but either provider/setup file is missing, say clearly that the host install can proceed now and the checkout wiring remains a separate code task.

## Phase 2: Install llama.cpp

### macOS

If `llama-server` is not already installed:

```bash
brew install llama.cpp
```

Verify:

```bash
llama-server --version
```

### Linux or Windows

If `llama-server` is already available in `PATH`, continue.

If it is missing, stop and tell the user this skill currently automates installation on macOS only. Offer to continue once `llama-server` is installed manually, or handle platform-specific installation as a separate task.

## Phase 3: Configure the Local Service

Create a local env file outside git:

```bash
mkdir -p "$HOME/.config/deus" "$HOME/.config/deus/scripts"
cat > "$HOME/.config/deus/llama-cpp.env" <<'EOF'
LLAMA_CPP_BIND_HOST=127.0.0.1
LLAMA_CPP_PORT=8080
LLAMA_CPP_MODEL=ggml-org/gemma-3-1b-it-GGUF:Q4_K_M
LLAMA_CPP_ALIAS=ggml-org/gemma-3-1b-it-GGUF:Q4_K_M
LLAMA_CPP_CTX_SIZE=8192
EOF
```

Create the launcher script:

```bash
cat > "$HOME/.config/deus/scripts/start-llama-cpp.sh" <<'EOF'
#!/usr/bin/env bash
set -euo pipefail

ENV_FILE="$HOME/.config/deus/llama-cpp.env"
if [ -f "$ENV_FILE" ]; then
  set -a
  . "$ENV_FILE"
  set +a
fi

: "${LLAMA_CPP_BIND_HOST:=127.0.0.1}"
: "${LLAMA_CPP_PORT:=8080}"
: "${LLAMA_CPP_MODEL:=ggml-org/gemma-3-1b-it-GGUF:Q4_K_M}"
: "${LLAMA_CPP_ALIAS:=$LLAMA_CPP_MODEL}"
: "${LLAMA_CPP_CTX_SIZE:=8192}"

exec llama-server \
  --host "$LLAMA_CPP_BIND_HOST" \
  --port "$LLAMA_CPP_PORT" \
  -hf "$LLAMA_CPP_MODEL" \
  --alias "$LLAMA_CPP_ALIAS" \
  -c "$LLAMA_CPP_CTX_SIZE" \
  --jinja
EOF
chmod +x "$HOME/.config/deus/scripts/start-llama-cpp.sh"
```

## Phase 4: Run as a Host Service

### macOS LaunchAgent

Write the LaunchAgent:

```bash
PLIST="$HOME/Library/LaunchAgents/com.deus.llama-cpp.plist"
mkdir -p "$HOME/Library/LaunchAgents" "$HOME/Library/Logs"
cat > "$PLIST" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>com.deus.llama-cpp</string>
    <key>ProgramArguments</key>
    <array>
      <string>$HOME/.config/deus/scripts/start-llama-cpp.sh</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>StandardOutPath</key>
    <string>$HOME/Library/Logs/deus-llama-cpp.log</string>
    <key>StandardErrorPath</key>
    <string>$HOME/Library/Logs/deus-llama-cpp.error.log</string>
  </dict>
</plist>
EOF
launchctl unload "$PLIST" 2>/dev/null || true
launchctl load "$PLIST"
launchctl kickstart -k "gui/$(id -u)/com.deus.llama-cpp"
```

### Linux or Windows

Do not invent a service wrapper if the platform path is unclear. Prefer a foreground verification run:

```bash
"$HOME/.config/deus/scripts/start-llama-cpp.sh"
```

If the user wants persistent background service management on Linux or Windows, treat that as a follow-up task after the endpoint is verified.

## Phase 5: Verify the Local Endpoint

Check health:

```bash
curl -fsS http://127.0.0.1:8080/health
curl -fsS http://127.0.0.1:8080/v1/models
```

Run a chat-completions smoke test:

```bash
curl -fsS http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "ggml-org/gemma-3-1b-it-GGUF:Q4_K_M",
    "messages": [{"role": "user", "content": "Reply with exactly: OK"}]
  }'
```

If the first model download takes time, inspect the log instead of assuming failure:

```bash
tail -50 "$HOME/Library/Logs/deus-llama-cpp.log" 2>/dev/null || true
tail -50 "$HOME/Library/Logs/deus-llama-cpp.error.log" 2>/dev/null || true
```

## Phase 6: Optional Deus Wiring

Run this phase only if:

- the user chose host install plus Deus wiring, and
- `evolution/generative/providers/llama_cpp.py` exists in the current checkout, and
- `setup/llama-cpp.ts` exists in the current checkout.

### Rebuild the agent container

If upgrading from a build predating llama.cpp support, rebuild the container so the new backend module is available:

```bash
./container/build.sh
```

### Configure repo env vars

Write or update the repo env:

```bash
if grep -q '^LLAMA_CPP_BASE_URL=' .env 2>/dev/null; then
  tmpf=$(mktemp) && sed 's#^LLAMA_CPP_BASE_URL=.*#LLAMA_CPP_BASE_URL=http://127.0.0.1:8080#' .env > "$tmpf" && mv "$tmpf" .env
else
  echo 'LLAMA_CPP_BASE_URL=http://127.0.0.1:8080' >> .env
fi

if grep -q '^LLAMA_CPP_MODEL=' .env 2>/dev/null; then
  tmpf=$(mktemp) && sed 's#^LLAMA_CPP_MODEL=.*#LLAMA_CPP_MODEL=ggml-org/gemma-3-1b-it-GGUF:Q4_K_M#' .env > "$tmpf" && mv "$tmpf" .env
else
  echo 'LLAMA_CPP_MODEL=ggml-org/gemma-3-1b-it-GGUF:Q4_K_M' >> .env
fi

mkdir -p data/env && cp .env data/env/env
```

> **Activate the backend:** Deus wiring is complete but the active backend is unchanged. To switch all future sessions to llama.cpp, run:
> ```bash
> deus backend set llama-cpp
> ```
> Ollama remains the default for embeddings and judge scoring regardless of this setting.

### Verify setup surface

```bash
LLAMA_CPP_BASE_URL=http://127.0.0.1:8080 npx tsx setup/index.ts --step llama-cpp
```

### Verify provider path

If the benchmark harness exists:

```bash
LLAMA_CPP_BASE_URL=http://127.0.0.1:8080 python3 -m evolution.benchmark_generative \
  --providers llama_cpp \
  --model llama_cpp=ggml-org/gemma-3-1b-it-GGUF:Q4_K_M \
  --json
```

If the benchmark file is missing, use a minimal availability check instead:

```bash
python3 - <<'PY'
from evolution.generative.providers.llama_cpp import LlamaCppGenerativeProvider
provider = LlamaCppGenerativeProvider()
print({"available": provider.is_available(), "model": provider.get_default_model()})
PY
```

## Troubleshooting

### `llama-server` exits immediately

Check whether the model slug is valid and whether the first download returned 404 or auth errors:

```bash
tail -100 "$HOME/Library/Logs/deus-llama-cpp.error.log" 2>/dev/null || true
```

If the chosen Hugging Face preset is invalid, switch `LLAMA_CPP_MODEL` in `~/.config/deus/llama-cpp.env` to a known-good preset and restart the service.

### Health works but `/v1/chat/completions` fails

The endpoint is up, but the model did not finish loading or prompt templating is wrong. Check logs first. For instruction-tuned GGUFs like Gemma, keep `--jinja` enabled in the launcher script.

### Deus wiring files are missing

Host installation is still complete. Tell the user clearly:

> `llama.cpp` is running on the host, but this checkout does not yet include the optional Deus-side `llama_cpp` provider wiring. That remains a separate source change.

### Revert

Stop the service and remove the local files:

```bash
launchctl bootout "gui/$(id -u)" "$HOME/Library/LaunchAgents/com.deus.llama-cpp.plist" 2>/dev/null || true
rm -f "$HOME/Library/LaunchAgents/com.deus.llama-cpp.plist"
rm -f "$HOME/.config/deus/scripts/start-llama-cpp.sh"
rm -f "$HOME/.config/deus/llama-cpp.env"
```


---
Source attribution: sliamh11
Original project: https://github.com/sliamh11/Deus/blob/2f24c38a2ae42d3161492f0729975b286452f4b4/.claude/skills/add-llama-cpp/SKILL.md
License: MIT
License information: https://opensource.org/license/mit
Original upstream instructions; downloaded from Dave AI.

## Upstream license notice

MIT License

Copyright (c) 2026 Gavriel
Copyright (c) 2026 Liam Steiner

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.



# Dave AI resource metadata

Creator / publisher: sliamh11
Original project: https://github.com/sliamh11/Deus/blob/2f24c38a2ae42d3161492f0729975b286452f4b4/.claude/skills/add-llama-cpp/SKILL.md
Attribution basis: Source repository record

Fields of work: Development & Engineering
Roles: Backend Developer, DevOps / SRE, ML / AI Engineer
Specialties: Backend
Tools: Deus, Llama.Cpp
Country applicability: Country not specified
Classification basis: Source catalog labels
Instruction SHA-256: cf1788b85da455bf46f94e3ca98dd5b7fe9fab4c0d875ad129a8fdd6d30b9ed4