Run a powerful AI coding assistant entirely on your own hardware — no cloud, no API keys, no rate limits. This guide walks through connecting OpenCode to a local llama.cpp server so you get intelligent coding assistance from any GGUF model running on your GPU.

On this page
  1. Why This Matters
  2. Prerequisites
  3. Step 1 — Start the llama.cpp Server
  4. Step 2 — Create the OpenCode Configuration
  5. Step 3 — Run OpenCode
  6. Using Different Models
  7. Troubleshooting
  8. Multiple Providers
  9. References

Why This Matters

OpenCode is a terminal-based AI coding assistant that gives you an interactive TUI for code generation, refactoring, and debugging. Out of the box it connects to hosted models like Claude, GPT, or Gemini. But with a local llama.cpp server, you get:

  • Zero cost — no per-token billing
  • Full privacy — your code never leaves your machine
  • Unlimited usage — no rate limits or context quotas
  • Any model — swap between Qwen, Gemma, Llama, or any GGUF model

Prerequisites

  • llama.cpp running on your local machine with a model loaded — see setup guide
  • OpenCode installed — npm install -g opencode-ai (source)
  • A web browser — to test the llama.cpp API endpoint

Step 1 — Start the llama.cpp Server

Start llama.cpp in server mode. It exposes an OpenAI-compatible API at http://localhost:8080:

.\bin\llama-server.exe -m .\models\Qwen3.5-35B-A3B-Q4_K_M.gguf --host 0.0.0.0 --port 8080 -ngl 99 -c 8192

Verify it's working by hitting the API directly:

curl "http://localhost:8080/v1/chat/completions" ^
  -H "Content-Type: application/json" ^
  -d "{\"model\":\"test\",\"messages\":[{\"role\":\"user\",\"content\":\"Say hi\"}],\"max_tokens\":10}"

You should get a JSON response with the model's text output.

Step 2 — Create the OpenCode Configuration

OpenCode looks for a file named opencode.json (not .opencode.json) in one of these locations:

Priority Path Scope
1 opencode.json (current directory) Project-local
2 ~/.config/opencode/opencode.json Global (all projects)

Create opencode.json in your project root:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "local": {
      "options": {
        "baseURL": "http://localhost:8080",
        "apiKey": "sk-dummy"
      },
      "models": {
        "qwen3": {}
      }
    }
  },
  "model": "local/qwen3"
}

Config Breakdown

  • provider.local — A custom provider name (any name except built-in providers like anthropic, openai, etc.).
  • options.baseURL — The llama.cpp server address. Must be inside options, not at the provider root.
  • options.apiKey — A placeholder key. llama.cpp doesn't authenticate, but OpenCode requires a value.
  • models.qwen3 — Your model name. The name is arbitrary — llama.cpp uses whatever model you loaded, regardless of what's in the request.
  • model — The default model to use, in the format provider/model-name.

Key Pitfalls

baseURL must be inside options. Putting it at the provider root means OpenCode sends undefined/chat/completions instead of http://localhost:8080/chat/completions.

// ❌ Wrong — baseURL at provider root, not in options
"local": {
  "baseURL": "http://localhost:8080"
}

// ✅ Correct — baseURL inside options
"local": {
  "options": {
    "baseURL": "http://localhost:8080"
  }
}

Use a custom provider name, not a built-in one. Using openai as the provider name works for the API format, but you can't add custom models to it. OpenCode's built-in openai provider has a fixed model list. Use a name like local, my-model, or llama to define your own.

Verify your config is loaded before running OpenCode:

opencode debug config

You should see your provider, options, and model reflected in the output.

Step 3 — Run OpenCode

Start OpenCode in your project directory:

opencode

It will connect to your local llama.cpp server on port 8080 using the model you specified. You can verify which model it's using with:

opencode models local

Using Different Models

To add more models, simply add entries under the models key:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "local": {
      "options": {
        "baseURL": "http://localhost:8080",
        "apiKey": "sk-dummy"
      },
      "models": {
        "qwen3": {},
        "gemma": {}
      }
    }
  },
  "model": "local/qwen3"
}

Switch between models in the OpenCode UI or via CLI:

opencode --model local/gemma

Note: llama.cpp serves only one model at a time. Switching models means restarting the llama.cpp server with the new model file.

Troubleshooting

undefined/chat/completions cannot be parsed as a URL
→ Your baseURL is at the wrong nesting level. Move it inside options.

OpenCode doesn't find your config
→ The file must be named opencode.json (no dot prefix). Run opencode debug paths to see the config directory, and opencode debug config to see the resolved configuration.

Connection refused
→ Make sure llama.cpp server is running and listening on the correct port. Test with curl "http://localhost:8080/v1/chat/completions".

401 Unauthorized
→ The proxy or server expects an API key. If you're connecting directly to llama.cpp (not through a proxy), this shouldn't happen. llama.cpp doesn't require authentication.

Multiple Providers

You can configure multiple providers in one file — for example, a direct llama.cpp connection and a proxy-based one:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "local": {
      "options": {
        "baseURL": "http://localhost:8080",
        "apiKey": "sk-dummy"
      },
      "models": {
        "qwen3": {}
      }
    },
    "local-proxy": {
      "options": {
        "baseURL": "http://localhost:8082",
        "apiKey": "sk-dummy"
      },
      "models": {
        "qwen3": {}
      }
    }
  },
  "model": "local/qwen3"
}

Switch between them with the --model flag or from the OpenCode model picker.

References