Connect SillyTavern to OpenLLM Buddy

Connect SillyTavern to OpenLLM Buddy through the Custom OpenAI-compatible chat completion source for unrestricted roleplay and creative writing.

Chat UI · Integration guide

What is SillyTavern?

SillyTavern is a local front end for roleplay, character chats, and long-form fiction, with character cards, lorebooks, and fine control over prompt formatting. Visit SillyTavern.

Why connect OpenLLM Buddy?

Creative writing is where cloud models refuse most often. OpenLLM Buddy's unrestricted models keep scenes going, and SillyTavern's Custom source needs nothing more than a base URL and key.

Before you start, create an API key in the console and deploy a model. If anything fails, run the curl test in Step 4 first: no tool can fix a broken endpoint.

1

Open API connections

In SillyTavern, click the plug icon (API Connections) at the top.

  • Set API to Chat Completion.
  • Set Chat Completion Source to Custom (OpenAI-compatible).
2

Enter the endpoint and key

Fill in the connection fields.

  • Custom Endpoint (Base URL): https://openllmbuddy-proxy.botbuddytech.workers.dev/v1
  • Custom API Key: your OpenLLM Buddy key.
  • Model ID: gemma4:26b

Click Connect. A green status dot and the model name confirm it worked.

3

Tune sampling for fiction

Open AI Response Configuration and start with a moderate temperature (0.8 to 1.0), a generous Max Response Length, and a context size that matches your deployment. Add your preferred story prompt in the prompt manager.

Use only on content that is lawful where you live. OpenLLM Buddy's terms apply.

4

Test the endpoint with curl

Confirm the key, URL, and model id work before debugging SillyTavern. A short JSON reply means the endpoint is healthy.

Terminal
bash
curl -sS https://openllmbuddy-proxy.botbuddytech.workers.dev/v1/chat/completions   -H "Authorization: Bearer YOUR_API_KEY"   -H "Content-Type: application/json"   -d '{"model": "gemma4:26b", "messages": [{"role": "user", "content": "Reply with OK"}], "max_tokens": 16}'

Troubleshooting

  • Connect fails — run the curl test first. SillyTavern shows only a generic error when the endpoint answers with HTML or a 404.
  • Replies cut off mid-sentence — raise Max Response Length, and check the deployment's context length is not exhausted by a long chat.
  • 401 or 403 — the API key is wrong, revoked, or missing the Bearer prefix. Create a fresh key in the console.
  • 404 on /chat/completions — the base URL must end in /v1, with no trailing /chat/completions and no extra path.
  • Model not found — the model id must match your deployment exactly. Run the curl test and copy the id you used.
  • Timeouts on the first request — a GPU that was idle may still be loading the model. Retry after a minute.
  • Connection refused from Docker or a remote host — use the public HTTPS base URL, never localhost.

Connection details

Use these values wherever SillyTavern asks for an OpenAI-compatible provider.

Reference
text
Base URL:  https://openllmbuddy-proxy.botbuddytech.workers.dev/v1
Chat URL:  https://openllmbuddy-proxy.botbuddytech.workers.dev/v1/chat/completions
API key:   your key from the console
Model id:  gemma4:26b