Connect OpenAI SDK (Python & JS) to OpenLLM Buddy

Use the official OpenAI Python and JavaScript SDKs with OpenLLM Buddy. Change base_url, keep the rest of your code, and stream responses.

SDK & framework · Integration guide

What is OpenAI SDK (Python & JS)?

The OpenAI SDKs are the most widely used client libraries for chat-completion APIs. Almost every tutorial and framework is written against them. Visit OpenAI SDK (Python & JS).

Why connect OpenLLM Buddy?

OpenLLM Buddy implements the same /v1/chat/completions shape, so existing code moves over by changing two values: base_url and api_key.

Before you start, create an API key in the console and deploy a model. If anything fails, run the curl test in Step 5 first: no tool can fix a broken endpoint.

1

Install the SDK

Pick the language you use.

Terminal
bash
# Python
pip install openai

# JavaScript / TypeScript
npm install openai
2

Python

Create the client with your base URL and key, then call chat completions as usual.

chat.py
python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://openllmbuddy-proxy.botbuddytech.workers.dev/v1",
    api_key=os.environ["OPENLLMBUDDY_API_KEY"],
)

resp = client.chat.completions.create(
    model="gemma4:26b",
    messages=[{"role": "user", "content": "Write a haiku about GPUs."}],
)
print(resp.choices[0].message.content)
3

JavaScript / TypeScript

The Node client takes the same two options, with the URL under baseURL.

chat.mjs
javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://openllmbuddy-proxy.botbuddytech.workers.dev/v1",
  apiKey: process.env.OPENLLMBUDDY_API_KEY,
});

const resp = await client.chat.completions.create({
  model: "gemma4:26b",
  messages: [{ role: "user", content: "Write a haiku about GPUs." }],
});
console.log(resp.choices[0].message.content);
4

Stream tokens

Set stream=True to receive tokens as they are generated.

stream.py
python
stream = client.chat.completions.create(
    model="gemma4:26b",
    messages=[{"role": "user", "content": "Explain KV caching in two paragraphs."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

Store the key in an environment variable such as OPENLLMBUDDY_API_KEY, never in source control.

5

Test the endpoint with curl

Confirm the key, URL, and model id work before debugging OpenAI SDK (Python & JS). A short JSON reply means the endpoint is healthy.

Terminal
bash
curl -sS https://openllmbuddy-proxy.botbuddytech.workers.dev/v1/chat/completions   -H "Authorization: Bearer YOUR_API_KEY"   -H "Content-Type: application/json"   -d '{"model": "gemma4:26b", "messages": [{"role": "user", "content": "Reply with OK"}], "max_tokens": 16}'

Troubleshooting

  • Wrong host — if requests still hit api.openai.com, base_url was not passed to this client instance.
  • `/responses` errors — newer SDK helpers use the Responses API. Stick to chat.completions, which is what the endpoint serves.
  • 401 or 403 — the API key is wrong, revoked, or missing the Bearer prefix. Create a fresh key in the console.
  • 404 on /chat/completions — the base URL must end in /v1, with no trailing /chat/completions and no extra path.
  • Model not found — the model id must match your deployment exactly. Run the curl test and copy the id you used.
  • Timeouts on the first request — a GPU that was idle may still be loading the model. Retry after a minute.
  • Connection refused from Docker or a remote host — use the public HTTPS base URL, never localhost.

Connection details

Use these values wherever OpenAI SDK (Python & JS) asks for an OpenAI-compatible provider.

Reference
text
Base URL:  https://openllmbuddy-proxy.botbuddytech.workers.dev/v1
Chat URL:  https://openllmbuddy-proxy.botbuddytech.workers.dev/v1/chat/completions
API key:   your key from the console
Model id:  gemma4:26b