Skip to content

Inference API

An OpenAI-compatible endpoint over open-weight models, billed from the same balance as everything else.

The inference endpoint speaks the OpenAI API. If your code already talks to one, it will talk to this by changing the base URL and the key.

from openai import OpenAI

client = OpenAI(
    api_key="<your inference key>",
    base_url="<the endpoint your operator gave you>",
)

client.chat.completions.create(
    model="<a model from the catalog>",
    messages=[{"role": "user", "content": "Hello"}],
)

Inference keys are not API keys

Two different credentials, and mixing them up is the most likely first mistake.

An API key drives the platform — deploying workloads, reading usage. An inference key calls models and does nothing else. It belongs to a project, not to you personally, so usage is attributable to the project that holds it and a colleague can revoke one you minted.

Mint them from the project, or from the Models view. Like every other credential here, the value is shown once.

Which models

The Models view is the catalog: what is available to you, and what each one costs. A key can be restricted to specific models, which is worth doing for anything running unattended — a key that may only call one small model cannot run up a bill on the largest.

A request for a model your key is not allowed to use is refused outright rather than quietly served by something else.

What it costs

Usage draws on the same credit balance as your workloads. There is no separate inference wallet to keep topped up.

Billing is per model and per token, and input tokens the provider served from its cache are billed at the cached rate, not the full prompt rate. On long conversations, where most of the prompt is resent each turn, that is the difference between a bill that makes sense and one that does not.

Usage is recorded per request. The Models view shows what has been spent.

When a call is refused

402 — the balance is empty. Every model call is checked against it before the request is forwarded, so an exhausted balance stops inference the same way it stops workloads. Top up and retry.

403 — the key is not allowed that model. Check the key’s model list rather than the catalog; the model may exist and simply not be one this key may call.

401 — the key is wrong, revoked, or you have sent an API key where an inference key belongs.

What this is not

The endpoint is the serving layer, not a training or fine-tuning service. It serves open-weight models the operator has published; you cannot upload weights to it.

If you want to run your own model, that is a workload with a GPU — deploy it like anything else, and it is billed as compute rather than per token.