vLLM API server

This app is an OpenAI-compatible API, not a website. There is nothing to log in to here: point any OpenAI-style client at this domain with the API key.

Checking model status…

Your API key

Generated on first run. In the Cloudron dashboard, open a Terminal for this app and run cat /app/data/.secrets/keys.env.

First request

curl https://THIS-DOMAIN/v1/chat/completions \
  -H "Authorization: Bearer YOUR-KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "MODEL-NAME", "stream": true,
       "messages": [{"role": "user", "content": "Hello"}]}'

Always use "stream": true: this is CPU inference and long non-streamed responses can hit the platform's 60 second proxy timeout. List the served model name with GET /v1/models (needs the key).

Endpooints

/v1/chat/completions, /v1/completions, /v1/models (key required) · /health liveness · /ready readiness (200 once the model is loaded) · /metrics Prometheus.