Inference API¶

PAIS exposes an OpenAI-compatible REST API via the Kong API gateway. Any client, SDK, or framework that supports OpenAI's Chat Completions and Embeddings APIs works with PAIS without modification.

Base URL¶

https://api.pais.auckland.ac.nz/v1

Note

Your welcome email or the services page will confirm the exact URL for your environment.

Supported endpoints¶

Endpoint	Description
`POST /chat/completions`	Generate text from a chat conversation
`POST /embeddings`	Generate embedding vectors from text
`GET /models`	List available models

Authentication¶

All requests require a Bearer token:

Authorization: Bearer pais-sk-...

See Authentication for full details.

Available models¶

Model ID	Type	Context	Notes
`llama-3-1-8b`	Chat	128k tokens	Llama 3.1 8B Instruct, AWQ INT4 quantised
`qwen3-vl-embedding-8b`	Embeddings	—	Qwen3-VL-Embedding-8B via ColQwen3 pooler

See Available Models for benchmarks, capabilities, and selection guidance.

Quick reference¶

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.pais.auckland.ac.nz/v1",
    api_key=os.environ["PAIS_API_KEY"],
)

# Chat
response = client.chat.completions.create(
    model="llama-3-1-8b",
    messages=[{"role": "user", "content": "Explain gradient descent."}],
)

# Embeddings
embedding = client.embeddings.create(
    model="qwen3-vl-embedding-8b",
    input="Neural network training",
)

Gateway architecture¶

Requests flow through two layers before reaching the model:

Client  →  Kong Gateway (auth, rate-limit)
        →  LiteLLM Proxy (model routing)
        →  vLLM (inference, GPU)

Kong handles authentication and rate limiting. LiteLLM provides model aliasing and can route to cloud-brokered models (GCP, AWS) when local GPU capacity is exhausted. vLLM runs the actual inference on the L40S GPU.