Available Models¶

Chat models¶

`llama-3-1-8b`¶

Property	Value
Full name	Meta Llama 3.1 8B Instruct
Quantisation	AWQ INT4 (4-bit)
Context window	128k tokens
GPU memory	~8 GB VRAM (INT4)
Strengths	General instruction following, code, summarisation, structured output

Llama 3.1 8B is the primary chat model on PAIS. The AWQ INT4 quantisation fits within the GPU's time-sliced allocation while preserving most of the full-precision model's capability.

Good for: - Summarising research papers - Generating code with explanation - Q&A over domain documents (with RAG) - Multi-step reasoning chains - Structured JSON output with response_format

Limitations: - 8B parameters — complex multi-step reasoning may require prompt chaining - AWQ quantisation introduces minor quality degradation versus FP16 - Shared GPU — latency varies with concurrent load

Embedding models¶

`qwen3-vl-embedding-8b`¶

Property	Value
Full name	Qwen3-VL-Embedding-8B
Architecture	ColQwen3 (multi-vector pooler)
Serving endpoint	`/pooling` (mapped to `/v1/embeddings` by Kong)
GPU memory	~16 GB VRAM
Strengths	Multi-modal embeddings, document retrieval, scientific text

The primary embedding model on PAIS. Produces dense vectors suitable for semantic search over research corpora.

Good for: - Semantic search over paper abstracts, code, or notes - RAG pipelines - Document clustering - Cross-modal retrieval (text ↔ figures, future capability)

Listing models programmatically¶

models = client.models.list()
for m in models.data:
    print(m.id, m.object)

Cloud-brokered models¶

When local GPU capacity is insufficient, LiteLLM can route requests to cloud-hosted models (GCP Vertex AI, AWS Bedrock). This is configured at the platform level by the PAIS team and is transparent to clients — the same API endpoint is used.

Contact the PAIS team if your workload requires: - Models larger than 8B parameters - Guaranteed low-latency inference SLAs - Closed-source frontier models (GPT-4o, Claude, Gemini)

Cloud-brokered usage is tracked separately and may be subject to additional cost attribution.