Skip to content

Architecture

Platform overview

PAIS is a Kubernetes-based AI platform running on university-owned hardware. Public access is via Cloudflare Tunnel (no open inbound firewall ports required).

flowchart TD
    U[Researcher] --> CF[Cloudflare Tunnel]
    CF --> KG[Kong Gateway\nAuth + Rate Limiting]

    KG --> WebUI[Open WebUI\nChat interface]
    KG --> Portal[PAIS Portal\nAPI key management]
    KG --> LLM[vLLM\nLlama 3.1 8B]
    KG --> Embed[vLLM\nEmbeddings]

    KG --> LiteLLM[LiteLLM Proxy\nModel routing]
    LiteLLM --> LLM
    LiteLLM --> Embed
    LiteLLM --> Cloud[Cloud models\nGCP / AWS]

    U2[Researcher] --> KF[Kubeflow\nNotebooks + Pipelines]
    KF --> GPU[L40S GPU\nTime-sliced ×4]
    KF --> VAST[VAST Storage\n10 TB]
    KF --> MLflow[MLflow\nExperiment tracking]

Request flow for API calls

Researcher's code
    │  Authorization: Bearer pais-sk-...
    ▼
Cloudflare Tunnel (TLS termination)
    ▼
Kong Gateway (port 30080)
    ├── key-auth plugin: validates API key
    ├── rate-limit plugin: 60 req/min per key
    └── proxy_pass → /v1/*
    ▼
LiteLLM Proxy (port 4000)
    ├── model routing (llama-3-1-8b → vLLM chat)
    └── model routing (qwen3-vl-embedding-8b → vLLM /pooling)
    ▼
vLLM (port 8000 / 8001)
    └── inference on L40S GPU

Cluster nodes

Node Role Hardware
c302-k8s-dev0 Control plane + worker 4 vCPU, 16 GB RAM
c302-k8s-dev0-vgpu GPU worker 4 vCPU, 16 GB RAM, L40S 48 GB
c302-k8s-dev1 CPU worker 2 vCPU, 8 GB RAM

Multi-tenancy

Each research group has an isolated Kubernetes namespace:

  • ResourceQuota — limits GPU, CPU, memory, and storage per namespace
  • LimitRange — sets default container resource requests
  • NetworkPolicy — restricts pod-to-pod communication between groups
  • RBAC — group members can only access their own namespace

Key design decisions

Single PostgreSQL instance — shared backend for Kong, Keycloak, Kubeflow Pipelines, MLflow, and Katib. Simplifies operations; acceptable risk for a dev/research cluster.

Time-slicing over MPS — the L40S is time-sliced (not Multi-Process Service). Time-slicing was chosen for simpler driver support and broader workload compatibility.

Outbound-only public access — Cloudflare Tunnel means no inbound firewall rules are needed on the university network.

Intent-based deployment — no CI/CD pipeline. Specs describe what to build; engineers (or Claude Code agents) determine commands by inspecting live cluster state.