Storage¶
VAST Data Platform¶
PAIS uses a VAST network-attached storage system as the shared storage backend. It provides high-throughput, low-latency access to large datasets, model checkpoints, and experiment artifacts across all nodes simultaneously.
| Property | Value |
|---|---|
| Technology | VAST Data Platform |
| VIP | 10.60.200.240 (C302-pool) |
| Management endpoint | 10.64.36.10 |
| Total allocation | 10 TB (model cache) + group quotas |
| Access method | VAST CSI Driver 2.6.5 via Kubernetes PVCs |
| Reclaim policy | Retain (data survives PVC deletion) |
Storage classes¶
| StorageClass | Reclaim policy | Use |
|---|---|---|
vast-retain |
Retain | Research data, model weights |
vast-retain-rwo |
Retain | ReadWriteOnce workloads |
vast-retain-rwx |
Retain | ReadWriteMany (shared notebooks) |
PVC layout¶
| PVC | Namespace | Mount path | Purpose |
|---|---|---|---|
model-cache-pvc |
pais-inference |
/model-cache |
HuggingFace model downloads |
rg-compsci-data |
rg-compsci |
/vast/rg-compsci |
Group datasets and results |
rg-abi-data |
rg-abi |
/vast/rg-abi |
Group datasets and results |
mlflow-artifacts |
mlflow |
/mlflow/artifacts |
MLflow run artifacts |
Root-squash behaviour¶
VAST uses root-squash: processes running as root inside containers have their UID mapped to nobody (UID 65534) at the storage level. This can cause unexpected ownership on files written by privileged containers.
Mitigation: The pvcviewer-uid-patcher deployment ensures PVCViewer pods run as UID 1000. For other cases, apply appropriate file permissions after writing: