Working with Storage¶

Storage types on PAIS¶

Type	Where	Persists?	Shared?	Best for
Workspace PVC	Notebook local volume	Yes (until deleted)	No	Code, small experiments
Group VAST PVC	`/vast/rg-<group>/`	Yes	Yes (group members)	Datasets, checkpoints, results
Model cache PVC	`/vast/model-cache/`	Yes	Platform-wide	Downloaded HuggingFace models

Accessing VAST storage in notebooks¶

Your group's VAST storage is mounted at /vast/rg-<group>/ inside notebooks that have the data volume attached.

from pathlib import Path

data_dir = Path("/vast/rg-compsci/datasets/my_project")
output_dir = Path("/vast/rg-compsci/results/my_project")
output_dir.mkdir(parents=True, exist_ok=True)

# Load data
import pandas as pd
df = pd.read_csv(data_dir / "train.csv")

# Save results
df.to_csv(output_dir / "processed_train.csv", index=False)

Using the model cache¶

Pre-downloaded HuggingFace models are stored in the shared model cache, avoiding repeated downloads:

import os
os.environ["HF_HOME"] = "/vast/model-cache/huggingface"

from transformers import AutoModelForCausalLM, AutoTokenizer

# This reads from cache if available — no internet download
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")

Contact the PAIS team to pre-cache a model you'll use repeatedly.

File permissions¶

VAST uses root-squash: files created by notebooks may appear as nobody-owned in other contexts. If you encounter permission errors when another process reads files you wrote:

# From a terminal in your notebook
chmod -R 755 /vast/rg-compsci/results/my_project

The PAIS platform includes a UID patcher for PVCViewer uploads — contact the team if you're seeing systematic permission issues.

Uploading data¶

Use the Kubeflow PVCViewer tool to upload files to VAST storage via the web interface:

Kubeflow → Volumes → find your group PVC
Click the browse icon
Use the upload button

For large uploads, use scp or rsync via an SSH connection to the control plane node (requires cluster access).

Storage quotas¶

Research group	VAST quota
`rg-compsci`	Contact PAIS team
`rg-abi`	Contact PAIS team

Contact the PAIS team to request a quota increase.