Distributed GPU Compute for AI — Powered by QuanticCompute

On‑Demand 5090/4090

Top‑tier consumer GPUs available in minutes for inference and fine‑tuning.

Distributed Inference

Sharded or batched loads across nodes for steady, predictable throughput.

API‑First

Simple REST endpoints. Queue jobs, fetch logs, and retrieve results.

Encrypted Containers

Customer images pulled securely with scoped tokens, zero-trust by default.

Usage Transparency

Per‑job metrics: latency, tokens/s, VRAM, GPU util, and cost.

Vast.ai Compatible

Optionally burst to external capacity pools while keeping cost controls.

REST quick start

Replace YOUR_API_KEY and values as needed.

curl -X POST \
  https://api.localgpu.ai/v1/jobs \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "image": "nvcr.io/nvidia/pytorch:24.05-py3",
    "command": ["python","serve.py"],
    "resources": {"gpu": 1, "vram_gb": 24},
    "env": {"MODEL": "llama3-8b"}
  }'

# Poll job status
curl -H "Authorization: Bearer YOUR_API_KEY" \
  https://api.localgpu.ai/v1/jobs/JOB_ID

# Stream logs (if enabled)
curl -N -H "Authorization: Bearer YOUR_API_KEY" \
  https://api.localgpu.ai/v1/jobs/JOB_ID/logs

Python

import os, requests

API = "https://api.localgpu.ai/v1"
KEY = os.getenv("LOCALGPU_API_KEY", "YOUR_API_KEY")
headers = {"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"}

job = {
  "image": "nvcr.io/nvidia/pytorch:24.05-py3",
  "command": ["python","serve.py"],
  "resources": {"gpu": 1, "vram_gb": 24},
  "env": {"MODEL": "llama3-8b"}
}

r = requests.post(f"{API}/jobs", json=job, headers=headers)
r.raise_for_status()
job_id = r.json()["id"]

status = requests.get(f"{API}/jobs/{job_id}", headers=headers).json()
print(status)

Endpoints are illustrative. Your account may use different base URLs or versions.

How it works

1. Submit a job (REST or dashboard)
2. Scheduler assigns nodes (latency+cost)
3. Container runs with isolated volumes
4. Metrics & results stream back

Security

JWT‑scoped tokens, private registries, per‑job secrets, and optional customer‑managed keys.

Observability

Structured logs, per‑job metrics, and export to your stack.