Multi-Tenant AI on Supermicro H14 with AMD MI350X

Instructions

1. Provision the VMs

Each VM was provisioned using standard AWS EC2 CLI commands — the only change from a normal AWS workflow is AWS_PROFILE=spinifex, which redirects the CLI to Spinifex's local EC2-compatible endpoint instead of AWS:

Six of the eight available MI350Xs are claimed here; the remaining two stay idle on the host, available for a fourth tenant without touching the existing three.

export AWS_PROFILE=spinifex

# vm-yolo — YOLO11x object detection, 200 GB disk
YOLO_ID=$(aws ec2 run-instances \
    --image-id ami-ubuntu-amd-gpu \
    --instance-type g7e.12xlarge \
    --key-name spinifex-key \
    --subnet-id <subnet-id> \
    --security-group-ids <sg-id> \
    --block-device-mappings 'DeviceName=/dev/sda1,Ebs={VolumeSize=200,DeleteOnTermination=true}' \
    --count 1 \
    --query 'Instances[0].InstanceId' --output text)
echo "vm-yolo launched: $YOLO_ID"

# vm-vlm — Qwen3-VL 235B FP8, 600 GB disk for weights
VLM_ID=$(aws ec2 run-instances \
    --image-id ami-ubuntu-amd-gpu \
    --instance-type g7e.12xlarge \
    --key-name spinifex-key \
    --subnet-id <subnet-id> \
    --security-group-ids <sg-id> \
    --block-device-mappings 'DeviceName=/dev/sda1,Ebs={VolumeSize=600,DeleteOnTermination=true}' \
    --count 1 \
    --query 'Instances[0].InstanceId' --output text)
echo "vm-vlm launched: $VLM_ID"

# vm-chat — Llama 3.3 70B, 300 GB disk
CHAT_ID=$(aws ec2 run-instances \
    --image-id ami-ubuntu-amd-gpu \
    --instance-type g7e.12xlarge \
    --key-name spinifex-key \
    --subnet-id <subnet-id> \
    --security-group-ids <sg-id> \
    --block-device-mappings 'DeviceName=/dev/sda1,Ebs={VolumeSize=300,DeleteOnTermination=true}' \
    --count 1 \
    --query 'Instances[0].InstanceId' --output text)
echo "vm-chat launched: $CHAT_ID"

# Wait for all three to reach running state
for ID in "$YOLO_ID" "$VLM_ID" "$CHAT_ID"; do
    aws ec2 wait instance-running --instance-ids "$ID"
    echo "$ID is running"
done

# Retrieve IPs (Spinifex assigns addresses from the internal VPC)
YOLO_IP=$(aws ec2 describe-instances --instance-ids "$YOLO_ID" \
    --query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
VLM_IP=$(aws ec2 describe-instances --instance-ids "$VLM_ID" \
    --query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
CHAT_IP=$(aws ec2 describe-instances --instance-ids "$CHAT_ID" \
    --query 'Reservations[0].Instances[0].PublicIpAddress' --output text)

echo "vm-yolo: $YOLO_IP"
echo "vm-vlm:  $VLM_IP"
echo "vm-chat: $CHAT_IP"

2. Verify GPU access

Once SSH is available, confirm both GPUs are visible inside each VM:

ssh -i ~/.ssh/spinifex-key ubuntu@$CHAT_IP 'lspci | grep -i amd'

Or install and run amd-smi:

ssh -i ~/.ssh/spinifex-key ubuntu@$CHAT_IP \
    'curl -fsSL https://repo.radeon.com/rocm/rocm.gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg >/dev/null && \
     echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/6.3 noble main" | sudo tee /etc/apt/sources.list.d/rocm.list && \
     sudo apt-get update -qq && sudo apt-get install -y -q amd-smi-lib && \
     amd-smi list'

Two MI350X entries with distinct UUIDs confirm direct PCIe passthrough is working.

3. The orchestration layer

Spinifex provides an EC2-compatible API on bare metal. The relevant capabilities for this demo:

• Multi-GPU claim per VM. Each g7e.12xlarge requires two PCIe addresses claimed as a unit. Spinifex queries the available GPU pool on the host and assigns them to the guest VM at creation time, rolling back if the full allocation cannot be satisfied.
• PCIe passthrough via vfio-pci. All 8 MI350Xs are bound to vfio-pci on the bare-metal host. Each VM gets its GPUs at the hardware level.
• Internal VPC (192.168.10.0/24). VMs communicate directly without traversing the host's public network.

The result: three completely independent EC2 instances, each running their own separate workloads using only the GPU resources they were allocated.

4. Test methodology

Two capture runs were made against the same workload script: 30 s baseline (YOLO and VLM models running, no chat load) → 25 scripted Llama chat prompts at 15 s intervals → 30 s settle tail.

The only difference between the two runs was tensor parallelism:

• Demo 1 (TP=1): each VM uses one of its two allocated GPUs. The second sits idle.
• Demo 2 (TP=2): each VM shards its model symmetrically across both GPUs using AMD Infinity Fabric (XGMI).

5. Results

Inter-tenant isolation

The charts below show the three independent workloads spiking activity on their associated GPUs, measured in GFX utilisation and power draw. YOLO average FPS stays at ~17 despite utilisation in the YOLO VM never exceeding 20%, indicating a bottleneck in networking or encoding rather than inference. Each chart highlights workload independence — pulses of GPU activity share no correlation across VMs.

GPU utilisation per VM across the full Demo 1 capture.

Power draw per VM during Demo 2.

Single vs double GPU

When TP=2 is enabled, each VM shards its model symmetrically across both GPUs. Inside the chat VM, both GPUs pulse together on every request, each holding exactly 248 GB of VRAM (weights split + KV cache shards). With TP=1, GPU 1 sat at 0% for the entire run.

Per-GPU utilisation inside vm-chat with TP=2.

Same VM with TP=1.

vLLM generation throughput

With TP=1, Qwen peaks at ~60 tok/s and Llama caps around 35 tok/s — both compute-bound on a single MI350X. Enabling TP=2 lifts Llama's peak to ~50 tok/s (+47%) as generation is distributed symmetrically across both GPUs. Qwen's throughput is slightly lower under TP=2 due to collective overhead on the shared-memory transport.

Generation throughput during Demo 1 (TP=1).

Generation throughput during Demo 2 (TP=2).

6. Teardown

# Stop workloads on each VM
for IP in "$YOLO_IP" "$VLM_IP" "$CHAT_IP"; do
    ssh -i ~/.ssh/spinifex-key ubuntu@$IP 'docker stop $(docker ps -q)' || true
done

# Terminate all three instances — releases 6× MI350X back to the Spinifex pool
aws ec2 terminate-instances --instance-ids "$YOLO_ID" "$VLM_ID" "$CHAT_ID"

The six MI350Xs are immediately returned to the pool once the instances terminate, ready for a new allocation without touching the host.

7. Conclusion

This tutorial demonstrates how Spinifex can turn a single bare-metal chassis into a multi-tenant AI serving platform. Spinifex utilises the flexibility of PCIe passthrough to allocate GPU resources in whichever configuration is required by the workload/s.

Importantly, it does so with standard aws ec2 CLI calls — run-instances, describe-instances, terminate-instances — against Spinifex's EC2-compatible endpoint. Teams already operating AWS infrastructure can point their existing tooling at a Spinifex node with a single profile change, against GPUs that sit in their own rack.