Spinifex Vision Pipeline on Cisco UCS

Instructions

1. Provision the three instances

cd terraform/
terraform apply

One g6.2xlarge (GPU) and two m8i.2xlarge (CPU) instances in a spread placement group — Spinifex schedules spread-group members onto distinct physical nodes, so each instance lands on a different one of the three. The gpu instance must land on the one node with GPU passthrough configured; the spread group guarantees the two CPU instances land on the remaining two nodes. A dedicated Viperblock data volume per instance (separate from the root volume, delete_on_termination = false) survives instance termination/replacement independently — relaunching a terminated instance reattaches the same volume rather than starting from empty storage.

gpu_instance  = { id = "i-...", public_ip = "192.168.12.151", type = "g6.2xlarge" }
cpu_instances = [
  { id = "i-...", public_ip = "192.168.12.152", type = "m8i.2xlarge" },
  { id = "i-...", public_ip = "192.168.12.154", type = "m8i.2xlarge" },
]

2. CPU (Intel AMX) vs GPU precision comparison

Before running the live pipeline, this section characterises each accelerator's throughput and accuracy on YOLO11m in isolation. Intel AMX (Advanced Matrix Extensions) is a relatively new instruction set, first introduced with 4th-gen Xeon Scalable (Sapphire Rapids, 2023) and supported by the Xeon 6543P-B CPUs in these nodes, designed to accelerate the matrix operations that dominate AI and ML workloads. Running AMX head-to-head against the NVIDIA L4 directly answers how much a modern, AI-targeted CPU instruction set can close the gap to GPU without any discrete accelerator.

YOLO11m at 640×640, COCO val2017 (5,000 images), fixed image order, 20-image warm-up excluded, COCO mAP validated per configuration (not just throughput — an unvalidated speed number is not a valid quantization comparison). CPUID flags alone don't prove AMX is executing — the proof is oneDNN's own kernel-selection trace during real inference. On this hardware, BF16 workloads select avx10_1_512_amx (the current naming scheme for this CPU generation — not the older brgemm_avx512_amx* string some documentation references); FP32 selects avx512_core only, with zero AMX selections. The table below measures what that kernel difference is worth on YOLO11m:

BF16/AMX gives a real 2.4–3.1x throughput gain over true FP32 on this hardware, at essentially zero accuracy cost (Δ mAP50-95 = −0.0001). INT8 costs a small but real −0.9% relative mAP and, counter to the usual expectation, ran *slower* than BF16 — the default quantization left some layers unquantized, introducing dequant/requant overhead that BF16's uniform precision avoids entirely.

GPU beats the best CPU path by ~3x at only 22–35% GPU utilisation — headroom-rich, with host-side pre/post-processing (letterbox, NMS) bundled into the reported figures alongside GPU inference itself.

3. Local volume vs Predastore-streamed

Same engines, now reading frames over S3 and writing detections back, instead of from the instance's own local Viperblock volume:

A solo cpu1 S3 run (no concurrent workers) scored 12.53 img/s — essentially identical to its 3-worker-concurrent number (13.44). Concurrency from the other two workers cost cpu1 almost nothing — the clearest evidence the storage-request latency itself, not contention between workers, is the limiter. Per-image latency breakdown confirms it: GPU's S3 round-trip (read+write ≈ 30.6 ms) is ~5x its actual inference cost (5.7 ms) — each frame is a small object (~163 KB), so this is a per-request-latency-bound access pattern, not a bandwidth-bound one. A production pipeline would batch/pipeline S3 reads rather than one GET per frame.

The ~30 ms round-trip at this object size (~163 KB) is dominated by fixed per-request cost — HTTPS/TLS, sigv4 signing, and the OVN gateway-chassis hop — rather than bandwidth. S3 latency grows only ~24–40% as workers scale while inference time grows ~11x, confirming the round-trip is a fixed per-request tax, not a contention effect.

4. Demo dashboard

cd demo-dashboard/
./venv/bin/uvicorn server:app --host 0.0.0.0 --port 8090

A local FastAPI dashboard renders the pipeline live: the annotated detection feed, the VLM's scene caption, GPU utilisation/power, Predastore CPU%, and per-instance Predastore GetObject latency.

Of particular note during the video below are the spikes in Predastore CPU %, attributed to its automatic compaction process.

5. Teardown

cd terraform/
terraform destroy

All three instances terminate and the NVIDIA L4 is immediately returned to the Spinifex GPU pool. The dedicated Viperblock data volumes (delete_on_termination = false) survive instance termination but are destroyed explicitly by Terraform here — any results or model artefacts worth keeping should be copied to Predastore before running this step.

6. Conclusion

This pipeline demonstrates a mixed CPU/GPU edge-AI workload — Intel AMX-accelerated detection and NVIDIA L4-accelerated captioning, running as independent EC2-compatible instances against a shared S3-compatible object store, on Cisco Unified Edge hardware managed entirely through standard AWS tooling. AMX delivers a real, accuracy-neutral 2.4–3x throughput gain on this hardware; the L4 leaves significant headroom at this workload's current scale; and streaming from a central bucket rather than a private local volume costs 59–62% throughput, because each frame is a small, latency- rather than bandwidth-bound request. Teams already operating AWS infrastructure can point their existing tooling at a Spinifex node with a single profile swap — EC2 instances, Viperblock EBS volumes, Predastore S3 buckets, and OVN VPC networking all provisioned from the same Terraform resources that work on AWS.