[FEATURE] Validated AMD (Vulkan) example InferenceService + benchmark

[Defilan]

Feature Description

A validated, end-to-end example of serving a model on the AMD node (Vulkan tier), plus a benchmark entry so the AMD tier has the same "here is a working manifest and here are the numbers" treatment as CUDA and Metal.

Problem Statement

As someone evaluating LLMKube on AMD, I want a known-good example manifest and real tokens/sec, so that I can trust the tier works and size my hardware.

The other tiers have validated examples and benchmark numbers. The AMD tier needs the same proof, and it doubles as the acceptance test that #696 actually landed.

Proposed Solution

• A Model + InferenceService example manifest targeting the AMD node (vendor amd, Vulkan runtime image, GPU layer offload set for the unified-memory budget).
• A documented end-to-end run: deploy, hit the OpenAI-compatible endpoint, confirm GPU offload, record decode/prefill tokens/sec at a couple of context lengths.
• A benchmark entry alongside the existing CUDA/Metal numbers (a MoE model such as a Qwen3 30B-class is a good showcase for the 90GB unified pool).
• Wire it into the heterogeneous-fleet story: this node becomes a real backend tier the gateway ([FEATURE] First-class Envoy AI Gateway integration: make the gateway fleet-aware #661) and router can target.

Alternatives Considered

• Folding this into the runtime-image issue: kept separate so the image PR stays focused and the example carries the reproducible numbers.

Additional Context

• Related issues: [FEATURE] Epic: first-class AMD accelerator tier (Strix Halo / RDNA, Vulkan + ROCm) #696 (epic), AMD Vulkan image, node runbook, [FEATURE] First-class Envoy AI Gateway integration: make the gateway fleet-aware #661 (gateway can route to this backend)
• Pairs with the homelab AI Gateway lab-acceptance scenario: the AMD node is a candidate fallback/second tier in a heterogeneous failover demo.

Priority

• High - Would significantly improve my workflow

Willingness to Contribute

• Yes, I can submit a PR

[Defilan]

First validated AMD (Vulkan) end-to-end run

Ran a Strix Halo node (Ryzen AI Max+ 395, gfx1151 / Radeon 8060S, 128GB unified) joined to a MicroK8s cluster, accelerator=amd, /dev/dri exposed via a generic device plugin.

Image: ghcr.io/ggml-org/llama.cpp:server-vulkan-b5974 (pinned: the floating :server-vulkan tag is broken, see #725)
Model: bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_M, -ngl 99

GPU offload confirmed:

ggml_vulkan: 0 = AMD Radeon Graphics (RADV GFX1151) (radv) | uma: 1 | fp16: 1 | matrix cores: KHR_coopmat
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon Graphics (RADV GFX1151)) - 64573 MiB free
load_tensors: offloaded 29/29 layers to GPU

OpenAI-compatible /v1/chat/completions returned a correct answer with proper chat.completion shape (system_fingerprint: b5974-a12363bb).

Benchmark (Q4_K_M, n_ctx 4096, short prompt):

metric	value
decode	87.5 tok/s
prefill	268 tok/s

This is the small-model smoke point. Remaining for this issue: a checked-in Model + InferenceService example manifest (vendor amd), and a larger MoE showcase (Qwen3 30B-class) to exercise the ~90GB unified pool and record decode/prefill at a couple of context lengths. Note the runtime image must be the pinned/own-built Vulkan image per #725, and the device-plugin resource is devic.es/dri-render with both renderD128 + card1 exposed (see notes on #698/#697).