A 200 OK does not mean your LLM used the GPU A low tok/s number cannot tell whether the model is slow or never made it onto the GPU. With 0/33 layers on the GPU, prefill fell 22x while the server kept returning 200.