#128 How an LLM Runs on a GPU: Qwen3-8B, Memory vs Compute
AI Software Engineering GPUI wrote this while moving a JSON-extraction service (a quantized 8B model on a single T4, around ten concurrent users) to a bigger GPU on Azure. The question “which GPU, and how fast will it be?” turned out to be answerable with arithmetic you can do on paper. The numbers in the widgets all come from the model’s config, so you can check every one of them.