← All field notes

Field notes / Local inference

Local AI,
without the mythology.

What running a 36-billion-parameter model on consumer hardware taught me about memory, GPU acceleration, agent overhead, and misleading benchmarks.

Qwen 3.6 · 36B Q4 quantization 23 GB model Intel Arc iGPU Ollama + Vulkan

I set out to run a large language model entirely on my own computer. The model loaded. It answered questions. Technically, the experiment worked. But “it runs” and “it is usable” turned out to be very different milestones.

01 / Capacity

Enough RAM does not mean good performance

A common assumption is that a computer can run a large model well as long as it has plenty of memory. My machine had about 62 GB of system RAM, more than enough to hold a 23 GB model. Ollama still started in CPU-only mode.

The assumption

“If the model fits in memory, the machine should run it quickly.”

What actually matters

Memory determines whether the model fits. Compute and memory bandwidth determine how fast it runs.

[01]CapacityCan the model fit in available memory?
[02]ComputeHow quickly can the hardware perform the math?
[03]BandwidthHow quickly can model data move through memory?

A dedicated GPU combines specialized compute with fast, dedicated VRAM. An integrated GPU shares ordinary system memory. That shared pool can make a large model possible, but it does not turn an integrated GPU into a high-end discrete card.

GPU offloading also does not guarantee a speedup. The honest test is to benchmark CPU and GPU modes using the same model, prompt, and context size.

02 / Configuration

The GPU existed. Ollama still could not use it.

The computer had Intel Arc integrated graphics and a working Vulkan driver. Two software details kept Ollama on the CPU:

  1. Arch Linux packages Ollama's Vulkan backend separately.
  2. Ollama ignores integrated GPUs unless they are explicitly enabled.
  configuration
# Install the missing backend
sudo pacman -S ollama-vulkan

# Systemd service override
[Service]
Environment="OLLAMA_IGPU_ENABLE=1"

# Result
library=Vulkan
device=Intel(R) Arc(tm) Graphics (MTL)
offloaded 42/42 layers to GPU

After the restart, Ollama reported 100% GPU placement and roughly 46.6 GB of shared GPU memory. That proved the configuration worked. It did not, by itself, prove that every workload would be faster.

03 / Agent overhead

The 20,000-token “hi”

I sent the word hi to the model twice. Directly through Ollama, the model received 11 input tokens. Through a full Hermes agent, it received 20,011.

Hermes agent
20,011 tokens
Direct Ollama
11 tokens

Hermes was not only sending my greeting. It attached its operating instructions, definitions for 29 tools, a list of 68 skills, memory rules, browser controls, terminal schemas, delegation instructions, and other agent behavior.

  fixed prompt budget
System prompt                 22,243 characters
Tool definitions              61,775 bytes
Skills index                   6,950 characters

Largest tool schemas:
  computer use                 9,699 bytes
  cron jobs                    7,314 bytes
  session search               6,457 bytes
  browser                      6,341 bytes
  file tools                   6,143 bytes
The model was not taking five minutes to understand “hi.” It was taking five minutes to read the technical manual attached to it.

Hosted models often process this overhead quickly enough that we barely notice it. Local models make the hidden cost visible. A capable agent harness is useful when the task requires tools; for casual chat, it can be expensive baggage.

04 / Measurement

Tokens per second can tell the wrong story

The Hermes request appeared to achieve 73 combined tokens per second. The direct Ollama request managed only about 12. That sounds like Hermes was faster. It was not.

RequestInputOutputPrompt rateGeneration rate
Full Hermes20,0113574.05 t/s9.74 t/s
Direct Ollama1118020.44 t/s11.32 t/s
Stage one Prompt processing

Reads many input tokens in parallel batches. Long prompts can keep the GPU busy and produce a high rate.

Stage two Generation

Produces one new token at a time. Each token depends on the token before it, so this stage is slower.

The Hermes average was dominated by 20,011 fast, batched input tokens. The direct request was dominated by 180 slower, sequential output tokens. Combining both stages into one number hid the part users actually experience.

Three numbers worth reporting

  • Prompt throughput: how quickly the model reads the input.
  • Generation throughput: how quickly it writes the response.
  • Time to first token: how long the user waits before anything appears.
Misleading comparison

“73 tokens per second is faster than 12 tokens per second.”

Fair comparison

Direct Ollama actually generated text faster: 11.32 versus 9.74 output tokens per second.

05 / Hidden work

A short answer may still involve a long internal monologue

The direct hi request generated 180 tokens, even though the visible task was trivial. Qwen is a reasoning model, so some of that work can happen before the final answer appears.

Reasoning tokens still take time. For planning, coding, and difficult analysis, the extra work may be useful. For a greeting, it is waste. Matching reasoning effort to the task can improve perceived speed without changing hardware.

  direct Ollama /api/chat
Input prompt                 11 tokens
Output                       180 tokens
Prompt processing            538 ms
Generation                   15.90 s
Total request                16.66 s
Estimated first token        626 ms
GPU placement                100% Vulkan
06 / Architecture

A faster runtime cannot remove an oversized prompt

We considered llama.cpp, OpenVINO GenAI, and vLLM. A different runtime may improve raw inference, but it cannot remove instructions inserted by the agent layer.

Ollama already uses a llama.cpp-derived server. Running llama.cpp directly offers more tuning controls, while OpenVINO is an interesting Intel-specific experiment. Neither one changes the fact that a 20,000-token request contains 20,000 tokens.

Agent harness Decides what gets sent

System instructions, tools, skills, memory, conversation history, and the user's message.

Inference runtime Decides how it runs

GPU backend, quantization, batching, context allocation, caching, and token generation.

The highest-impact fix depends on the bottleneck. Reduce the harness when the prompt is bloated. Tune or replace the runtime when raw prompt and generation rates are poor.

07 / What I would do next time

Practical takeaways

  1. Separate capacity from speed.RAM may let the model load. Compute and bandwidth determine how well it responds.
  2. Verify the compute device.Do not assume a GPU is active. Check runtime logs, layer placement, and the serving process.
  3. Benchmark CPU and GPU fairly.Use the same model, quantization, context, and prompt before declaring a winner.
  4. Inspect the real input size.A one-word message can become a 20,000-token agent request.
  5. Match the harness to the job.Use a lightweight chat interface for conversation and a full agent only when its tools are needed.
  6. Report the right metrics.Track prompt rate, generation rate, first-token latency, model load time, and total time separately.
  7. Tune reasoning deliberately.Deep reasoning is valuable for hard work and unnecessary for a greeting.
  8. Treat local AI as a stack.The model, quantization, runtime, drivers, memory, context, and harness all affect the result.
Final perspective

“It runs locally” is the beginning of the story.

The model may fit but run slowly. The GPU may exist but remain unused. The GPU may be active without beating the CPU. A powerful agent may spend more time explaining its tools to the model than answering the user.

Instead of asking for one headline benchmark, ask a few more useful questions:

01How many tokens did the system actually send?
02How quickly did the model process them?
03How long before the user saw the first token?
04How quickly was the answer generated?
05Was all of that context necessary?