My Embedding Cache Remembered the Model Name. That Wasn’t Its Whole Identity.
A source inspection of my bookmark vector cache found what model-name checks, text hashes, and dimension guards do—and what they cannot prove.
Benchmarks and real-world performance of language models running on ordinary consumer hardware.
A source inspection of my bookmark vector cache found what model-name checks, text hashes, and dimension guards do—and what they cannot prove.
I revisited eight saved Qwen3-Coder requests to separate generation speed, first-token delay, and total waiting time.
Remote requests timed out while local requests and SSH still worked. Inspecting the Windows desktop revealed a pending prompt; remote requests recovered after Allow.
Moving an idle occupier from one GPU layer to two coincided with a 29% generation-speed drop. The 70MB counter delta does not establish the cause.
At depth 7,168, compressed cache reached 5.27 tok/s versus 9.31 for fp16 in the same fork. A later run exited with 0xC0000409; the crash cause is unconfirmed.
Ten technical Korean-to-English cases completed without errors, yet several outputs changed the meaning. I retained review instead of automating unchecked posting.
One GPU occupancy comparison showed 28.15 versus 41.00 tok/s. A server restart separates the conditions, and the memory-spill mechanism was not directly observed.
A mismatched VAE ended a WAN 2.2 run after 28 minutes of sampling. The successful retry used different settings, so its timing is not a controlled comparison.
A resident whisper server reduced the recorded STT stage from about 9.9 to 1.0 seconds. End-to-end latency and answer-quality tradeoffs were not measured.
Two local embedding models ranked the same 24 bookmarks identically — 8 of 8 correct each. Their similarity scores sat in almost non-overlapping ranges, and one hardcoded 0.75 cutoff silently erased every result from one of them.