This is the setup, the run prompt and the scoring I used for Qwen3.8-Flash-Next vs Claude Opus 5.5, made generic so you can run it with your own models, harnesses and hardware. Everything here is public. My 12 PRs are in deepu105/llamastash-b4r2 if you want to compare.
- Repo: llamastash/llamastash, latest
main. Write down the SHA you start from and use the same one for every run. The feature isn't in LlamaStash yet, it's the openTODO.mditem "Keep a model's prompt cache across an unload". I used623c28ca11c7f10969208ff4a81392e795f08a31if you want the exact same starting point. - A scratch GitHub repo for the PRs, with only
mainat your start SHA. Turn off its scheduled workflows. All runs open their PRs against it. - llama-server: a recent [llama.cpp