Skip to content

Instantly share code, notes, and snippets.

@mdierolf
mdierolf / Qwen 3.8 Chat Template Modification.md
Last active August 15, 2026 20:28
Qwen 3.8 Chat Template Modification

Using the Modified Qwen3.8 “High” Reasoning Template with llama.cpp

Qwen3.8-27B supports reasoning_effort, but the stock Unsloth template currently exposes only three distinct behaviors: xhigh, medium, and low. In fact, Unsloth's current template explicitly converts high into xhigh, so sending "reasoning_effort": "high" does not give you an intermediate reasoning level.

The modification described in the Reddit post adds a real high mode by giving it a prompt that deliberately blends the model's low and xhigh reasoning instructions. The author reports that this produces reasoning substantially shorter than xhigh while retaining more of its quality than the stock medium mode.

This tutorial shows how to download Unsloth's current Qwen3.8 template, modify it, and tell llama.cpp to use your modified version instead of the template embedded in the GGUF.

1. Install a recent llama.cpp

@mdierolf
mdierolf / gist:b081f598988159e92d91abf81d2619a4
Created August 15, 2026 08:08
Qwen 3.8 27B Chat Template
{%- set image_count = namespace(value=0) %}
{%- set video_count = namespace(value=0) %}
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
{%- if content is string %}
{{- content }}
{%- elif content is iterable and content is not mapping %}
{%- for item in content %}
{%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
{%- if is_system_content %}
{{- raise_exception('System message cannot contain images.') }}

user > use the deep search tool to search for the latest stock market news, and output a summary of the current events going on in the stock market

Multi-line mode enabled. Press Meta+Enter to submit, Enter for new line.

[runChat] Using model glm/glm-air-4.5

The user wants me to use the deep search tool to search for the latest stock market news and provide a summary of current events. Let me analyze what parameters I need for the websearch_deepSearch function:

Required parameters:

  • query: "latest stock market news"
  • searchCount: Since this is for news, I should set this to 0 or a low number
@mdierolf
mdierolf / test.md
Last active November 25, 2025 01:42

user > use the deep search tool to search for the top 25 us stock market events, and output a summary of the current events going on in the stock market

[runChat] Using model gpt-5-mini Calling tool websearch/deepSearch [websearch/deepSearch] Deep searching: current US stock market events top stories today (search: 25, news: 25, fetch: 10) I ran a deep search across Reuters, CNBC, WSJ, Yahoo Finance, NYSE and other market outlets. Below is a concise, up-to-date summary of the major US stock-market events and themes currently moving markets.

Quick market snapshot (from Reuters/WSJ/CNBC snapshots)

  • S&P 500: +~1.55% (rallying)
  • Nasdaq: +~2.7% (big tech-led surge)
  • Dow: +~0.4%

user > use the deep seearch tool to research the latest 5 articles about quantum computing

[runChat] Using model gpt-5-mini Calling tool websearch/deepSearch

[websearch/deepSearch] Deep searching: quantum computing latest articles 2025 November (search: 10, news: 0, fetch: 5)

I ran a deep web search and fetched the top recent articles about quantum computing. Here are the latest five (with short summaries, dates, and links):

  1. These Are the 2 Biggest Hurdles for the Quantum Computing Industry Right Now — The Motley Fool
  • Date: Nov 24, 2025
llama-server \
-hf bartowski/cerebras_GLM-4.5-Air-REAP-82B-A12B-GGUF:Q6_K \
-a glm/glm-air-4.5 \
--chat-template-file glm-chat-template.jinja \
--jinja \
--host 0.0.0.0 \
--port 11434 \
--ctx-size 145000 \
--no-mmap \
-fa on \
@mdierolf
mdierolf / gist:1f3c43f48f6fbd9b2497e9f538ae42e7
Created November 5, 2025 04:57
GLM 4.6 Air Chat Template with working tool calling
{%- macro visible_text(content) -%}
{%- if content is string -%}
{{- content }}
{%- elif content is iterable and content is not mapping -%}
{%- for item in content -%}
{%- if item is mapping and item.type == 'text' -%}
{{- item.text }}
{%- elif item is string -%}
{{- item }}
{%- endif -%}
INFO 09-22 10:52:06 [__init__.py:216] Automatically detected platform cuda.
(APIServer pid=1200988) INFO 09-22 10:52:07 [api_server.py:1801] vLLM API server version 0.10.2rc3.dev236+g38db529f6
(APIServer pid=1200988) INFO 09-22 10:52:07 [utils.py:328] non-default args: {'model_tag': 'Qwen/Qwen3-Next-80B-A3B-Instruct-FP8', 'port': 11434, 'enable_auto_tool_choice': True, 'tool_call_parser': 'hermes', 'model': 'Qwen/Qwen3-Next-80B-A3B-Instruct-FP8', 'trust_remote_code': True, 'max_model_len': 262144, 'gpu_memory_utilization': 0.92}
(APIServer pid=1200988) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=1200988) INFO 09-22 10:52:27 [__init__.py:710] Resolved architecture: Qwen3NextForCausalLM
(APIServer pid=1200988) `torch_dtype` is deprecated! Use `dtype` instead!
(APIServer pid=1200988) INFO 09-22 10:52:27 [__init__.py:1769] Using max model len 262144
(APIServer pid=1200988) INFO 09-22 10:52:31 [scheduler.py:222] Chunked prefill is enabled w
@mdierolf
mdierolf / spotify.preload.c
Last active January 20, 2022 14:32
An LD_PRELOAD fix for a really stupid spotify problem
// Save this file somewhere as spotify.preload.c
// Then build the library:
// gcc -fPIC -shared -o ~/spotify.preload.so spotify.preload.c -ldl
// When you execute spotify, you need to preload this library, which wraps the "setsockopt" function call with another function that gives spotify the results it expects:
// LD_PRELOAD=~/spotify.preload.so spotify
// To make the desktop icon work, edit: /usr/share/applications/spotify.desktop
// Change Exec=... to: