headroom #1099 repro: technique=preserve reports (1-N)*100% 'saved' because original_tokens counts the FIRST image and compressed_tokens sums ALL images

headroom_1099_repro.py

"""Repro for headroom #1099 — "negative savings" under technique=preserve.

Root cause (headroom 0.39.1, commit d13e196), headroom/image/compressor.py:

  line 254  _extract_image_data()   -> docstring: "Extract FIRST image data from messages"
  line 530  _apply_compression()    -> for "preserve", returns messages UNCHANGED
  line 730  original_tokens   = _estimate_tokens(<FIRST image>) + tile_saved
  line 738  compressed_tokens = _count_result_tokens(<ALL images in ALL messages>)

The two counters measure different sets, so with N images in the transcript the
log line reports (1 - N) * 100 percent "saved" while the payload is untouched.

Run:
  git clone --depth 1 https://github.com/headroomlabs-ai/headroom.git
  # shim so headroom/__init__.py (heavy otel imports) is bypassed:
  mkdir -p shim/headroom && : > shim/headroom/__init__.py
  ln -s "$PWD/headroom/headroom/image" shim/headroom/image
  uv run --no-project --python 3.12 --with pillow \
      python -c "import sys; sys.path.insert(0,'shim'); exec(open('headroom_1099_repro.py').read())"
"""
import base64, io, logging, sys
from PIL import Image

logging.basicConfig(level=logging.INFO, format="%(message)s", stream=sys.stdout)

from headroom.image.compressor import ImageCompressor
from headroom.image.image_types import Technique


def img_b64(w, h):
    b = io.BytesIO()
    Image.new("RGB", (w, h), (120, 30, 200)).save(b, format="PNG")
    return base64.b64encode(b.getvalue()).decode()


def anthropic_msgs(n, w=1024, h=512):
    """n screenshots in one message - what Claude Code resends every request."""
    content = [{"type": "text", "text": "what does this architecture diagram show?"}]
    for _ in range(n):
        content.append({"type": "image", "source": {"type": "base64",
                        "media_type": "image/png", "data": img_b64(w, h)}})
    return [{"role": "user", "content": content}]


c = ImageCompressor()

# Pin the router to PRESERVE. Without ONNX models the code already falls back to
# PRESERVE at compressor.py:733 ("Router failed, preserving image") - this just
# makes it deterministic.
class R:
    def classify(self, *_a, **_k):
        class D:
            technique = Technique.PRESERVE
            confidence = 1.0
        return D()

c._get_onnx_router = lambda: R()

print("n_images | technique | original | compressed | savings%  | bytes changed?")
print("---------+-----------+----------+------------+-----------+----------------")
for n in (1, 2, 5, 9, 20):
    msgs = anthropic_msgs(n)
    before = [i["source"]["data"] for i in msgs[0]["content"][1:]]
    out = c.compress(msgs, provider="anthropic")
    after = [i["source"]["data"] for i in out[0]["content"][1:]]
    r = c.last_result
    print(f"{n:8d} | {r.technique.value:9s} | {r.original_tokens:8d} | "
          f"{r.compressed_tokens:10d} | {r.savings_percent:8.0f}% | "
          f"{'NO (identical)' if before == after else 'YES'}")

print()
print("Control: ONE image under preserve must report exactly 0% "
      "(image_types.py:22 'Keep full quality (0% savings)')")
c.compress(anthropic_msgs(1, 900, 500), provider="anthropic")
print(f"  {c.last_result.original_tokens} -> {c.last_result.compressed_tokens} "
      f"= {c.last_result.savings_percent:.0f}%")