Michal Lukaszewski mlukasze

Moonshine Streaming Medium — OpenVINO Enablement Investigation Report

⚠️ AUTOMATICALLY GENERATED BY OMEGA AGENT — REQUIRES HUMAN REVIEW ⚠️ This report was created by an AI agent as part of automated model enablement. A human maintainer must review and approve it before any of its findings are acted upon. No PR was opened for this ticket — per explicit scope override, this is an investigation/report deliverable only.

Model: UsefulSensors/moonshine-streaming-medium (HF revision 57b843633a8c183cadf6699ffa761377a933a866, matching transcribe.cpp's pinned 57b8436). License: MIT (confirmed — streaming, English-only variant; no gate-check concerns).

GPU Performance Analysis: INT8 vs INT4 on Xe2 (Arc Pro B60) for Bonsai-27B / Qwen3.6-27B

Ticket: openvinotoolkit/omega#66 Reported symptom: INT8 weight-compressed inference of Qwen/Qwen3.6-27B (base model for prism-ml/Bonsai-27B-gguf) on GPU.1 (Intel Arc Pro B60, discrete, Xe2/"Battlemage") is ~75x slower (~3.35 s/token) than INT4 on the same device (~44.75 ms/token), despite both being functionally correct (WWB similarity 0.962 for INT8, 0.911 for INT4, both ≥ 0.9 threshold). CPU and GPU.0 (integrated Xe-LPG) show no such anomaly.

Bottom line up front: the dominant root cause is **GPU memory (VRAM) capacity

cohere_asr OpenVINO follow-up — bug investigation & fix (2026-07 iteration)

Related: optimum-intel PR #1788, omega issue #12, prior investigation gist.

Task

A user testing PR #1788 (adds cohere_asr / CohereLabs/cohere-transcribe-03-2026 OpenVINO export+inference support) reported two problems:

Export failure with --task automatic-speech-recognition-with-past: MatMul dimension mismatch (1024 vs 1280) in encoder_decoder_proj.
After upgrading optimum to 2.2.0.dev0, export succeeded, but inference via OVModelForSpeechSeq2Seq through the transformers ASR pipeline crashed with AttributeError: 'NoneType' object has no attribute 'shape'.

CohereLabs/cohere-transcribe-03-2026 — Working OpenVINO Inference Instructions

Companion to openvinotoolkit/omega#12 and huggingface/optimum-intel#1788.

This gist documents the verified-working way to run this model on OpenVINO today, and the known limitation that is not yet fixable without new upstream work.

TL;DR

Intriguing properties of neural networks (2013)

Explaining and harnessing adversarial examples (2014)

Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples (2016)

Towards evaluating the robustness of neural networks (2017)

Synthesizing robust adversarial examples (2018)

	{
	"model_id": "nomic-ai/nomic-embed-text-v1.5",
	"canonical_type": "genai",
	"primary_metric": "similarity",
	"runs": [
	{
	"run_id": 1,
	"timestamp": "2026-07-13T17:11:03.405866Z",
	"mode": "full",
	"triggered_by": "optimum-genai",