Skip to content

Instantly share code, notes, and snippets.

View Steboss's full-sized avatar

Steboss Steboss

View GitHub Profile
@Steboss
Steboss / mask_form_ablation.py
Created October 6, 2026 13:52
compares the original dense boolean mask with key_value_seq_lengths using the real in-model Q/K/V and pair-bias shapes
"""
The trunk grid attention passes tokamax a `(N,1,1,N)` boolean key mask built as
the outer product of the token mask (evoformer.py:115). Because the token mask
is a strict prefix (features.py pads with zeros at the end), that boolean mask
carries no more information than one integer per row:
k_end[b] = L if token b is real, else 0 (L = number of real tokens)
tokamax accepts exactly that form as `key_value_seq_lengths`, which becomes
`Mask(k_end=...)` and turns the kernel's per-k-tile mask load + `jnp.where` into
@Steboss
Steboss / consistency.py
Created October 6, 2026 13:45
Consistency script for comparing pre-post optimization results
"""An optimization is valid ONLY if AlphaFold3 still predicts
the same structure and confidence after the change.
Usage
-----
1. Capture the baseline reference ONCE, from the unmodified inference path:
python consistency.py capture --af3-output-dir <baseline_out> \
--ref out/reference.json
2. After each optimization, compare the candidate's output against it:
python consistency.py check --af3-output-dir <candidate_out> \
@Steboss
Steboss / consistency.py
Created October 6, 2026 13:41
consistency file to compare pre-post optimization results
"""An optimization is valid ONLY if AlphaFold3 still predicts
the same structure and confidence after the change.
Usage
-----
1. Capture the baseline reference ONCE, from the unmodified inference path:
python consistency.py capture --af3-output-dir <baseline_out> \
--ref out/reference.json
2. After each optimization, compare the candidate's output against it:
python consistency.py check --af3-output-dir <candidate_out> \
@Steboss
Steboss / consistency.py
Created October 6, 2026 13:41
consistency file to compare pre-post optimization results
"""An optimization is valid ONLY if AlphaFold3 still predicts
the same structure and confidence after the change.
Usage
-----
1. Capture the baseline reference ONCE, from the unmodified inference path:
python consistency.py capture --af3-output-dir <baseline_out> \
--ref out/reference.json
2. After each optimization, compare the candidate's output against it:
python consistency.py check --af3-output-dir <candidate_out> \
@Steboss
Steboss / q4km_vs_q40.csv
Created July 29, 2026 10:47
Compare Q4_K_M tensors vs Q4_0
tensor Q4_0 (dB) Q4_K (dB) improvement
blk.0.attn_k 19.60 21.18 +1.58
blk.1.attn_k 20.44 22.34 +1.90
blk.2.attn_k 20.66 22.53 +1.87
blk.30.attn_k 20.19 22.31 +2.12
blk.31.attn_k 20.60 22.53 +1.93
blk.0.attn_q 19.95 21.37 +1.42
@Steboss
Steboss / edgeai_true_false.csv
Created July 1, 2026 10:30
what's true and false in edge AI
Claim Why isn't true Correct claim
It fits because the file fits The file is the weight payload at one precision. Peak memory also includes activations, workspace, and a KV cache that grows linearly with the context Fit is undetermined wihtout context length and runtime overhead
Parameter count equals memory usage The weight payload should be used, it depends on the precision, wegiths are resident, activations and KV come and go Peak is runtime-specific
FLOPS decide latency FLOPs ignore data movement. A decode step and a prefill step over the same weights do identical airthmetic but differ in intensity The number of tokens M is what decide everything about memory and compute-bound cases
Claim, Why isn't true, Correct claim
"It fits because the file fits", "The file is the weight payload at one precision. Peak memory also includes activations, workspace, and a KV cache that grows linearly with the context", "Fit is undetermined wihtout context length and runtime overhead"
@Steboss
Steboss / gist:90850e0d9e23822fc5073d16aae824ae
Created July 1, 2026 10:26
edgeai_true_false_claims.csv
Claim, Why isn't true, Correct claim
"It fits because the file fits", "The file is the weight payload at one precision. Peak memory also includes activations, workspace, and a KV cache that grows linearly with the context", "Fit is undetermined wihtout context length and runtime overhead"
@Steboss
Steboss / iteration_4.rs
Created January 16, 2026 10:36
Final iteration where the model starts to use a more neon-like style
use rayon::prelude::*;
use std::arch::aarch64::*;
use std::env;
use std::fs;
fn main() {
let args: Vec<String> = env::args().collect();
if args.len() != 4 {
eprintln!("Usage: {} M K N", args[0]);
return;
@Steboss
Steboss / iteration_0.rs
Created January 16, 2026 10:35
First iteration from the model. It generates a Strassen algorithm
use rayon::prelude::*;
use std::env;
const N: usize = 2; // Fixed block size
// HELPER: Convert slice to fixed array for safety (optional but good for debugging)
fn to_array(slice: &[f64]) -> [f64; N * N] {
slice.try_into().expect("Slice has incorrect length")
}