This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """ | |
| The trunk grid attention passes tokamax a `(N,1,1,N)` boolean key mask built as | |
| the outer product of the token mask (evoformer.py:115). Because the token mask | |
| is a strict prefix (features.py pads with zeros at the end), that boolean mask | |
| carries no more information than one integer per row: | |
| k_end[b] = L if token b is real, else 0 (L = number of real tokens) | |
| tokamax accepts exactly that form as `key_value_seq_lengths`, which becomes | |
| `Mask(k_end=...)` and turns the kernel's per-k-tile mask load + `jnp.where` into |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """An optimization is valid ONLY if AlphaFold3 still predicts | |
| the same structure and confidence after the change. | |
| Usage | |
| ----- | |
| 1. Capture the baseline reference ONCE, from the unmodified inference path: | |
| python consistency.py capture --af3-output-dir <baseline_out> \ | |
| --ref out/reference.json | |
| 2. After each optimization, compare the candidate's output against it: | |
| python consistency.py check --af3-output-dir <candidate_out> \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """An optimization is valid ONLY if AlphaFold3 still predicts | |
| the same structure and confidence after the change. | |
| Usage | |
| ----- | |
| 1. Capture the baseline reference ONCE, from the unmodified inference path: | |
| python consistency.py capture --af3-output-dir <baseline_out> \ | |
| --ref out/reference.json | |
| 2. After each optimization, compare the candidate's output against it: | |
| python consistency.py check --af3-output-dir <candidate_out> \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """An optimization is valid ONLY if AlphaFold3 still predicts | |
| the same structure and confidence after the change. | |
| Usage | |
| ----- | |
| 1. Capture the baseline reference ONCE, from the unmodified inference path: | |
| python consistency.py capture --af3-output-dir <baseline_out> \ | |
| --ref out/reference.json | |
| 2. After each optimization, compare the candidate's output against it: | |
| python consistency.py check --af3-output-dir <candidate_out> \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| tensor | Q4_0 (dB) | Q4_K (dB) | improvement | |
|---|---|---|---|---|
| blk.0.attn_k | 19.60 | 21.18 | +1.58 | |
| blk.1.attn_k | 20.44 | 22.34 | +1.90 | |
| blk.2.attn_k | 20.66 | 22.53 | +1.87 | |
| blk.30.attn_k | 20.19 | 22.31 | +2.12 | |
| blk.31.attn_k | 20.60 | 22.53 | +1.93 | |
| blk.0.attn_q | 19.95 | 21.37 | +1.42 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Claim | Why isn't true | Correct claim | |
|---|---|---|---|
| It fits because the file fits | The file is the weight payload at one precision. Peak memory also includes activations, workspace, and a KV cache that grows linearly with the context | Fit is undetermined wihtout context length and runtime overhead | |
| Parameter count equals memory usage | The weight payload should be used, it depends on the precision, wegiths are resident, activations and KV come and go | Peak is runtime-specific | |
| FLOPS decide latency | FLOPs ignore data movement. A decode step and a prefill step over the same weights do identical airthmetic but differ in intensity | The number of tokens M is what decide everything about memory and compute-bound cases |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Claim, Why isn't true, Correct claim | |
| "It fits because the file fits", "The file is the weight payload at one precision. Peak memory also includes activations, workspace, and a KV cache that grows linearly with the context", "Fit is undetermined wihtout context length and runtime overhead" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Claim, Why isn't true, Correct claim | |
| "It fits because the file fits", "The file is the weight payload at one precision. Peak memory also includes activations, workspace, and a KV cache that grows linearly with the context", "Fit is undetermined wihtout context length and runtime overhead" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| use rayon::prelude::*; | |
| use std::arch::aarch64::*; | |
| use std::env; | |
| use std::fs; | |
| fn main() { | |
| let args: Vec<String> = env::args().collect(); | |
| if args.len() != 4 { | |
| eprintln!("Usage: {} M K N", args[0]); | |
| return; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| use rayon::prelude::*; | |
| use std::env; | |
| const N: usize = 2; // Fixed block size | |
| // HELPER: Convert slice to fixed array for safety (optional but good for debugging) | |
| fn to_array(slice: &[f64]) -> [f64; N * N] { | |
| slice.try_into().expect("Slice has incorrect length") | |
| } |
NewerOlder