Skip to content

Instantly share code, notes, and snippets.

View wasade's full-sized avatar

Daniel McDonald wasade

View GitHub Profile
@wasade
wasade / thp_madvfree_check.c
Last active November 2, 2026 17:35
Reproducer: x86 Linux data loss with MADV_FREE + transparent huge pages (pmd_modify dirty bit, fixed by f7491d7c81db / CVE-2026-97945)
// thp_madvfree_check: does this x86-64 kernel lose data written to MADV_FREE'd transparent huge pages?
//
// Linux commit f7491d7c81db ("x86/mm: Fix user-space data loss with MADV_FREE and THP", CVE-2026-97945):
// pmd_modify() drops the hardware dirty bit of a PMD-mapped huge page. A huge page that was lazily freed
// (MADV_FREE) and then written again is therefore discarded by reclaim, and later reads return zeros.
// mprotect() and NUMA-balancing hinting faults both go through pmd_modify().
//
// thp_madvfree_check <MiB> <trigger> <thp> <madv> [pressure_file ...]
// trigger: mprotect = mprotect(PROT_READ) then mprotect(PROT_READ|PROT_WRITE) after the rewrite
// numa = only read the buffer for $NUMA_SECONDS (default 90), leaving it to NUMA hinting
#!/bin/bash
#SBATCH -t 24:00:00
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --array 1-97
cd /home/mcdonadt/agp_demux_barcode_assessment-8.4.21
source activate qiime2-2020.11
export TMPDIR=/panfs/dtmcdonald/
python processor.py demux.details $SLURM_ARRAY_TASK_ID
@wasade
wasade / summarize-q2-demux-stats.py
Last active August 9, 2021 14:58
summarize q2-demux stats
import qiime2
import pandas as pd
import glob
from collections import defaultdict
import sys
import time, os
details = [l.strip() for l in open(sys.argv[1])]
f = details[int(sys.argv[2]) - 1]
@wasade
wasade / q2-2017.2.0.mvp.sh
Last active September 9, 2017 15:29
Run the MVP tutorial for q2 2017.2.0 using q2-deblur
#!/bin/bash
set -e
set -x
envname=q2deblur_mvp_test
executiondir=mvp
redeploy=$1
source deactivate
@wasade
wasade / sparcc.py
Created July 28, 2016 00:15
Get SparCC correlations and a network
# python 2
# conda install biom-format -c bioconda
# conda install matplotlib # pysurvey setup.py is incomplete
# pip install -e "hg+https://bitbucket.org/yonatanf/pysurvey#egg=pysurvey"
# note: the pysurvey build in pypi does not work. unsure why.
import click
# this little gem is because pysurvey depends on an older version of scipy
# and attempts to import nanmedian from scipy.stats. This is a dirty hack.
@wasade
wasade / snappy_unifrac.py
Last active August 15, 2016 17:37
Block decomposition method for unifrac + rapid subsetting of biom direct from disk + joblib + balanced parentheses tree
import numpy as np
import bp
import biom
import skbio
import sys
import joblib
import h5py
class biom_counts_wrapper(object):
def __init__(self, table_f):
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@wasade
wasade / agmissing.ipynb
Created April 13, 2016 20:09
AG samples not in "clean" metadata
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@wasade
wasade / joblib_poc.ipynb
Last active March 18, 2016 22:16
Example of using joblib to run systemcalls in parallel complete with ipy magics
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.