Skip to content

Instantly share code, notes, and snippets.

@andrewxhill
andrewxhill / research-failures.md
Created November 8, 2026 17:56
How AI-assisted scientific studies go wrong

How AI-assisted scientific studies go wrong

A catalogue for people designing, running or reviewing studies with AI agents. These failures can arise when an agent designs the experiment, builds the analysis, optimises a method or writes up the result. People make them too. An agent can carry a mistaken assumption through enough code and polished prose to make it hard to spot.

The central check is whether the claim matches what the study measured. For each headline or conclusion, write down:

Which system and setup were tested, what inputs did each method receive, what comparison and scoring rule were used, and which data influenced the design?

Then compare that sentence with the published claim. A claim that performance generalises to new data needs an evaluation independent of the choices made during development. Check the measured sentence against the saved inputs, code and results. A well-written methods section cannot establish that the code followed the method.

@andrewxhill
andrewxhill / Chain Of Trust (but verify).ipynb
Last active March 5, 2025 14:31
Chain of Trust - but verify...
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@andrewxhill
andrewxhill / ReadTablelandValue.sol
Last active November 29, 2022 00:24
Uses midpointapi.com to return a single cell value from any table or sql request on Tableland
// SPDX-License-Identifier: UNLICENSED
pragma solidity ^0.8.17;
import "@openzeppelin/contracts/utils/Strings.sol";
import "@openzeppelin/contracts/utils/structs/EnumerableMap.sol";
interface IMidpoint {
function callMidpoint(uint64 midpointId, bytes calldata _data) external returns(uint256 requestId);
}
@pooja
pooja / developing-on-filecoin.md
Last active March 26, 2025 16:15
Developing on Filecoin: Resources
@andrewxhill
andrewxhill / minimum_spanning_tree.sql
Created March 13, 2016 14:07
Minimum spanning tree in SQL... just because
CREATE TYPE minimum_spanning_tree_internal AS (
d NUMERIC[],
a INT[],
b INT[],
geoms GEOMETRY[],
ids TEXT[]
);
CREATE TYPE minimum_spanning_tree_unit AS (
d NUMERIC,
@andrewxhill
andrewxhill / cartodb-utils.py
Last active June 11, 2021 15:27
command-line python interface for manipulating data on CartoDB
import os
import urllib
import urllib2
import base64
import json
import sys
import argparse
try:
import requests
except ImportError:
@jsanz
jsanz / gist:5141420
Created March 12, 2013 09:15
From CartoDB to DXF, with all my love to CAD users
# With the GeoJSON OGR driver one can use the CartoDB SQL API to create any OGR supported format output
# OGR Formats: http://gdal.org/ogr/ogr_formats.html
# CartoDB SQL API: http://developers.cartodb.com/documentation/sql-api.html
# This sample creates a DXF making a "select the_geom from meteo" from my account at CartoDB
ogr2ogr -skipfailures -f DXF out.dxf "http://xurxosanz.cartodb.com/api/v2/sql?q=select%20the_geom%20from%20meteo&format=GeoJSON" OGRGeoJSON
@mhkeller
mhkeller / gist:3834498
Created October 4, 2012 15:45 — forked from abelsonlive/gist:3751902
google geocoding API, R
Forked from @BrianAbelson
library('rjson') # For parsing json in R
library('RCurl') # For sending an http request
library('plyr') # For ddply
# Takes an data frame with a unique id column and an address column at the very least
# it returns a data frame with the uid, lat, lng and specificity of geocoding
# use the uid to join these results back to your data if you want
# you can also accomplish roughly the same thing with a for loop instead of ddply