switchspan

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

switchspan

#!/usr/bin/env bash

# This script generates a list of repos in a GitHub org.
# The list can be pasted directly to a Microsoft Excel sheet.
# You will need to use your GitHub username in the username field.
# Update the page=N number to get the complete list of repos.
curl --silent --user "username" "https://api.github.com/orgs/vmware/repos?page=1&per_page=100" | npx jq '.[].html_url' | while read repo
do
    echo "=Hyperlink("$repo","$repo")"
done

switchspan

AWS CLI Cheatsheet

http://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html https://www.youtube.com/watch?v=_wiGpBQGCjU

Setup

switchspan

kubernetes shortcuts & cheatsheet

1. Learning Docker Basics

2. Logging

Use a DaemonSet for Pods that need to run one per machine, because they provide a machine-specific system service.

Learning k8s basics

For example, to mark a node unschedulable, run this command:

switchspan

var amqp = require('amqplib/callback_api');

// if the connection is closed or fails to be established at all, we will reconnect
var amqpConn = null;
function start() {
  amqp.connect(process.env.CLOUDAMQP_URL + "?heartbeat=60", function(err, conn) {
    if (err) {
      console.error("[AMQP]", err.message);
      return setTimeout(start, 1000);
    }

switchspan

'use strict';

const driveUploadPath = 'https://www.googleapis.com/upload/drive/v3/files';

// 'id' is driveId - unique file id on google drive
// 'version' is driveVersion - version of file on google drive
// 'name' name of the file on google drive
// 'appProperties' keep the custom `ifid` field
const fileFields = 'id,version,name,appProperties';

switchspan

on El Capitan, after installing the brew...

$ brew update
$ brew tap caskroom/cask
$ brew install Caskroom/cask/java

And Java 8 will be installed at /Library/Java/JavaVirtualMachines/jdk1.8.xxx.jdk/

Check version:

switchspan

Build docker image

$ cd /path/to/Dockerfile
$ sudo docker build .

View running processes

switchspan

PSQL

Magic words:

psql -U postgres

Some interesting flags (to see all, use -h):

• -E: will describe the underlaying queries of the \ commands (cool for learning!)
• -l: psql will list all databases and then exit (useful if the user you connect with doesn't has a default database, like at AWS RDS)

switchspan

# dump all databases once every 24 hours
45 4 * * * root nice -n 19 su - postgres -c "pg_dumpall --clean" | gzip -9 > /var/local/backup/postgres/postgres_all.sql.gz

# vacuum all databases every night (full vacuum on Sunday night, lazy vacuum every other night)
45 3 * * 0 root nice -n 19 su - postgres -c "vacuumdb --all --full --analyze"
45 3 * * 1-6 root nice -n 19 su - postgres -c "vacuumdb --all --analyze --quiet"

# re-index all databases once a week
0 3 * * 0 root nice -n 19 su - postgres -c 'psql -t -c "select datname from pg_database order by datname;" | xargs -n 1 -I"{}" -- psql -U postgres {} -c "reindex database {};"'