Shoaib Burq sabman

KYC 2.0: which Datalake-served views answer which questions (short)

Audience: SDI and partners. Basis: Persona discovery and the five city platform module surfaces (Situational analysis, Planning, Monitoring, KYC.TV, cross-city dashboard), using the platform module dataset family grouping for KYC 2.0.

Datalake holds versioned city data. Datalake-served apps are the city platform and cross-city product surfaces that read from that lake (maps, dashboards, story views, exports). Field teams use their own capture tools; those tools are data sources that feed ingest paths into the lake.

Pipeline

flowchart LR

Hybrid persistence — touch points (Sandy × Overeasy)

This note only maps where layers meet. Code lives in the linked repos.

Think of a running agent as three things that can be saved separately:

Layer	Plain English	What it saves	Survives kill/reboot?	Who provides it today
Disk	The agent’s files under `/workspace` (scripts, artifacts, `context_N`, the dill file on disk)	Bytes on a filesystem	Yes, if mounted via an Overeasy-like session	Overeasy (`oe mount` / session id) + Sandy paths that already use `/workspace`
Mind	The agent’s live Python state (variables, imports, REPL locals)	Process / dill namespace	Only while the sandbox stays warm (Tier A), or after a future VM checkpoint (Tier B)	Sandy dill in `agentd/repl_runner.py`; keep-alive via `rlm_env.py` + `reaper.py`

Sandy vs CubeSandbox

Side quest comparison (2026-07). CubeSandbox = Tencent open-source E2B-class microVM platform. Sandy = Decision Labs RLM-native sandbox runtime.

Sources: TencentCloud/CubeSandbox, Sandy repo + open PRs.

Persistence / keep-alive work lives on feature branches (not main yet):

Branch: feature/persistence-tier-a — PR #21
Follow-ups on top: feature/persistence-tier-a-followups — PR #22 (targets the branch above)

Atelier Sénégal - ordre du jour simple (lun. 20 juil.)

Horaire : 14:00-16:30 Dakar
OpenTalk : https://meet.opentalk.eu/room/60fb1399-1fb0-49dd-8af5-45b19acaa730?invite=dda1a71a-762c-4df7-93b3-6d1b170a1886
Questionnaire : https://kyc20-data-lineage-production.up.railway.app/

Ordre du jour

Senegal workshop — simple agenda (Mon 20 Jul)

When: 14:00-16:30 Dakar
Where: OpenTalk (link in invite)
Survey: https://kyc20-data-lineage-production.up.railway.app/

Agenda

Exercise: roles, personas, and one user story each

For: Personas & use-cases sessions (Track A/B, and the usages part of the Senegal French session)
Time: about 20-30 minutes in the room
Goal: Agree who the platform is for, then write one clear user story per role so Release 1 stays concrete.

What you will do (two steps)

File Format	Indices (within file)	Index Types	Sharding	Analysis Library DB Interfaces (Examples)	Performance	Granularity	Compression	Data Types	Durability	Security	Community/Support	Maturity	Cost/License	Basin Use-cases
Nimble	Columns and streams	Block encoding, cascading (recursive/composite) encoding, pluggable encoding selection policies	Supported	Flatbuffers, SIMD, GPU	Designed for wide workloads, extensibility APIs	Thousands to tens of thousands of columns and streams	Flatbuffers, block encoding, recursive/composite encoding	Many, with extensibility for additional encodings	In development, no stability/versioning guarantees yet	Focus on a single unified library to prevent fragmentation	Work in progress, community support through Meta	Active development, no stable release yet	Open-source, dependenc

	<!DOCTYPE html>
	<html>
	<head>
	<title>Leaflet with D3 popups</title>
	<meta charset="utf-8" />
	<meta name="viewport" content="width=device-width, initial-scale=1.0">

	<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/4.13.0/d3.min.js"></script>
	<link rel="stylesheet" href="https://unpkg.com/leaflet@1.3.1/dist/leaflet.css" integrity="sha512-Rksm5RenBEKSKFjgI3a41vrjkw4EVPlJ3+OiI65vTjIdo9brlAacEuKOiQ5OFh7cOI1bkDwLqdLw3Zg0cRJAAQ==" crossorigin=""/>
	<script src="https://unpkg.com/leaflet@1.3.1/dist/leaflet.js" integrity="sha512-/Nsx9X4HebavoBvEBuyp3I7od5tA0UzAxs+j83KgC8PU0kgB4XiK4Lfe4y4cgBtaRJQEIFCW+oC506aPT2L1zw==" crossorigin=""></script>