Brent Salisbury nerdalert

ExtProc end-to-end qualification for Grid signal routing and llm-d

Purpose

This E2E qualification proves that a real inference request follows the deployed Gateway, ExtProc, Grid, and llm-d serving path, and that a real load signal can move a subsequent request from one serving pool to another. It tests the integrated runtime, not just the configuration objects or the routing algorithm in isolation.

The central claim is specific: when one pool becomes pressured, the serving Gateway receives and uses Grid's updated routing state, and a fresh request reaches the other pool; when pressure is removed, the route recovers. A successful run must identify the request's actual backend and the routing revision the Gateway served. A changed metric, a new ConfigMap, or a passing direct backend request alone cannot establish that claim.

This is a Grid signal-routing qualification within the ExtProc repository. RHOAI and MaaS supply platform readiness and ingress/authentication context. ExternalModel, External

For an OpenAI Chat Completions client, Praxis AI supports:

Target	What Praxis AI does
Azure OpenAI	Keeps the Chat Completions body; handles Azure's `api-version` and response differences.
Vertex AI Gemini	Translates requests to `generateContent` and responses back.
AWS Bedrock Converse	Translates requests and normal or streaming responses.
Anthropic native Messages API	Not yet supported in this direction.

Separately, Praxis AI supports OpenAI Responses API -> Chat Completions. That is an API-format conversion, not a specific cloud provider.

AI Grid Policy Architecture

Multi-Cluster Inference, Active-Active Placement, Cloud Burst, Token Governance, and Closed-Loop Optimization

Status: architecture direction, grounded in the current burst-routing-v1 branches
Scope: Grid + Praxis AI + Praxis core + llm-d/EPP + self-hosted and cloud inference providers + shared token state
Primary design rule: Grid decides asynchronously; Praxis executes locally from an immutable snapshot
LoRA / multi-LoRA: deferred

Current branch checkpoints

Demo step	What to do	Where to show it in the UI
1. Open the demo	Log in with the demo credentials.	Login page
2. Show the applications	Introduce App 1, App 2, and App 3.	Application budgets
3. Show independent quotas	Point out that each app has its own allocation and usage.	Token allocation governance
4. Send normal traffic	Send a few requests through both consumers.	Live topology and request history
5. Show shared routing	Explain that East and West consumers use the same provider overlay.	Live topology
6. Show token accounting	Explain that each request reserves tokens, then reconciles actual usage.	Application budgets and quota activity
7. Show soft-cap behavior	Exceed an app’s soft allocation. The request still succeeds but is marked over allocation.	Token allocation governance
8. Create GPU pressure	Click Create GPU pressure.	GPU/provider queue pressure

OSAC/OSC and Praxis Grid Gap Summary

Based on osc-grid-gap-analysis.md.

Completed or Comparable Parity

“Parity” here means that Praxis Grid has a comparable capability for the inference-routing portion of the OSAC/OSC vision. It does not mean that the implementations or ownership models are identical.

Findings

1. High: Task type and complexity need a clearer contract

The proposal says semantic_classify produces both:

task type: code, math, creative, general
complexity: easy, hard

Installing Praxis Grid on an Existing EC2 Kubernetes Cluster

This primer is for an operator who already has a working Kubernetes cluster on Amazon EC2 and wants to install Praxis Grid using the existing-cluster Helm workflow.

It does not create EC2 instances, build a Kubernetes cluster, configure an AWS VPC, or install a CNI. It begins with an existing cluster that is reachable through kubectl.

	#!/usr/bin/env bash
	# claude launcher — route to either Vertex or the dogfood gateway.
	# Usage:
	# ./claude <route> [shortcuts...] [extra claude args...]
	#
	# Routes:
	# vertex route via Google Vertex
	# dogfood route via dogfood AI gateway (default)
	#
	# Shortcuts (expand to claude flags; combine freely, any order):


	00:00:00,000 --> 00:00:05,783
	This demonstration shows one authenticated token budget shared across distributed consumer gateways.

	2
	00:00:05,783 --> 00:00:11,798
	Alice can enter through either gateway, but both processes enforce the same model quota through Valkey.

	3
	00:00:11,798 --> 00:00:16,887

	Demo: grid-glb-demo
	Subcommand: run-grid-glb-demo
	Forge config: /home/fedora/praxxis/ai-grid/prs/44-grid-demo-image-source/demos/demos/grid-glb-demo/forge.yaml
	Demo root: /home/fedora/praxxis/ai-grid/prs/44-grid-demo-image-source/demos/demos/grid-glb-demo
	Grid repo: /home/fedora/praxxis/ai-grid/prs/44-grid-demo-image-source/grid

	Compiling xtask v0.1.3 (/home/fedora/praxxis/ai-grid/prs/44-grid-demo-image-source/grid/xtask)
	Finished `dev` profile [unoptimized + debuginfo] target(s) in 7.95s
	Compiling libc v0.2.186
	Compiling serde_core v1.0.228