Zheng Zeng zheng022

September 29

Customer	Bundle ID	Hostname	Path
RBC	207967	rbcgithub-fg-rbc-com-primary	/data/multi-disk/db
RBC	207868	rbcgithub-fg-rbc-com-replica-babffafcfc	/data/multi-disk/db
SAP SE	207830	prod-tools-frontend-sofia-84a78	/data/multi-disk/db
BMW AG	207821	cc-github-bmwgroup-net-replica-fcebfafffe	/data/multi-disk/db
BMW AG	207821	cc-github-bmwgroup-net-replica-fcebfafffe	/data/multi-disk/git
SAP SE	207647	prod-tools-frontend-athens-7e663	/data/multi-disk/db

MySQL multi-disk during GHES install and upgrade

Summary

Customers should be able to place GHES MySQL data on multi-disk storage at the two moments when the product can make the safest storage decision:

First install: the first-run wizard can guide the admin to attach and confirm an eligible MySQL multi-disk device before initial config-apply initializes MySQL.
Upgrade: existing standalone customers and HA can opt in to moving MySQL to newly attached multi-disk storage during that node's planned upgrade window.

The product outcome is simple: customers get a clearer, safer way to separate database storage from the primary data disk without needing to discover and run a manual storage procedure after the instance is already in service.

Validate Muti-disk setup for BMW

Setup Multi-disk

Create stand-alone ghebooted instance

.gheboot 3.18.10

func (a *UpgradeGatewayAPIV1) runCluster() http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		clientNodes, _ := connector.AgentClients(r.Context())
		
		// filter out cluster delegate and non-delegate nodes
		var clusterDelegateNode *catalog.Agent
		var nonDelegateNodes []*catalog.Agent
		for _, node := range clientNodes {

customer instance

https://github.com/github/ghes/issues/16434
bundle: https://esbtools-staff.githubapp.com/bundles/191214 (no increase on resource)
69,336 active series with a total of 211.8 million datapoints across the ~48-hour window
⚠️ observability.metrics.prometheus-endpoint-enabled true

Factor	Customer
Active series	69,336

🚀 Improving the GHES Manage Developer Experience — What We Did and What's Changed

You've told us this was hard!!
Building new APIs in GHES Manage has too often felt like a challenging expedition: confusing paths, hidden "gotchas," and the occasional moment where you're just… banging your head against the desk wondering why something that should be straightforward is taking so long.

GHES Manage was created three years ago as a modern alternative to the legacy Enterprise Management Console, but adoption hasn't met our expectations. We've heard consistent feedback that implementing new API endpoints can be challenging and unintuitive.

So we've been focused on improving the GHES Manage API.

The following snippet helps to copy script from enterprise2 to a ghebooted instance:

#!/bin/bash
#/ Usage: ./copy.sh <ghe-boot-host-name>
TARGET_HOST="${1}"
TARGET_USER="admin"
TARGET_PORT="122"

set -e

What Operators do

Operator manage and configure instance with UI
Operator builds automation with API
Operator does fire fighting with ssh + bash script

What can the purpose of an API/CLI

Part of the UI architecture and support a robust UI solution
Help operators build an reliable and efficient automation

Availability Review Retrospect

We initiated a systematic availability review process following our July 2024 offsite (see Revival of the GHES Availability Review Process). The first availability issue was then created on August 16th, marking almost a year since our previous review.

Our journey began by exploring what availability truly means for GHES. We recognized that an escalation's value extends beyond mere resolution - we aimed to foster deeper discussions, prevent recurrence through measured repair items, and share knowledge via comprehensive runbooks.

Over the past 6 months, we've made significant strides in our availability review processes:

Created 29 availability issues
Generated 29 repair items, with 22 successfully resolved
Conducted 9 availability review meetings

	#!/bin/bash
	#/ Usage: ghe-storage-verify-mysql-migration <backup-path> <destination-path>
	#/ Verifies that two MySQL datadir trees are identical by comparing md5sums
	#/ of every file beneath each path.
	#/
	#/ Example (after ghe-storage-migrate-mysql):
	#/ ghe-storage-verify-mysql-migration /data/user/mysql-backup/github_enterprise /data/multi-disk/db/mysql/github_enterprise

	set -e