# Install on GKE

Install Agent Relay on an existing Google Kubernetes Engine cluster in 16 numbered steps, each with a check before you move on.

Rendered page: https://agentrelay.com/docs/self-hosting/install-gke
Markdown endpoint: https://agentrelay.com/docs/self-hosting/install-gke.md

---

Run the steps in order. Copy each block, replace every `<...>`, and check the
**Done when** line before you move on. Expect about an hour and a half, most of
it waiting on Google Cloud: the HTTPS certificate alone can take up to an hour.

**Release:** `agent-relay-2026.11.16` (infra commit `ece71b9d`, chart 0.7.54).
The downloadable `values.yaml` renders against this release, which was not
installed on GKE. Read [Known limits](#known-limits) before a production rollout.

**You need:**

- Owner in the Google Cloud project (or the roles listed under
  [Who runs the install](https://agentrelay.com/docs/self-hosting/bring-your-own-gcp#who-runs-the-install)),
  and `kubectl` admin on the GKE cluster;
- whoever manages DNS for your domain (step 6);
- the person who will be the first admin (step 15).

**The cluster:** GKE Standard with **Workload Identity** on, the **HTTP load
balancing** add-on on (the default), in a VPC with **Private Services Access**
configured: Cloud SQL and Memorystore get private IPs from it. For nodes, plan
on at least **3 × `e2-standard-4` with 100 GB disks**. Small nodes with 20 GB
disks run out of ephemeral storage while pulling the images.

**No cluster or project yet?** See [Starting from an empty project](#starting-from-an-empty-project)
at the end, then come back to step 1. Creating a project needs
`resourcemanager.projects.create` on the organization (or folder) and
`billing.user` on the billing account; many organizations grant those to only a
few people, so ask early.

**Network:**

- Steps 1, 4, 5 and 6 need only Google Cloud access, not `kubectl`. Start them
  while cluster access is being sorted out.
- Nothing connects to the database from your machine. Every database step runs
  as a short-lived Job inside the cluster, so Cloud SQL needs no public IP.

## 1. Get the files

```sh
mkdir agent-relay-install && cd agent-relay-install
base=https://agentrelay.com/self-hosting/files/gke
curl -fsSLO "$base/SHA256SUMS"
for f in $(awk '{print $2}' SHA256SUMS); do curl -fsSLO "$base/$f"; done
shasum -a 256 -c SHA256SUMS
git clone https://github.com/AgentWorkforce/infra.git
git -C infra checkout agent-relay-2026.11.16
test "$(git -C infra rev-parse HEAD)" = ece71b9df93df64cb63676d2a0ca2767eccc6599 && echo "release ok"
mkdir release && gh release download agent-relay-2026.11.16 -R AgentWorkforce/infra -D release
(cd release && shasum -a 256 -c SHA256SUMS && tar -xzf agent-relay-0.7.54.tgz) | tail -1
```

No `gh`? Download the same assets from the
[release page](https://github.com/AgentWorkforce/infra/releases/tag/agent-relay-2026.11.16)
into `release/`.

Tools: `gcloud` (with `gke-gcloud-auth-plugin`), `terraform` 1.11 or later (or
OpenTofu), `helm` 3, `kubectl`, `jq`, `yq`, `openssl`, `python3`, and
[`crane`](https://github.com/google/go-containerregistry/releases) for step 5.
Cloud Shell has everything except `crane` and a new enough Terraform.

**Helm 3, not Helm 4.** The chart needs Helm 3.10 or later. Helm 4 is
not supported yet. If `helm version` prints `v4`, install Helm 3 beside
it (for example as `helm3`) and use it for steps 10, 11 and 13.

**Done when:** every `shasum` line in the install folder ends in `OK`,
`release ok` prints, and the last line is `release-notes.md: OK`.

## 2. Connect to Google Cloud and the cluster

```sh
gcloud auth login && gcloud auth application-default login
gcloud config set project <project id>
gcloud services enable container.googleapis.com sqladmin.googleapis.com servicenetworking.googleapis.com \
  redis.googleapis.com pubsub.googleapis.com secretmanager.googleapis.com dns.googleapis.com \
  artifactregistry.googleapis.com compute.googleapis.com iamcredentials.googleapis.com
gcloud container clusters get-credentials <cluster> --location <region>   # add --internal-ip for a private endpoint
kubectl auth can-i '*' '*' --all-namespaces
gcloud container clusters describe <cluster> --location <region> \
  --format='value(workloadIdentityConfig.workloadPool,addonsConfig.httpLoadBalancing.disabled)'
gcloud container clusters describe <cluster> --location <region> \
  --format='value(networkConfig.datapathProvider,networkPolicy.enabled)'
gcloud container node-pools list --cluster <cluster> --location <region> \
  --format='table(name,config.workloadMetadataConfig.mode)'
gcloud compute addresses list --global --filter='purpose=VPC_PEERING' --format='value(name,network)'
kubectl get storageclass
```

**Done when:**

- `can-i` prints `yes`;
- the `describe` line prints `<project id>.svc.id.goog` and nothing after it
  (HTTP load balancing is not disabled);
- every node pool shows `GKE_METADATA`;
- the `addresses` line lists a Private Services Access range on your cluster's
  VPC;
- one StorageClass is marked `(default)`;
- the second `describe` line prints `ADVANCED_DATAPATH` (Dataplane V2), or it
  ends in `True` (the NetworkPolicy add-on) and
  `kubectl -n kube-system get daemonset calico-node` shows as many ready pods
  as the cluster has nodes. The add-on enforces policies only on nodes
  recreated after it was turned on.

**Network policies must be enforced.** The chart's NetworkPolicies, and
relaygate's rule that only the gate may reach Relaycast and Sessions, have no
effect on a cluster that doesn't enforce them: GKE accepts the objects and
ignores them. Dataplane V2 can only be chosen when a cluster is created. On an
existing cluster without it, turn on the add-on, then enforcement. If the
cluster already runs other workloads, check their policies first
(`kubectl get networkpolicy -A`): policies that were ignored start to apply.

```sh
gcloud container clusters update <cluster> --location <region> --update-addons=NetworkPolicy=ENABLED
gcloud container clusters update <cluster> --location <region> --enable-network-policy
```

The second command recreates every node pool's nodes, so pods restart. Each
pool follows its own upgrade settings (`maxSurge`, `maxUnavailable`), which
decide how many nodes restart at once. Check them first, and run the command
before you install or in a maintenance window.

**Private control plane?** Use the cluster's DNS endpoint
(`get-credentials --dns-endpoint`), or run the install from a machine inside the
VPC with `--internal-ip`.

**`billing account ... disabled` from Cloud Storage or Service Networking?**
Your Application Default Credentials bill API calls to a quota project that
has no billing. Point them at the install project:
`gcloud auth application-default set-quota-project <project id>`, or export
`GOOGLE_CLOUD_QUOTA_PROJECT=<project id>` and
`CLOUDSDK_BILLING_QUOTA_PROJECT=<project id>`.

## 3. Using your own Cloud SQL, bucket or other resources? (skip if Terraform creates them)

Follow [Bring your own Google Cloud resources](https://agentrelay.com/docs/self-hosting/bring-your-own-gcp)
instead of step 4, then continue at step 5.

## 4. Create the Google Cloud resources (Terraform, about 5 minutes)

This creates a regional Cloud SQL for PostgreSQL 17 instance (private IP only,
Google-managed shared CA, admin password in Secret Manager), the sessions
bucket, a Standard HA Memorystore for Redis with TLS, a Pub/Sub topic and
subscription with a dead-letter queue, a private DNS zone that resolves the
Cloud SQL certificate name inside your VPC, and the Workload Identity grants.
Every grant is on one resource, never on the project.

```sh
cd infra/terraform/examples/gcp
cat > terraform.tfvars <<'EOF'
project_id           = "<project id>"
region               = "<cluster region, e.g. us-central1>"
name                 = "agent-relay"
namespace            = "agent-relay"
release_name         = "agent-relay"
network_self_link    = "projects/<project id>/global/networks/<cluster VPC>"
sessions_bucket_name = "<globally unique bucket name, e.g. acme-agent-relay-sessions>"
deletion_protection  = true
image_mirror_repository = "agent-relay-mirror"   # step 5's registry, readable by the GKE nodes
# desktop_feed_bucket_name = "<globally unique, e.g. acme-agent-relay-desktop-feed>"   # optional: leave it out for now (see below)
# gke_node_service_account = "<email>"   # only if the node pool has its own service account
# kms_key_name = "projects/<p>/locations/<region>/keyRings/<ring>/cryptoKeys/<key>"   # optional CMEK
labels = { app = "agent-relay" }
EOF
grep -n '<' terraform.tfvars
terraform init
terraform plan -out tfplan
terraform apply tfplan
terraform output -json chart_values > ../../../../managed-values.json
cd -
```

**Done when:** the `grep` prints nothing, the plan shows `29 to add, 0 to
change, 0 to destroy` (5 more with `desktop_feed_bucket_name`, 2 fewer without
`image_mirror_repository`), the apply completes, and `managed-values.json`
exists.

**`desktop_feed_bucket_name` (optional)** gives your branded desktop app a
signed update feed in your project: a private bucket, and a Google service
account that relay-control uses through Workload Identity to write releases
there and sign download links. Creating that service account needs Service
Account Admin. The feed also needs a signing key, so **leave this line
commented out during the install** and add it later with the key, in
[Signed desktop update feed](#signed-desktop-update-feed-optional).

The bucket puts the feed's storage settings into `managed-values.json`, and
relay-control won't start with storage but no signing key. It exits with
`SELFHOST_DESKTOP_FEED_SIGNER and SELFHOST_DESKTOP_FEED_STORAGE must be set together`,
and the pod restarts again and again. Already hit it? See
[Troubleshooting](https://agentrelay.com/docs/self-hosting/troubleshooting#relay-control-exits-signer-and-storage-must-be-set-together).
If your `values.yaml` sets `relay-control.extraEnv`, copy the `env` entries
from `terraform output desktop_feed` into it once the feed is on: Helm
replaces a list instead of merging it.

**Where the Terraform state lives.** The example has no `backend` block, so
`terraform init` keeps the state in a local file. To keep it in Cloud Storage
instead, create a private, versioned bucket and add a backend before
`terraform init`:

```sh
gcloud storage buckets create gs://<project id>-tfstate --location <region> \
  --uniform-bucket-level-access --public-access-prevention
gcloud storage buckets update gs://<project id>-tfstate --versioning
printf 'terraform {\n  backend "gcs" {}\n}\n' > backend_override.tf
terraform init -backend-config="bucket=<project id>-tfstate" -backend-config="prefix=agent-relay"
```

Either way, keep the state: upgrades and teardown need it.

- **`Error 403 ... servicenetworking`** or **`no private services access
  connection`:** the VPC has no Private Services Access. Your network team
  configures it on the cluster's VPC; then apply again.
- **CMEK:** first grant the Cloud SQL, Cloud Storage and Pub/Sub service agents
  `roles/cloudkms.cryptoKeyEncrypterDecrypter` on the key (see
  [Bring your own Google Cloud resources](https://agentrelay.com/docs/self-hosting/bring-your-own-gcp#customer-managed-keys)).

## 5. Mirror the images to Artifact Registry

Copy the release images, by digest, into the registry step 4 created. Run this
from **Cloud Shell or a VM in Google Cloud**: there it takes about a minute,
from a laptop it can take half an hour.

```sh
REGION=<region>
export MIRROR_ROOT=$(terraform -chdir=infra/terraform/examples/gcp output -raw image_mirror_root)
gcloud auth print-access-token | crane auth login $REGION-docker.pkg.dev -u oauth2accesstoken --password-stdin
for ref in $(jq -r '.images[].ref' release/release-manifest.json); do
  crane copy "$ref" "$MIRROR_ROOT/${ref%@*}"
  [ "$(crane digest "$MIRROR_ROOT/$ref")" = "${ref#*@}" ] && echo "ok ${ref%@*}"
done
sed "s|__IMAGE_MIRROR_ROOT__|$MIRROR_ROOT|g" release/release-images.values.yaml > release-images.values.yaml
grep -c "$MIRROR_ROOT" release-images.values.yaml
```

The GKE **node service account** pulls these images. Step 4 granted it
`roles/artifactregistry.reader` on this repository only: the project's default
Compute Engine service account, or `gke_node_service_account`. Check which
account your node pool uses:

```sh
gcloud container node-pools list --cluster <cluster> --location $REGION --format='value(name,config.serviceAccount)'
```

`default` means the default Compute Engine service account. If it names another
account, set `gke_node_service_account` to it and apply step 4 again. In a new
project nothing else grants that read, and every pull fails with `403` (step
8's Job is the first pod that pulls).

**Done when:** `MIRROR_ROOT` is not empty, eight `ok` lines, and the `grep -c`
line prints a number above 10.

**Using a registry Terraform doesn't manage?** Leave `image_mirror_repository`
out of step 4, create the repository yourself, set
`MIRROR_ROOT=$REGION-docker.pkg.dev/<project id>/<repository>`, and grant the
node service account `roles/artifactregistry.reader` on it.

The release has eight images. A GKE install runs five of them (control API,
dashboard, Relaycast, the sessions services and the PostgreSQL client); the
other three are for other profiles, and mirroring them is harmless.

## 6. Reserve the IP address and point DNS at it (DNS owner)

The HTTPS certificate is a Google-managed certificate. Google issues it only
after all four names resolve to the load balancer's address, so create the
records now:

```sh
gcloud compute addresses create agent-relay-ingress --global --ip-version IPV4
gcloud compute addresses describe agent-relay-ingress --global --format='value(address)'
```

Create **A records** for `<domain>`, `api.<domain>`, `cast.<domain>` and
`history.<domain>`, all pointing at that address.

**Done when:** `dig +short api.<domain>` prints the address, for all four names.

**CAA records on your domain?** They must allow `pki.goog` and `letsencrypt.org`.

## 7. Create the settings file

```sh
cat > settings.sh <<'EOF'
export DOMAIN=<app hostname, e.g. ar.example.com>
export ORG_ID=<stable id, e.g. org_acme: never change it>
export ORG_NAME='<organization name>'
export ADMIN_EMAIL=<first admin's email>
export GCP_PROJECT_ID=<project id>
export REGION=<region>
export MIRROR_ROOT=<region>-docker.pkg.dev/<project id>/agent-relay-mirror
export NAMESPACE=agent-relay
export HELM_RELEASE=agent-relay
export STATIC_IP_NAME=agent-relay-ingress
export SERVICES_IMAGE=$MIRROR_ROOT/ghcr.io/agentworkforce/sessions-pipeline@sha256:fd2efd61dca4761dfb55256aa4797cb7f61c3434cc8af4833287e0e7bb7b59a9
export PSQL_IMAGE=$MIRROR_ROOT/ghcr.io/cloudnative-pg/postgresql@sha256:ed6923f9ba2399cf9070379a353c710c29ced8a294be73986a2c3dae7e599ce3
export INSTALL_DIR="$PWD"   # run "source settings.sh" from the agent-relay-install folder
export CHART="$INSTALL_DIR/release/agent-relay"
TF="$INSTALL_DIR/infra/terraform/examples/gcp"
PG="$(terraform -chdir="$TF" output -json postgres)"
export CLOUD_SQL_INSTANCE="$(jq -r '.connection_name | split(":")[2]' <<<"$PG")"
export CLOUD_SQL_HOST="$(jq -r .dns_name <<<"$PG")"
export CLOUD_SQL_ADMIN_USER="$(jq -r .admin_username <<<"$PG")"
export CLOUD_SQL_ADMIN_SECRET_ID="$(jq -r .admin_password_secret_id <<<"$PG")"
export CLOUD_SQL_CA_SHA256="$(jq -r '.["sessions-pipeline"].worker.postgresCa.sha256' "$INSTALL_DIR/managed-values.json")"
export SESSIONS_BUCKET="$(jq -r '.["sessions-pipeline"].worker.locker.bucket' "$INSTALL_DIR/managed-values.json")"
unset PG TF
EOF
source settings.sh
bash -c 'source ./settings.sh; for v in DOMAIN ORG_ID ORG_NAME ADMIN_EMAIL GCP_PROJECT_ID REGION MIRROR_ROOT CLOUD_SQL_INSTANCE CLOUD_SQL_HOST CLOUD_SQL_ADMIN_USER CLOUD_SQL_ADMIN_SECRET_ID CLOUD_SQL_CA_SHA256 SESSIONS_BUCKET; do [ -n "${!v}" ] && [ "${!v}" != null ] && [[ "${!v}" != *"<"* ]] || echo "MISSING: $v"; done; echo "settings checked"'
```

**`SERVICES_IMAGE`, `PSQL_IMAGE` and `agent-relay-0.7.54.tgz` belong to
`agent-relay-2026.11.16`** (chart 0.7.54), the release this guide installs.
Installing another release? Don't reuse these digests: the Jobs in steps 8, 13
and 16 would run an older services image than the rest of the install. Read
the three values from that release's own files in `release/` (step 1, with its
tag):

```sh
ls release/agent-relay-*.tgz
jq -r '.images[] | select(.name == "sessions-pipeline" or .name == "sessions-migration-postgresql") | "\(.name) \(.ref)"' release/release-manifest.json
```

`SERVICES_IMAGE` is `$MIRROR_ROOT/` followed by the `sessions-pipeline` ref,
and `PSQL_IMAGE` is `$MIRROR_ROOT/` followed by the
`sessions-migration-postgresql` ref. The `.tgz` name replaces
`agent-relay-0.7.54.tgz` in steps 11 and 13. The infra checkout also records
the services image for each release, in
`infra/releases/<release tag>/pins.json` (`.pins["sessions-pipeline"]`).

**Brought your own resources (step 3)?** Replace the six lines between `PG=` and
`unset` with the values from
[Bring your own Google Cloud resources](https://agentrelay.com/docs/self-hosting/bring-your-own-gcp#then-the-settings-file).

**Done when:** only `settings checked` prints.

**In every new terminal,** first run `cd agent-relay-install && source settings.sh`.

## 8. Set up the databases

```sh
bash cloud-sql-setup.sh pre
```

It creates the namespace, the `cloud-sql-ca` ConfigMap, the three databases and
their roles (in a Job inside the cluster), and writes every database credential
to `~/.agent-relay-install` (mode 0700). It reads the Cloud SQL admin password
from Secret Manager and never prints it.

**Done when:** it ends with `pre done`, after `tls: true` and
`extensions: btree_gin,pg_trgm,plpgsql,vector`.

Copy `~/.agent-relay-install` to your secret store: it holds the database
passwords. Never commit it. With Secret Manager, for example:

```sh
tar -C ~ -czf - .agent-relay-install | gcloud secrets create agent-relay-install-files \
  --project $GCP_PROJECT_ID --replication-policy automatic --labels app=agent-relay --data-file=-
```

Later copies add a version to the same secret: replace
`secrets create ... --replication-policy automatic --labels app=agent-relay`
with `secrets versions add`.

- **It waits about 10 minutes, then says the Job did not succeed:** look at the
  Job's pod (`kubectl -n $NAMESPACE describe pod -l job-name=agent-relay-cloud-sql-pre`).
  `ImagePullBackOff` or `403 Forbidden` means the node service account can't
  read the mirror: grant it as in step 5, then run `pre` again. Nothing was
  written to Cloud SQL, and the script recreates its Job.
- **`differs from CLOUD_SQL_CA_SHA256`:** `managed-values.json` is from an older
  apply. Rerun the last Terraform command of step 4, then `source settings.sh`.
- **The Job is evicted (`low on resource: ephemeral-storage`):** the nodes'
  disks are too small (see "The cluster" above).
- Run `pre` **once**, before step 11. After step 13 it refuses to run again, by
  design.

## 9. Fill in values.yaml

```sh
sed -i.orig -e "s|__NAMESPACE__|$NAMESPACE|g" -e "s|__ORG_ID__|$ORG_ID|g" \
  -e "s|__ORG_NAME__|$ORG_NAME|g" -e "s|__ADMIN_EMAIL__|$ADMIN_EMAIL|g" -e "s|__DOMAIN__|$DOMAIN|g" \
  -e "s|__REGION__|$REGION|g" -e "s|__SESSIONS_BUCKET__|$SESSIONS_BUCKET|g" \
  -e "s|__STATIC_IP_NAME__|$STATIC_IP_NAME|g" values.yaml
grep -n '__[A-Z_]*__' values.yaml
```

**Done when:** the only line printed is `__LLM_MODEL__`. You set that one later,
when you [turn on titles and summaries](https://agentrelay.com/docs/self-hosting/after-install).

**Optional: co-branding.** The product stays Agent Relay; your logo goes beside
it. The sign-in page shows your square mark under the Agent Relay lockup, and
the dashboard shows one wide image with both logos. Put both PNGs in a
ConfigMap before step 11:

```sh
kubectl -n $NAMESPACE create configmap relay-brand \
  --from-file=mark.png=<your square mark>.png --from-file=cobrand.png=<Agent Relay + your logo>.png
```

Then uncomment the three "Optional co-branding" blocks in `values.yaml`
(`global.branding`, `relay-control.login`, `relay-webapp.customerProfile`) and
fill in their placeholders. For a branded desktop app,
`customerProfile.desktop` takes that build's link scheme and site.
`$CHART/values-branding-example.yaml` explains each value.

## 10. Create the secrets

This generates the remaining keys and creates the five Kubernetes Secrets.
Nothing is printed.

```sh
S=~/.agent-relay-install/secret-files
( umask 077
for k in selfhost-bootstrap-token selfhost-setup-key relaycast-internal-secret \
         relaycast-workspace-bootstrap-secret relaycontrol-introspect-secret session-read-api-secret \
         selfhost-encryption-key; do
  [ -s "$S/agent-relay-secrets/$k" ] || openssl rand -hex 32 | tr -d '\n' > "$S/agent-relay-secrets/$k"
done
[ -s "$S/agent-relay-secrets/SUMMARY_MASTER_KEYS" ] || printf 'k1:%s' "$(openssl rand -hex 32)" > "$S/agent-relay-secrets/SUMMARY_MASTER_KEYS"
cp "$S/agent-relay-secrets/relaycontrol-introspect-secret" "$S/agent-relay-ingest-edge-runtime/introspect-secret" )
put_secret() { # create the Secret, or replace it in place; never kubectl apply, which copies
               # every value into a last-applied annotation (replace also drops an old one)
  if kubectl -n $NAMESPACE get secret "$1" >/dev/null 2>&1; then
    kubectl -n $NAMESPACE create secret generic "$@" --dry-run=client -o yaml | kubectl replace -f -
  else
    kubectl -n $NAMESPACE create secret generic "$@"
  fi
}
for name in agent-relay-secrets agent-relay-read-api agent-relay-ingest-edge-runtime \
            search-indexer-runtime search-indexer-owner; do
  put_secret "$name" --from-file="$S/$name"
done
```

**Done when:** five lines end in `created`. Copy `~/.agent-relay-install` to
your secret store again (step 8 shows how): it now holds the encryption keys.

**Keep your secrets in SOPS (for example with Cloud KMS) or Google Secret
Manager?** Use one of the options in [Secrets](https://agentrelay.com/docs/self-hosting/secrets)
instead of this block. vals works with Google Secret Manager.

## 11. Install with Helm

```sh
helm upgrade --install $HELM_RELEASE "$INSTALL_DIR/release/agent-relay-0.7.54.tgz" --namespace $NAMESPACE \
  -f "$CHART/values-gcp.yaml" -f "$CHART/values-gcp-native.yaml" \
  -f "$INSTALL_DIR/managed-values.json" -f "$INSTALL_DIR/values.yaml" \
  -f "$INSTALL_DIR/release-images.values.yaml" --timeout 25m
```

The order matters: the two GCP profiles, then Terraform's managed services,
then your site values, then the release images **last**. Without
`release-images.values.yaml` you would get the previous release's images.

**Done when:** it prints `STATUS: deployed`. Don't add `--wait`.

## 12. Wait for the database migrations

```sh
for j in $(kubectl -n $NAMESPACE get jobs -o name | grep -E 'relay-control-migrate|relaycast-migrate|agent-relay-worker-migrate'); do
  kubectl -n $NAMESPACE wait --for=condition=complete "$j" --timeout=20m
done
```

**Done when:** three lines say `condition met` (about 2 minutes).

**Expected at this point:** the read API, ingest edge, search indexer and the
`search-indexer-bootstrap` Job fail. Step 13 fixes them.

## 13. Switch on the service logins

```sh
bash cloud-sql-setup.sh post
kubectl -n $NAMESPACE rollout restart deploy/agent-relay-read-api deploy/agent-relay-ingest-edge deploy/agent-relay-search-indexer
for j in $(kubectl -n $NAMESPACE get jobs -o name | grep search-indexer-bootstrap); do kubectl -n $NAMESPACE delete "$j"; done
helm upgrade --install $HELM_RELEASE "$INSTALL_DIR/release/agent-relay-0.7.54.tgz" --namespace $NAMESPACE \
  -f "$CHART/values-gcp.yaml" -f "$CHART/values-gcp-native.yaml" \
  -f "$INSTALL_DIR/managed-values.json" -f "$INSTALL_DIR/values.yaml" \
  -f "$INSTALL_DIR/release-images.values.yaml" --timeout 25m
kubectl -n $NAMESPACE wait --for=condition=complete "$(kubectl -n $NAMESPACE get jobs -o name | grep search-indexer-bootstrap)" --timeout=10m
kubectl -n $NAMESPACE rollout status deploy --timeout=10m
kubectl -n $NAMESPACE get pods
```

**Done when:**

- `post done` prints after three `login ok ... tls=true` lines;
- the bootstrap Job shows `condition met`;
- every pod is `Running` and ready (completed Jobs show `Completed`).

**`PostgresTlsConfigError` in the read API, ingest edge or indexer logs:** the
`cloud-sql-ca` ConfigMap doesn't match the pin in `managed-values.json`.
Compare `kubectl -n $NAMESPACE get cm cloud-sql-ca -o jsonpath='{.data.ca\.crt}' | sha256sum`
with `echo $CLOUD_SQL_CA_SHA256`.

## 14. Wait for the certificate and check (up to 60 minutes)

```sh
kubectl -n $NAMESPACE get managedcertificate agent-relay -o jsonpath='{.status.certificateStatus}{"\n"}'
```

Repeat until it prints `Active` (what is normal while you wait is below).
Then:

```sh
curl -fsS "https://api.$DOMAIN/readyz" && echo " api ok"
curl -fsS "https://cast.$DOMAIN/health" && echo " cast ok"
curl -fsS -o /dev/null -w "dashboard %{http_code}\n" "https://$DOMAIN/"
curl -s -o /dev/null -w "history %{http_code}\n" "https://history.$DOMAIN/v1/sessions"
```

**Done when:** `Active`, then `api ok`, `cast ok`, `dashboard 200` and
`history 401`.

**The Ingress can take 15 to 20 minutes to get its address** after step 11.
The certificate can't leave `Provisioning` before that. To look:

```sh
kubectl -n $NAMESPACE get ingress
kubectl -n $NAMESPACE describe ingress | sed -n '/^Events:/,$p'
```

A `Translation failed ... no BackendConfig for service port exists` warning
on the first sync is harmless if it doesn't repeat. The load balancer
controller read the Services before their BackendConfigs, and a later sync
fixes it by itself. Worry only if the `ADDRESS` column is still empty, or the
warning still repeats, after about 20 minutes; then send us the events.

- **Still `Provisioning` after an hour:** one of the four names doesn't resolve
  to the address from step 6, or a CAA record blocks Google. Fix DNS; the
  certificate retries by itself.
- **`FailedNotVisible`:** the same: DNS doesn't point at the load balancer yet.
- **502 for a few minutes after `Active`:** the load balancer's health checks
  are still converging. Wait 5 minutes.
- **502 on `/internal` and other closed paths is not an outage.** The chart
  sends paths that must not be public (for example
  `https://api.<domain>/internal/...`) to the `...-cast-edge-deny` Service,
  which has no endpoints on purpose. Without a Cloud Armor policy the load
  balancer answers those paths with 502, not 403 or 404. Only the four checks
  above tell you whether the install is up.

## 15. First sign-in, then add users

The first admin does this **before signing in any other way**, including from
the desktop app:

```sh
kubectl -n $NAMESPACE get secret agent-relay-secrets -o jsonpath='{.data.selfhost-setup-key}' | base64 -d; echo
```

1. Open `https://<domain>/login` and choose **First-time setup: enter setup
   key**.
2. Paste the key and choose a password.

**No `kubectl` from the admin's machine?** With a private control plane, the
first admin often can't reach the cluster. Whoever runs the install does the
same first sign-in through the API instead, without the key or the password
ever being shown:

```sh
bash first-admin.sh "$DOMAIN" "$GCP_PROJECT_ID" "$NAMESPACE"
```

It stores a generated password in Secret Manager as
`agent-relay-first-admin-password`, then posts it with the setup key, and
prints `first admin set`. The admin then reads the password with
`gcloud secrets versions access latest --secret agent-relay-first-admin-password --project <project id>`
and signs in at `https://<domain>/login`. The setup key works once, within 24
hours: the script never retries, and it stops if the secret already exists. A
`403 csrf_rejected` means the request lacked your dashboard's origin; it is
refused before the key is read, so the key stays unused.

Two settings, both optional:

- **The API isn't at `api.<domain>`** (you chose your own host names): name it
  with `API_HOST=<api hostname> bash first-admin.sh ...`. Give only the host
  name, without `https://` or a path. The script calls that host and still
  sends `<domain>` as the origin.
- **No Secret Manager** (the API is off, or you may not create secrets there):
  `PASSWORD_FILE=<path> bash first-admin.sh ...` keeps the password in that
  new local file (mode 0600) instead. `gcloud` isn't needed then, and the
  project ID isn't used: you can pass `-`. The script stops if the file (or a
  link with that name) already exists. Hand the file to the admin over a
  channel you trust, then delete it. Without `PASSWORD_FILE`, the script needs
  `secretmanager.googleapis.com` and permission to create a secret in the
  project.

Then add users:

```sh
bash add-user.sh "$DOMAIN"
```

It prints a temporary password for each user; they choose their own at first
sign-in.

**Done when:** the admin is signed in to the dashboard.

## 16. Turn on agent search (once, after first sign-in)

Searching Sessions for `@<agent name>` answers "The session agent index isn't
ready yet" (`501 agent_filter_not_ready` from the API) until this one-time
activation runs. It checks that every session's agent names are indexed, then
marks the index ready; later sessions are indexed as they arrive. Run it once
the first session has uploaded, or on a brand-new install with
`--fresh-install` (below).

**16a. Get the script and plan (read-only).** It runs in a pod in the
cluster as the migration owner: the URL comes from `HOOPDB_MIGRATION_URL` in
`agent-relay-secrets` and never reaches your shell. The script isn't in the
services image, so this fetches it from the commit the image was built from:

```sh
REV="$(crane config "ghcr.io/agentworkforce/sessions-pipeline@${SERVICES_IMAGE#*@}" | jq -r '.config.Labels["org.opencontainers.image.revision"]')"
curl -fsSL -o catalog-activation.ts "https://raw.githubusercontent.com/AgentWorkforce/relayhistory-pipeline/$REV/scripts/native-agent-catalog-activation.ts"
kubectl -n $NAMESPACE create configmap agent-catalog-activation --from-file=catalog-activation.ts --dry-run=client -o yaml | kubectl apply -f -
catalog() {  # catalog --plan | catalog --apply <fingerprint>
  [ -n "$NAMESPACE" ] && [ -n "$SERVICES_IMAGE" ] || { echo "missing settings: run 'source settings.sh' (step 7) first"; return 1; }
  ARGS="$(printf ', "%s"' "$@")"
  kubectl -n $NAMESPACE delete pod agent-catalog --ignore-not-found
  kubectl -n $NAMESPACE apply -f - <<YAML
apiVersion: v1
kind: Pod
metadata: { name: agent-catalog }
spec:
  restartPolicy: Never
  automountServiceAccountToken: false
  securityContext: { runAsNonRoot: true, runAsUser: 1000, runAsGroup: 1000, fsGroup: 1000, seccompProfile: { type: RuntimeDefault } }
  volumes:
    - { name: tmp, emptyDir: { sizeLimit: 64Mi } }
    - { name: ca, configMap: { name: cloud-sql-ca } }
    - { name: script, configMap: { name: agent-catalog-activation } }
  containers:
    - name: catalog
      image: $SERVICES_IMAGE
      workingDir: /app
      command: [bun, /app/scripts/native-agent-catalog-activation.ts$ARGS]
      env:
        - { name: HOOPDB_DB_MODE, value: selfhosted }
        - { name: HOOPDB_URL, valueFrom: { secretKeyRef: { name: agent-relay-secrets, key: HOOPDB_MIGRATION_URL } } }
        - { name: NODE_EXTRA_CA_CERTS, value: /etc/catalog-ca/ca.crt }
        - { name: HOME, value: /tmp }
      securityContext: { allowPrivilegeEscalation: false, readOnlyRootFilesystem: true, capabilities: { drop: [ALL] } }
      volumeMounts:
        - { name: tmp, mountPath: /tmp }
        - { name: ca, mountPath: /etc/catalog-ca, readOnly: true }
        - { name: script, mountPath: /app/scripts/native-agent-catalog-activation.ts, subPath: catalog-activation.ts, readOnly: true }
YAML
  t=0
  until p="$(kubectl -n $NAMESPACE get pod agent-catalog -o jsonpath='{.status.phase}')"; [ "$p" = Succeeded ] || [ "$p" = Failed ]; do
    sleep 3; t=$((t + 3))
    [ $t -lt 300 ] || { echo "agent-catalog still $p after 5 min"; kubectl -n $NAMESPACE describe pod agent-catalog | tail -20; return 1; }
  done
  kubectl -n $NAMESPACE logs agent-catalog | tail -1
  rc="$(kubectl -n $NAMESPACE get pod agent-catalog -o jsonpath='{.status.containerStatuses[0].state.terminated.exitCode}')"
  echo "exit $rc"; [ "$rc" = 0 ]
}
catalog --plan
```

**Done when:** one JSON line, then `exit 0` or `exit 2`. The plan is read-only
and safe to run any time. Its line shows `"eligible"` and a 64-character
`fingerprint`; `"eligible":false` with only `unprojected` above 0 is fine at
this point.

**16b. Pause uploads, plan, apply, resume (about a minute).** Apply only
works on a plan made while nothing writes, so stop uploads, give the indexer a
minute to catch up, then stop it too. `pause` waits until a Deployment's pods
are gone; `resume` puts back the replica counts read here:

```sh
IE="$(kubectl -n $NAMESPACE get deploy agent-relay-ingest-edge -o jsonpath='{.spec.replicas}')"
SI="$(kubectl -n $NAMESPACE get deploy agent-relay-search-indexer -o jsonpath='{.spec.replicas}')"
pause() {
  sel="$(kubectl -n $NAMESPACE get deploy "$1" -o json | jq -r '.spec.selector.matchLabels | to_entries | map("\(.key)=\(.value)") | join(",")')"
  [ -n "$sel" ] && kubectl -n $NAMESPACE scale deploy/"$1" --replicas=0 || return 1
  until [ -z "$(kubectl -n $NAMESPACE get pods -l "$sel" -o name)" ]; do sleep 3; done
}
resume() {
  kubectl -n $NAMESPACE scale deploy/agent-relay-search-indexer --replicas=$SI
  kubectl -n $NAMESPACE scale deploy/agent-relay-ingest-edge --replicas=$IE
  kubectl -n $NAMESPACE rollout status deploy/agent-relay-search-indexer deploy/agent-relay-ingest-edge --timeout=10m
}
if [ -n "$IE" ] && [ -n "$SI" ]; then
  pause agent-relay-ingest-edge && sleep 60 && pause agent-relay-search-indexer && catalog --plan
else
  echo "couldn't read the replica counts; nothing was scaled"
fi
```

Check the plan shows `"eligible":true`, then apply that plan's fingerprint.
Writers resume and the script is removed only if the apply succeeds:

```sh
catalog --apply <fingerprint from the plan above> && resume &&
  kubectl -n $NAMESPACE delete pod/agent-catalog configmap/agent-catalog-activation
```

**Done when:** the plan shows `"eligible":true`, the apply shows
`"activated":true` and `exit 0`, both rollouts finish, and searching Sessions
for `@<agent name>` lists sessions (or none) instead of "The session agent
index isn't ready yet".

- **Brand-new install, no sessions yet:** the plan shows `"liveRows":0` and
  `"eligible":false`. Either upload a session first, or add `--fresh-install`
  as the last argument to both: `catalog --plan --fresh-install`, then
  `catalog --apply <fingerprint> --fresh-install`. Use it only on an install
  that has never had a session, deleted ones included; anywhere else the plan
  stays ineligible.
- **It briefly pauses writers.** Uploads are refused while
  `agent-relay-ingest-edge` is at 0, and the apply itself locks the session
  index (it waits at most 1 second for those locks). Expect about a minute.
  Run it at a quiet time, and before anyone deletes a session.
- **To give up or try later:** run `resume`. Uploads stay off until you do.
- **`unprojected` above 0 in the paused plan:** the indexer hadn't caught up.
  Run `kubectl -n $NAMESPACE scale deploy/agent-relay-search-indexer --replicas=$SI`,
  wait a minute, then `pause agent-relay-search-indexer && catalog --plan`.
- **`native_catalog_plan_changed`:** something wrote between plan and apply.
  Plan again and apply the new fingerprint.
- **`native_catalog_operation_failed` with `exit 1` from apply:** usually a
  writer still held the locks. Nothing was written and writers are still
  paused; run the same `catalog --apply ... && resume ...` line again.
- **`native_catalog_schema_validation_required`:** the pod couldn't reach the
  database, or the migrations (step 12) haven't finished.
- **Any other count above 0** (for example `unknownUsers`): run `resume` and
  send us the JSON line.

## Next

- **Before people start:** in the dashboard, under **Settings › Policies**, set
  what the desktop app uploads.
- **Desktop app:** [Desktop app rollout](https://agentrelay.com/docs/self-hosting/desktop-app).
- **Search:** transcript search is stored in Cloud SQL (`search.backend:
  postgres`); there is no search cluster to run.
- **Problems:** [Troubleshooting](https://agentrelay.com/docs/self-hosting/troubleshooting), then
  [Support](https://agentrelay.com/docs/self-hosting/support).

## Signed desktop update feed (optional)

With a feed, your branded desktop app offers each new version to its users and
installs it when they choose to update. Without one, a new version means
installing the new build over the old one. Agent Relay builds and publishes
your desktop app, and gives you its profile ID and the SHA-256 hash of the
token it publishes with. The token itself stays with Agent Relay.

1. **Bucket:** set `desktop_feed_bucket_name` in step 4's `terraform.tfvars`,
   then, from the install folder, apply it and refresh `managed-values.json` so
   it carries the feed's settings:

   ```sh
   (cd infra/terraform/examples/gcp && terraform plan -out tfplan && terraform apply tfplan \
     && terraform output -json chart_values > ../../../../managed-values.json)
   ```

   Don't run step 11's or step 13's `helm upgrade` again until item 3 below:
   until the signing key is in place, relay-control won't start with these
   settings.
2. **Signing key:** create an Ed25519 key for the profile and store it in the
   cluster. The private key never leaves your cluster and Secret Manager:

   ```sh
   PROFILE=<profile id from Agent Relay>; BACKUP=agent-relay-desktop-feed-key-$PROFILE; umask 077
   if out=$(gcloud secrets describe $BACKUP --format='value(name)' 2>&1); then BK=yes
   elif printf '%s' "$out" | grep -q NOT_FOUND; then BK=no
   else BK=error; fi
   if [ $BK = error ]; then
     echo "STOP: could not check for the backup $BACKUP (sign in to gcloud and run this again)"
   else
     if ! kubectl -n $NAMESPACE get secret desktop-feed-keys >/dev/null 2>&1; then
       if [ $BK = yes ]; then
         gcloud secrets versions access latest --secret $BACKUP > "$PROFILE.pem"   # restore the saved key
       else
         openssl genpkey -algorithm ed25519 -out "$PROFILE.pem"
       fi
       if openssl pkey -in "$PROFILE.pem" -noout 2>/dev/null; then
         kubectl -n $NAMESPACE create secret generic desktop-feed-keys --from-file="$PROFILE.pem"
       fi
       rm -f "$PROFILE.pem"
     fi
     kubectl -n $NAMESPACE get secret desktop-feed-keys -o jsonpath="{.data.$PROFILE\.pem}" | base64 -d > "$PROFILE.pem"
     if openssl pkey -in "$PROFILE.pem" -noout 2>/dev/null && [ $BK = no ]; then
       gcloud secrets create $BACKUP --data-file="$PROFILE.pem" && BK=yes
     fi
     if openssl pkey -in "$PROFILE.pem" -noout 2>/dev/null && [ $BK = yes ] \
       && gcloud secrets versions access latest --secret $BACKUP | cmp -s - "$PROFILE.pem"; then
       openssl pkey -in "$PROFILE.pem" -pubout -outform DER | tail -c 32 | base64
     else
       echo "STOP: the cluster key is missing, unreadable or differs from its backup"
     fi
     rm -f "$PROFILE.pem"
   fi
   ```

   This needs OpenSSL 1.1.1 or later, as in Cloud Shell. The `openssl` that
   ships with macOS (LibreSSL) has no Ed25519; use Homebrew's `openssl@3`. The
   last line prints the public key. Send it to Agent Relay, which builds it into
   the app. The key is created once. Running the block again reuses the key in
   the cluster, or restores it from the Secret Manager backup if the cluster
   lost it, so desktop builds keep trusting it. It makes a new key only when
   Secret Manager answers that no backup exists, and stores a key only after
   checking it is valid. If it prints `STOP`, check that you can read the
   backup (`gcloud secrets versions access latest --secret $BACKUP >/dev/null && echo readable`,
   which never shows the key); if both
   copies exist and differ, keep the one whose public key Agent Relay has, and
   ask Agent Relay before changing either.
3. **Values:** add this to `values.yaml`, with the hash from Agent Relay, then
   run step 11's `helm upgrade` command again:

   ```yaml
   relay-control:
     desktopFeed:
       signingKeys:
         existingSecret: desktop-feed-keys
         profiles: [<profile id>]
     extraEnv:
       - { name: SELFHOST_DESKTOP_FEED_KEY_LABEL, value: f1 }
       - { name: SELFHOST_DESKTOP_FEED_PUBLISHER_TOKEN_HASHES, value: '{"<profile id>":["<sha256 hex>"]}' }
   ```

   Keep the feed entries from `terraform output desktop_feed` in the same
   `extraEnv` list: this list replaces the one in `managed-values.json`.
4. **Register the profile:** an organization admin registers the profile and
   binds the workspaces whose users get the app, with Agent Relay. The public
   key the control API returns must match the one from step 2.

**Done when:** `kubectl -n $NAMESPACE get pods` shows relay-control ready after
the upgrade, and after Agent Relay publishes the first version to your feed,
the dashboard's **Downloads** card lists it.

## Re-stamp sessions uploaded before 2026.11.6 (upgrades only)

Releases 2026.11.1 to 2026.11.5 recorded sessions uploaded to GCS as if they
were in S3, so the read API refuses them (`501`). 2026.11.6 records new uploads
correctly; this one-off tool fixes the rows written before. A fresh install of
2026.11.6 or later doesn't need it. It changes only rows whose objects it finds in your
bucket, writes nothing in plan mode, and is safe to rerun.

Run it in a terminal where you ran `cd agent-relay-install && source settings.sh`
(step 7): it needs `NAMESPACE`, `REGION` and `SESSIONS_BUCKET` from there.

```sh
IMG="$(kubectl -n $NAMESPACE get deploy agent-relay-read-api -o jsonpath='{.spec.template.spec.containers[0].image}')"
CA_SHA="$(kubectl -n $NAMESPACE get deploy agent-relay-read-api -o jsonpath='{.spec.template.spec.containers[0].env[?(@.name=="SESSION_POSTGRES_CA_SHA256")].value}')"
restamp() {  # restamp plan | restamp apply
  [ -n "$NAMESPACE" ] && [ -n "$REGION" ] && [ -n "$SESSIONS_BUCKET" ] && [ -n "$CA_SHA" ] && [ -n "$IMG" ] \
    || { echo "missing settings: run 'source settings.sh' (step 7) first"; return 1; }
  APPLY=""; [ "$1" = apply ] && APPLY=", --apply"
  kubectl -n $NAMESPACE delete pod index-restamp --ignore-not-found
  kubectl -n $NAMESPACE apply -f - <<YAML
apiVersion: v1
kind: Pod
metadata: { name: index-restamp }
spec:
  restartPolicy: Never
  serviceAccountName: agent-relay-read-api
  automountServiceAccountToken: false
  securityContext: { runAsNonRoot: true, runAsUser: 1000, runAsGroup: 1000, fsGroup: 1000, seccompProfile: { type: RuntimeDefault } }
  volumes:
    - { name: tmp, emptyDir: { sizeLimit: 64Mi } }
    - { name: ca, configMap: { name: cloud-sql-ca } }
    - { name: url, secret: { secretName: agent-relay-secrets, defaultMode: 0400, items: [ { key: HOOPDB_MIGRATION_URL, path: url } ] } }
  containers:
    - name: restamp
      image: $IMG
      command: [bun, run, /app/src/session-pipeline/index-provider-restamp.ts$APPLY]
      env:
        - { name: LOCKER_PROVIDER, value: gcs }
        - { name: LOCKER_BUCKET, value: "$SESSIONS_BUCKET" }
        - { name: LOCKER_REGION, value: "$REGION" }
        - { name: SESSION_POSTGRES_TLS_PROFILE, value: custom-ca }
        - { name: SESSION_POSTGRES_CA_FILE, value: /etc/restamp-ca/ca.crt }
        - { name: SESSION_POSTGRES_CA_SHA256, value: "$CA_SHA" }
        - { name: INDEX_PROVIDER_RESTAMP_DATABASE_URL_FILE, value: /run/restamp/url }
        - { name: HOME, value: /tmp }
      securityContext: { allowPrivilegeEscalation: false, readOnlyRootFilesystem: true, capabilities: { drop: [ALL] } }
      volumeMounts:
        - { name: tmp, mountPath: /tmp }
        - { name: ca, mountPath: /etc/restamp-ca, readOnly: true }
        - { name: url, mountPath: /run/restamp, readOnly: true }
YAML
  t=0
  until p="$(kubectl -n $NAMESPACE get pod index-restamp -o jsonpath='{.status.phase}')"; [ "$p" = Succeeded ] || [ "$p" = Failed ]; do
    sleep 5; t=$((t + 5))
    [ $t -lt 600 ] || { echo "index-restamp still $p after 10 min"; kubectl -n $NAMESPACE describe pod index-restamp | tail -20; return 1; }
  done
  kubectl -n $NAMESPACE logs index-restamp | tail -1
  kubectl -n $NAMESPACE get pod index-restamp -o jsonpath='exit {.status.containerStatuses[0].state.terminated.exitCode}{"\n"}'
}
restamp plan
```

Check the plan's JSON line, and keep a copy of it: `restamp apply` replaces the
pod, so its output is gone afterwards. `scanned` is the rows still recorded as
S3, `proven` those whose objects are in your bucket (they will be fixed) and
`unproven` those left alone. Then apply it:

```sh
restamp apply
kubectl -n $NAMESPACE delete pod index-restamp
```

**Done when:** the last JSON line shows `"complete":true` with `restamped`
equal to the `proven` in the plan line you kept, and `exit 0`.

- **`missing settings`:** this terminal hasn't run `source settings.sh`.
- **`still Pending after 10 min`:** the `describe` output shows why (usually an
  image pull from your mirror). Fix it, then run the same command again.

- **`exit 3`:** incomplete (a page limit, or a row refused mid-run). Run
  `restamp apply` again.
- **`exit 5`:** done, but `guard_skipped` rows were kept by a database guard
  (an erase fence, or a publication still in progress). Rerunning won't change
  them; send us the JSON line.
- **`exit 2`:** misconfigured (for example `LOCKER_BUCKET` is empty). Nothing
  was read or written.
- **`exit 4`:** any other error; check `kubectl -n $NAMESPACE logs index-restamp`
  and run it again.

## Known limits

What release `agent-relay-2026.11.16` doesn't do on GKE yet:

1. **Upgrading from 2026.11.5 or earlier: re-stamp once.** Those releases
   recorded sessions uploaded to Cloud Storage as if they were in S3, so their
   reads answer `501`. After the upgrade, run
   [Re-stamp sessions uploaded before 2026.11.6](#re-stamp-sessions-uploaded-before-2026116-upgrades-only)
   once. A fresh install doesn't need it.
2. **Long sessions need the longer read deadline.** The read API reads a
   session's chunks one after another (about 30 ms each from Cloud Storage), so
   the default 3 s deadline fails sessions of more than about 100 chunks
   (`501 turns_canonical_deadline`). The downloadable `values.yaml` sets
   `READ_API_CANONICAL_DEADLINE_MS=15000`; keep it. Parallel reads come in a
   later release.
3. **`/turns` and `/metadata` are empty on self-host.** They serve Agent Relay
   conversation-turn records, which only the hosted service produces, so
   `/turns` answers with no turns and `/metadata` with `404`. `/events` and the
   dashboard carry the whole transcript.
4. **No automated install or upgrade.** The install is the manual sequence
   above, and upgrades are manual Helm upgrades with a Cloud SQL backup first.
5. **Search runs in PostgreSQL, not OpenSearch.** Fine for a first install, but
   not sized for search at large scale.
6. **Memorystore without AUTH.** The services can't send Redis credentials, so
   the instance uses TLS with AUTH off. Keep it reachable only from the
   cluster's VPC.
7. **Not highly available.** The control API, dashboard, Relaycast and search
   indexer each run one replica.
8. **Not covered by this guide:** GKE Autopilot, Shared VPC, and titles and
   summaries with a model.

## Starting from an empty project

The steps above assume a GKE cluster in a VPC with Private Services Access.
If you have neither, this creates a dedicated project with a minimal shape that
meets "The cluster" at the top: private nodes, Cloud NAT for their outbound
traffic, and a public control plane open only to the addresses you list. Change
the ranges if they overlap your other networks.

```sh
P=<project id>; R=<region, e.g. us-central1>
gcloud projects create $P --organization <organization id> --labels app=agent-relay
gcloud billing projects link $P --billing-account <billing account id>
gcloud config set project $P
gcloud services enable container.googleapis.com compute.googleapis.com servicenetworking.googleapis.com \
  cloudresourcemanager.googleapis.com iam.googleapis.com storage.googleapis.com
gcloud compute networks create agent-relay-vpc --subnet-mode custom
gcloud compute networks subnets create agent-relay-subnet --network agent-relay-vpc --region $R \
  --range 10.10.0.0/20 --secondary-range pods=10.20.0.0/16,services=10.30.0.0/20 \
  --enable-private-ip-google-access
gcloud compute addresses create agent-relay-psa --global --purpose VPC_PEERING \
  --prefix-length 16 --network agent-relay-vpc --addresses 10.40.0.0
gcloud services vpc-peerings connect --service servicenetworking.googleapis.com \
  --network agent-relay-vpc --ranges agent-relay-psa
gcloud compute routers create agent-relay-router --network agent-relay-vpc --region $R
gcloud compute routers nats create agent-relay-nat --router agent-relay-router --region $R \
  --auto-allocate-nat-external-ips --nat-all-subnet-ip-ranges
gcloud container clusters create agent-relay --location $R --num-nodes 1 \
  --machine-type e2-standard-4 --disk-type pd-balanced --disk-size 100 \
  --network agent-relay-vpc --subnetwork agent-relay-subnet --enable-ip-alias \
  --cluster-secondary-range-name pods --services-secondary-range-name services \
  --workload-pool=$P.svc.id.goog --workload-metadata=GKE_METADATA \
  --addons HttpLoadBalancing,GcePersistentDiskCsiDriver --release-channel regular \
  --enable-private-nodes --master-ipv4-cidr 172.16.0.0/28 \
  --enable-master-authorized-networks --master-authorized-networks <your IP>/32 \
  --enable-shielded-nodes --enable-dataplane-v2
```

A regional cluster with `--num-nodes 1` gets one node in each of the region's
three zones. Then continue at step 1 with `<cluster>` = `agent-relay` and
`<cluster VPC>` = `agent-relay-vpc`, and grant the node service account in step 5.
Run everything that needs `kubectl` from an address in the authorized list.

**Done when:** step 2's checks pass on the new cluster.
