Troubleshooting¶

Reference for diagnosing a KeycloakInstance that is stuck, failing, or not behaving as expected. Start with State inspection, then match your symptom in the sections below.

State inspection — start here¶

NS=my-keycloak
NAME=my-keycloak

# 1. Phase and print columns
kubectl get kci $NAME -n $NS

# 2. Full status including conditions
kubectl describe kci $NAME -n $NS

# 3. Conditions as JSON (often the most direct answer)
kubectl get kci $NAME -n $NS \
  -o jsonpath='{.status.conditions}' | jq

# 4. StatefulSet rollout status
kubectl get statefulset $NAME -n $NS
kubectl rollout status statefulset/$NAME -n $NS

# 5. Keycloak pods
kubectl get pods -n $NS
kubectl describe pod -n $NS <stuck-pod-name>

# 6. Managed Postgres (if postgres.managed: true)
kubectl get perconapgcluster ${NAME}-pg -n $NS
kubectl describe perconapgcluster ${NAME}-pg -n $NS

Where are the logs?¶

Component	Command
Operator	`kubectl logs -n bnerd-keycloak-system deploy/bnerd-keycloak-operator --tail=200`
Keycloak pod	`kubectl logs -n $NS $NAME-0 --tail=200`
Percona PG operator	`kubectl logs -n pgo deploy/percona-postgresql-operator --tail=200`

Phase: Pending¶

Symptom: kubectl get kci shows Pending for more than a few minutes.

Causes:

KeycloakVersionMap/default not found, or the version/alias not (yet) present in it — condition reason VersionResolutionPending:
```
kubectl get kcvm default
# Error from server (NotFound): ...
kubectl apply -f examples/versionmap-default.yaml
```
This is transient and requeues every 30 seconds — no manual reconcile trigger needed once the map exists.
BYO credentialsSecret not found yet — condition reason WaitingForCredentials:
```
kubectl describe kci $NAME -n $NS | grep -i "waiting for"
```
Create the missing Secret; the operator progresses on the next reconcile (30-second requeue).

Phase: Provisioning¶

Symptom: Stuck in Provisioning for more than 10–15 minutes.

Managed Postgres not becoming ready¶

Condition reason PostgresProvisioning.

kubectl describe perconapgcluster ${NAME}-pg -n $NS
kubectl get pods -n $NS | grep pg

Common causes:

No default StorageClass — PVCs stay Pending. Set a default StorageClass.
Percona PG Operator not installed or not running: kubectl get pods -n pgo.
Insufficient cluster resources (CPU/memory). Check pod events.

BYO credentials Secret invalid¶

Condition reason CredentialsSecretInvalid — the Secret exists but is missing a required key (host, dbname, user, or password):

kubectl get secret <credentialsSecret-name> -n $NS -o yaml

This is phase Failed but bounded-retried (~60s requeue): correcting the Secret recovers without editing the CR.

Phase: Failed¶

Symptom: phase: Failed with a condition reason.

`ValidationFailed`¶

The spec failed basic validation:

kubectl get kci $NAME -n $NS \
  -o jsonpath='{.status.conditions[?(@.type=="Validated")]}' | jq .message

Common reasons: missing spec.hosts.host, invalid spec.postgres.topology, negative or BYO-only spec.postgres.nodes, or an invalid provider name.

`PerconaCRDMissing`¶

spec.postgres.managed: true but the pgv2.percona.com CRD is not installed. The operator never falls back to BYO or an in-cluster database:

kubectl get crd perconapgclusters.pgv2.percona.com

Install the Percona PG Operator (see Installation) and change any spec field (or re-apply) to trigger a reconcile.

`VersionDowngradeBlocked`¶

A spec.version change resolved to an image older than the running one. The workload is left untouched. See Upgrades § No downgrades for the break-glass override.

`ServiceMonitorCRDMissing`¶

Not a Failed phase — a warning condition (MetricsExporterReady=False). spec.metrics.serviceMonitor: true was set but the monitoring.coreos.com ServiceMonitor CRD is absent; the instance still deploys without metrics scraping.

kubectl get crd servicemonitors.monitoring.coreos.com

`GatewayAPIUnavailable`¶

Symptom: phase: Failed, Validated=False, reason GatewayAPIUnavailable.

spec.ingress.mode: GatewayAPI was set but the cluster does not serve HTTPRoute at gateway.networking.k8s.io/v1. This fails loudly on purpose: a silent stall here would leave a CR that looks fine and a hostname that never resolves.

# Is the bundle installed at all, and which versions does it SERVE?
kubectl get crd httproutes.gateway.networking.k8s.io \
  -o jsonpath='{range .spec.versions[*]}{.name}={.served}{"\n"}{end}'

The message distinguishes two situations — read it before hunting:

"HTTPRoute is not served at gateway.networking.k8s.io/v1…" — the bundle is missing or too old.
"Gateway API IS installed, but it was installed after the operator started…" — the CRDs are present and the operator simply has not been restarted. One command, below.

Two distinct causes:

Nothing installed — install Gateway API (bundle v1.0.0 or newer, standard channel). Most Gateway implementations ship and own their own CRD bundle; prefer that.
Too old — the CRD exists but serves only v1beta1. The operator renders and reads v1 only, so an old bundle counts as unavailable rather than half-working.

After installing the CRDs, restart the operator. The condition itself clears on its own within ~5 minutes (the capability probe is re-checked), but the HTTPRoute watch and informer cache are fixed at startup, so route status changes will not wake a reconcile until you restart:

kubectl rollout restart deployment/<release>-controller-manager -n bnerd-keycloak-system

`RoutesAccepted` is not True¶

Only in GatewayAPI mode. The instance can be perfectly Ready while this is False or Unknown — Ready is not gated on it, because the workload being healthy and the path to it being wired are different questions.

kubectl get kci <name> -o jsonpath='{range .status.conditions[?(@.type=="RoutesAccepted")]}{.status} {.reason}: {.message}{"\n"}{end}'
kubectl get httproute -n <ns> -o yaml    # the controller's own verdict

Status / reason	Cause	Fix
`False` / `NotAllowedByListeners`	The Gateway's listener does not admit this namespace.	Set `allowedRoutes.namespaces` on the listener to admit it.
`False` / `NoMatchingListenerHostname`	No listener hostname matches `hosts.host` / `hosts.adminHost`.	Widen the listener hostname, or pin the right listener with `gateway.parentRef.sectionName`.
`False` / `BackendNotFound`	Route accepted, backend Service missing.	Usually transient on first reconcile; otherwise check the instance's Services.
`Unknown` / `RoutesPending`, message `no Gateway controller has reported on parentRef <ns>/<name>`	Nobody has adjudicated that attachment — the Gateway does not exist, or no installed controller owns its `GatewayClass`. This is the shape a typo'd `parentRef` takes, and it persists rather than resolving.	`kubectl get gateway -A`; fix the name/namespace in `spec.ingress.gateway`.
`Unknown` / `RoutesPending`, generic message	The controller has not written status yet.	Wait; re-checked every 30s.

See the Serving guide for the full mode reference.

StatefulSet not converging / Ready never reported¶

Symptom: phase stays Deploying/Progressing (reason WorkloadNotReady) even though pods look Running.

Ready is gated on the rollout fully converging: updatedReplicas >= spec.replicas, updateRevision == currentRevision, and observedGeneration caught up. A slow or stuck rolling update (e.g. a readiness probe failing on the new revision) keeps the instance in Progressing indefinitely.

kubectl get statefulset $NAME -n $NS -o jsonpath='{.status}' | jq
kubectl describe pod ${NAME}-0 -n $NS   # check readiness/liveness probe events

Common causes:

A provider image failed to start (see the numeric non-root user requirement in the KeycloakInstance Guide § Providers).
Insufficient CPU/memory for spec.resources on available nodes.
A misconfigured spec.env override breaking Keycloak startup.

Getting more detail¶

Operator logs for a specific instance¶

kubectl logs -n bnerd-keycloak-system deploy/bnerd-keycloak-operator --tail=500 \
  | grep $NAME

Full condition dump¶

kubectl get kci $NAME -n $NS -o json \
  | jq '.status | {phase, host, observedVersion, observedGeneration, conditions}'

Events¶

kubectl get events -n $NS --field-selector involvedObject.name=$NAME \
  --sort-by='.lastTimestamp'