# Troubleshooting

Search this page for the error text from your command output.

## Existing cert-manager or Prometheus installation

```text
Error: INSTALLATION FAILED: Unable to continue with install: CustomResourceDefinition "certificates.cert-manager.io" exists and cannot be imported into the current release
```

Your cluster already runs cert-manager, and the chart bundles its own. The quick
start requires a cluster without an existing cert-manager installation. Use a
dedicated cluster or discuss an alternative configuration with your Zuplo
solutions architect.

The same error with `monitoring.coreos.com` CRDs means that the cluster has an
existing Prometheus Operator installation.

## Missing image registry password

```text
Error: execution error at (zuplo/templates/zuplo-secrets.yaml:2:31): .Values.zuploImageRegistry.password is required.
```

Your credentials file was not passed to Helm, or was passed before the values
file and overwritten.

Pass both files, credentials last:

```bash
helm install zuplo ... -f zuplo-values.yaml -f zuplo-secrets.yaml
```

## Missing certificate email address

```text
Error: execution error at (zuplo/templates/cert-issuer.yaml:...): cert-manager.acme.email field is required if using cert-manager
```

`cert-manager.enabled` is `true` but `cert-manager.acme.email` is empty. Set an
address that you monitor. The certificate authority sends expiration warnings
there.

## Missing account name

```text
Error: execution error at (zuplo/templates/configuration.yaml:1:15): .Values.account.name is required.
```

`account.name` is missing from `zuplo-values.yaml`. Copy it exactly from
**Account Settings → General**
([portal](https://portal.zuplo.com/+/account/settings/general)). The management
API rejects API keys whose account does not match this string.

## HTTP-01 DNS lookup failure

The full string looks like this:

```
Waiting for HTTP-01 challenge propagation: failed to perform self check GET
request 'http://zuplo-admin.example.com/.well-known/acme-challenge/<token>':
Get "http://zuplo-admin.example.com/.well-known/acme-challenge/<token>":
dial tcp: lookup zuplo-admin.example.com on 10.128.0.10:53: no such host
```

Your DNS record does not exist yet, or cluster DNS cannot resolve it.
cert-manager checks the challenge URL from inside the cluster before asking the
certificate authority to validate it. A failed self-check doesn't consume the
certificate authority's rate limit.

Check it:

```bash
kubectl get challenge -A
dig +short A zuplo-admin.example.com
```

Create the missing record, pointing at the `EXTERNAL-IP` of
`zuplo-haproxy-ingress`. cert-manager retries on its own; no restart needed.

## Certificate not ready after DNS resolves

```text
READY: False
```

Check what the challenge says:

```bash
kubectl get challenge -A -o jsonpath='{range .items[*]}{.metadata.name}{"\n  "}{.status.reason}{"\n"}{end}'
```

If DNS resolves correctly, check the following possible causes:

- The certificate authority can't reach port 80. HTTP-01 validation uses plain
  HTTP on port 80. Confirm that the ingress responds from outside your network:

  ```bash
  curl -i "http://zuplo-admin.example.com/.well-known/acme-challenge/probe"
  ```

  A `404` or `302` means HAProxy is answering. A timeout means it is not.

- A proxy or external load balancer changes the HTTP-01 request or response.
  Configure it to forward `/.well-known/acme-challenge/*` without
  authentication, caching, or response modification.

If neither cause applies, share the challenge status with your Zuplo solutions
architect.

## Management API authorization failure

```json
{ "detail": "Authorization Failed" }
```

The request carried an API key, but the management API rejected it. The gateway
validates the key and then checks that its account matches `account.name` in
your values file. A valid key from another Zuplo account fails this check.

Compare what the cluster was installed with against **Account Settings →
General** ([portal](https://portal.zuplo.com/+/account/settings/general)):

```bash
kubectl get configuration default -n zuplo-system \
  -o jsonpath='{.spec.account.name}{"\n"}'
```

If they differ, correct `account.name` in `zuplo-values.yaml` and run
`helm upgrade`. If they match, the key itself may belong to a different account.
Create one under **Account Settings → API Keys**
([portal](https://portal.zuplo.com/+/account/settings/api-keys)) while the right
account is active.

`"detail": "No Authorization Header"` means no key was sent at all. Set
`ZUPLO_API_KEY` in the environment the Zuplo CLI runs in.

## Environment variables unavailable

```
We are unable to fetch the environment variables from Zuplo for this project.
To fix this, check that the project, <name> exists and this api-key has access to it.
If you want to force deployment without the environment variables, set
ZUPLO_ALLOW_DEPLOY_WITH_EMPTY_VARS to true.
```

`zuplo deploy` stops before uploading the project. At this point, the CLI is
communicating with Zuplo rather than your management API, so this error doesn't
indicate a problem with your installation.

The project name comes from the `project` field in your `zuplo.jsonc`. Confirm a
project by that name exists in your account, and that the API key you are using
belongs to that same account.

Setting `ZUPLO_ALLOW_DEPLOY_WITH_EMPTY_VARS=true` forces the deploy through. The
gateway builds and serves traffic without environment variables, so any route
that reads one fails at runtime. Use this setting only to test the cluster.

## Fallback ingress certificate

```text
CN=kubernetes-ingress-ca
```

HAProxy serves its fallback certificate when the requested hostname doesn't have
an issued certificate.

```bash
kubectl get certificate -A
```

If the certificate isn't `READY: True`, follow the DNS troubleshooting steps on
this page.

## Hostname missing from certificate

```text
SSL: no alternative certificate subject name matches target host name
```

cert-manager issues a certificate for each deployment hostname. A deployment can
report `Ready` before its certificate is available. Retry the request for up to
a minute.

If the error persists, run `kubectl get certificate -n zuplo` and follow the DNS
troubleshooting steps on this page.

## ACME challenge probe redirects

```text
HTTP/1.1 302 Found
```

You probed a hostname that already has an Ingress. Those hosts redirect port 80
to HTTPS.

This redirect doesn't affect certificate issuance. cert-manager creates a
separate challenge Ingress that continues to serve on port 80. To test the
ingress, use a hostname with no Ingress, as described in
[Verify your install](./verify.md).

## Failed to deploy the environment but the cluster shows the build succeeding

```text
Failed to deploy the environment
```

The status polled by the Zuplo CLI might not reflect the build result inside a
self-hosted cluster. The CLI can report a failure even when the build and
gateway rollout succeed.

Confirm the result in the cluster:

```bash
kubectl get jobs -n zuplo-system         # build-<name>  Complete  1/1
kubectl get deploy -n zuplo              # the gateway Deployment exists
curl https://<deployment hostname>/<a route>
```

If these checks pass, the deployment succeeded. If the build Job failed or no
Deployment appears, search this page for the reported error.

## Builder Job fails after dependency installation

```
{"severity":"INFO","message":"Using zuplo CLI for compilation"}
{"severity":"EMERGENCY","message":"Failed to run NPM commands","error":"exit status 1"}
```

The upload is missing its `.zuplo/` directory. The Zuplo CLI generates
`.zuplo/worker.ts` and `.zuplo/build.json`, which the in-cluster compile step
needs. The archive extracts and `npm install` completes before this error, so
the final log line can look like a dependency failure.

The usual cause is a `.zupignore` that excludes `.zuplo/`. When a project has a
`.zupignore`, the CLI uses it without modification. The CLI removes `.zuplo/`
from `.gitignore` rules automatically, but it doesn't modify `.zupignore`.
Remove `.zuplo/` from `.zupignore`, and then deploy again.

Read the complete Job log. The compiler error appears before the `EMERGENCY`
line:

```bash
kubectl logs -n zuplo-system -l job-name=<build-job-name> --tail=-1
```

## Builder Job pods are pending or rejected

The builder Job runs a privileged container because it builds container images.
A `restricted` Pod Security Standard on `zuplo-system` blocks it:

```
pods "build-..." is forbidden: violates PodSecurity "restricted:latest":
privileged (container "builder" must not set securityContext.privileged=true)
```

Label the namespace for the `privileged` policy, or talk to your Zuplo solutions
architect about alternatives.

## Deployments not found after Helm installation

```text
Error from server (NotFound): deployments.apps not found
```

Look in the right namespace. The management plane runs in `zuplo-system`;
gateway deployments and every subchart run in `zuplo`.

```bash
kubectl get pods -n zuplo-system
kubectl get pods -n zuplo
```

## Container registry authorization failure

```text
Error: failed to authorize: failed to fetch anonymous token: unexpected status from GET request ... 403 Forbidden
```

`helm registry login` has not run, or the credential expired.

```bash
printf '%s' "$ZUPLO_REGISTRY_KEY" |
  helm registry login us-docker.pkg.dev -u _json_key_base64 --password-stdin
```

The same credential pulls the chart and the component images. If pods remain in
`ImagePullBackOff`, check that `zuploImageRegistry.password` is set.

## Gateway image pull failure

```text
ImagePullBackOff
```

The built image pushed successfully but the cluster cannot pull it back.
`builder.registry` credentials need **both** push and pull rights, and the
`builder-secret` is used for both.

```bash
kubectl describe pod -n zuplo <pod> | tail -20
```

## Kubernetes client version skew warning

```
WARNING: version difference between client (1.32) and server (1.36) exceeds the
supported minor version skew of +/-1
```

Upgrade `kubectl` before continuing. A client outside the supported version skew
can omit fields added by the server.
