PKI certificate issuance and renewal: prove identity and verify the served certificate
Explain certificate issuance, ACME challenges and renewal failure modes. Verify hostname, chain and live endpoint identity instead of trusting a successful renewal job.
TL;DR: Issuance proves the requested identity under the issuer's policy and produces a signed certificate. Renewal is operationally complete only when the intended endpoint serves the new valid chain and its private key remains protected.
The certificate and private key have different jobs
The certificate binds an identity, such as a DNS name, to a public key under an issuer's signature. The server proves possession of the corresponding private key during TLS authentication. Copying the certificate file is usually not secret distribution; copying the private key is.
Clients evaluate the trust chain, certificate validity and the hostname they intended to reach. A valid certificate for reports.example.test does not authenticate billing.example.test. A browser accepting one frontend also does not prove that every load-balancer target or internal client trusts the same chain. The existing TLS handshake explanation covers those client checks.
Domain validation has operational dependencies
ACME automates interactions with a certificate authority. HTTP-01 serves a challenge through HTTP on port 80, while DNS-01 publishes a DNS TXT record and supports wildcard issuance. Multiple frontends and DNS propagation can make validation fail intermittently. Let's Encrypt's challenge documentation explains the mechanisms and their restrictions.
DNS automation needs credentials with an appropriate scope. Giving a public web process broad DNS administration access can turn a server compromise into control of unrelated domains. Prefer a restricted validation identity or delegated challenge zone when the provider and architecture support it.
A failed challenge and a failed deployment need different remediation. Preserve which phase failed before retrying. Repeated issuance requests do not repair a reverse proxy that never reloads its certificate.
Verify the endpoint after the reload
Inspect the leaf certificate presented through the real server name, including Server Name Indication. Compare its serial or fingerprint and expiry with the issued artifact. Verify the chain using the trust store relevant to actual clients; a developer laptop can trust a root that an application image lacks.
For an authorized endpoint, an OpenSSL inspection can supply -servername, -verify_hostname and -verify_return_error to s_client. The OpenSSL s_client reference explains these flags. Without explicit verification behavior, a diagnostic command can print a certificate or verification warning and still mislead a script that checks only whether output exists.
| Evidence | Confirms | Does not confirm |
|---|---|---|
| ACME order completed | Issuer produced the certificate | Any frontend loaded it |
| Certificate file has new expiry | Local artifact changed | Network traffic reaches that artifact |
| One endpoint verifies | That connection's identity and trust | Every region or backend is updated |
| Representative clients succeed | Tested clients accept the chain | Untested private trust stores match |
Work through a partial rollout
Suppose three edge instances should serve the same hostname. Two load the new certificate, while the third has a broken reload hook and retains the old one. A synthetic check through the public load balancer sometimes succeeds and sometimes reports imminent expiry.
Collect the presented fingerprint alongside the connection path where possible. Use controlled checks that reach each intended frontend while preserving SNI and hostname validation. Fix the failed deployment and verify again. Requesting another certificate can consume issuance capacity while leaving the bad frontend unchanged.
Monitor remaining validity at the endpoint and the renewal workflow's result separately. A successful job with a stale endpoint is a useful discrepancy. Renewal timing should follow the current issuer/client policy, including ACME Renewal Information when supported, rather than a hard-coded assumption that every certificate has the same lifetime. Let's Encrypt documents ARI integration.
Rehearse trust checks without a public issuer
A local exercise can create a temporary test CA and a leaf certificate with a DNS Subject Alternative Name. Verify that the correct hostname passes, an unrelated hostname fails, and a trust store without the test root fails. Keep the test CA out of the host's permanent trust store and remove the temporary keys afterward. This tests verification behavior, not public ACME issuance or a real load balancer reload.
Self-check: the renewal dashboard is green, but clients report an expired certificate. What is the first useful comparison? Inspect the certificate actually served for the affected hostname and compare it with the newly issued artifact. That separates an issuance problem from deployment, routing or stale process state before another renewal attempt adds noise.