Jenkins controller and agent architecture: executors, isolation and credentials
Explain Jenkins controllers, agents, nodes and executors. Diagnose queued builds and design isolation between untrusted builds, release credentials and controller state.
TL;DR: The Jenkins controller coordinates builds and stores their configuration. Agents execute work through executor slots. Add capacity only after identifying the queue constraint, and keep untrusted build execution outside the controller's security boundary.
An executor is a scheduling slot
A node describes an execution environment in Jenkins. Its agent process connects that environment to the controller, and its configured executors determine how many tasks can run concurrently there. Several executors on one machine still share that machine's resources and, depending on configuration, its trust boundary.
Older interview material calls these roles “master” and “slave.” Current Jenkins documentation uses controller and agent. The architectural question is who schedules work, where that work runs, and what it can access. The nodes reference describes executor capacity and cautions about parallel jobs sharing a node.
A controller has persistent operational state: job configuration, credentials, plugin configuration and build records among other data. Treat its recovery and access controls separately from disposable build workers. An agent that can be recreated after every job does not eliminate the need to recover the controller or the external artifact repository.
Isolate the controller from build code
A build can execute repository-controlled commands. Running those commands on the controller puts its filesystem and process environment within reach. Jenkins recommends avoiding builds on the built-in node, commonly by setting its executors to zero. Agent-to-controller security controls also remain necessary; moving execution to another node does not make every agent harmless. See controller isolation.
Separate trusted release jobs from pull requests that can contain arbitrary code. A label selects matching agents; it is not a security policy by itself. Restrict which jobs can obtain credentials, which repositories can change privileged pipelines, and which identities can create or reconfigure agents. An ephemeral agent may still inherit a powerful cloud instance role or mount a host container socket.
| Resource | What to protect | Failure from a weak boundary |
|---|---|---|
| Controller home | Configuration and credentials | Broad CI compromise or difficult recovery |
| Agent workspace | Source, outputs and temporary secrets | Cross-build residue or tampering |
| Agent cloud identity | Infrastructure API permissions | Build code acts outside the job's purpose |
| Artifact repository | Published release identity | Downstream jobs consume a replaced output |
Explain a queue before buying machines
Assume six agents each have two executors. If every job takes two minutes and jobs use one executor, ideal saturated throughput is twelve jobs per two minutes, or six jobs per minute. This is an illustrative upper bound. Image pulls, startup time, shared storage and uneven job duration reduce useful capacity.
Now suppose a release job waits while eight executor slots are idle. It requires the release-linux label, which only one busy agent has. Fleet-wide utilization hides the actual constraint. Check the queue's stated reason, the required label expression, node connectivity, concurrency controls and any resource locks before adding general-purpose workers.
Increasing executors from two to eight on a memory-constrained node can make every build slower. Measure time waiting separately from time running. Then observe CPU, memory, disk latency and dependency-service limits on the matching pool. The appropriate fix may be a smaller build working set, a corrected label or another isolated release worker.
Keep execution and release authority visible
A pipeline can declare agent none at its top level and assign execution environments to individual stages. This helps explain why a lightweight review stage and a credentialed release stage have different requirements. It does not prove that an untrusted branch cannot modify the privileged stage. Repository permissions, pipeline trust rules and credential scope must enforce that boundary.
Never interpolate a secret into a command just because console masking exists. Build tools can leak it through process arguments, generated files or a network request. Jenkins's build security guidance discusses risks from shared execution environments and credentials.
Self-check: an untrusted PR and a release job run as the same OS user on separate executors of one persistent agent. Does masking the release token make this safe? No. The other job may inspect shared process or filesystem state. Move the jobs to appropriately isolated environments and review every inherited identity. Console masking addresses one output channel, not the execution boundary.