troubleshooting Servers & hardware
Find a server resource bottleneck
Distinguish CPU, memory, storage and network bottlenecks by matching resource-wait evidence to the slow requests, workload scope and time window. A busy resource is a candidate to investigate, not proof that it delayed the affected work.
What it means
A resource bottleneck constrains the work you are trying to complete. Finding it requires more than locating the highest activity reading: you need evidence of what the affected work waited for. This exercise isolates one part of that investigation—the share of a declared interval with some CPU-stalled or I/O-stalled tasks—without claiming to identify the slow request's cause.
Try it
Simulation: inspect the baseline result, then set Time with some CPU-stalled tasks (ms) to 0 and Time with some I/O-stalled tasks (ms) to 0. Inspect how the result and explanation change. Select Compare with baseline to contrast the current result with the documented starting result. These controls change only synthetic inputs, not a server.
Compare resource-stall evidence
Change one condition and inspect the computed result, its cause, and the model boundary.
Synthetic stall intervals, not utilization or a diagnosis. CPU and I/O stalls may overlap; do not add their percentages or assume a universal saturation threshold.
CPU-stall share 20.0%; I/O-stall share 5.0%. Match these intervals to the affected requests before assigning a cause.
A busier device is not automatically the bottleneck. This exercise compares waiting evidence, not root causes.
| Input | Supplied value |
|---|---|
| Observation window (ms) | 60000 |
| Time with some CPU-stalled tasks (ms) | 12000 |
| Time with some I/O-stalled tasks (ms) | 3000 |
Synthetic local learning example. No account, real network, server, filesystem, credential, command or production change is accessed.
About this model
Synthetic stall intervals, not utilization or a diagnosis. CPU and I/O stalls may overlap; do not add their percentages or assume a universal saturation threshold.
These are bounded teaching rules. Actual behavior needs the linked subject specifications and independent observations. The examples are not live checks.
The baseline result is “CPU stalls occupy more of this window”: CPU-stall share 20.0%; I/O-stall share 5.0%. CPU waiting occupies more of the supplied interval, but the model does not know which requests were waiting. Match these intervals to the affected requests before assigning a cause.
Keep Observation window (ms) at 60000. Only the two stall-duration inputs change for this contrast.
Reset restores Observation window (ms) to 60000, Time with some CPU-stalled tasks (ms) to 12000 and Time with some I/O-stalled tasks (ms) to 3000. It removes input errors and clears the comparison. It preserves your independent open or closed choices for FAQ answers and About this model.
How it works
A utilization observation describes resource activity. A stall observation describes work waiting for a resource. If the waiting work is needed to complete an affected request, that wait can delay the request. The missing link in many diagnoses is whether the observed waiting actually belongs to the slow path.
Here, each input represents elapsed time during which some tasks were stalled on one resource—not the sum of every task's wait. The model divides that duration by the observation window and multiplies by 100. Using the same denominator makes the two waiting shares comparable, but it does not make them independent parts of a whole.
The comparison then asks only which duration is larger. More CPU-stalled time yields “CPU stalls occupy more of this window”; more I/O-stalled time yields the corresponding I/O result. Neither branch identifies a device, application or request as the cause. The intervals can overlap, and the waiting may concern work unrelated to the symptom.
Worked example
This is a synthetic baseline, not a production observation.
| Input | Value |
|---|---|
| Observation window (ms) | 60000 |
| Time with some CPU-stalled tasks (ms) | 12000 |
| Time with some I/O-stalled tasks (ms) | 3000 |
The CPU calculation is 12000 ÷ 60000 × 100 = 20.0%. The I/O calculation is 3000 ÷ 60000 × 100 = 5.0%. Because 12000 ms exceeds 3000 ms, the model reports that CPU stalls occupy more of this window.
That result establishes a difference in supplied waiting durations, not a difference in request latency. It does not mean CPU use was 20.0%, that total stalled time was 25.0% or that adding CPU capacity would fix the slowdown. The next useful observation is whether the affected requests depended on the work that was CPU-stalled.
What changes when the condition changes
Set Time with some CPU-stalled tasks (ms) to 0 and Time with some I/O-stalled tasks (ms) to 0, leaving Observation window (ms) at 60000.
The result becomes “No stall evidence in these two counters,” with CPU-stall share 0.0% and I/O-stall share 0.0%. Both supplied waiting durations have disappeared, so neither counter now supports a resource-wait explanation for this window. The model returns unknown rather than a healthy-system verdict: other resources, locks, network delay or a different sampling window may matter. Editing these teaching inputs has not repaired or retested a real server.
In practice
Use this as a read-only triage plan for an environment you administer or are authorized to monitor. Start with existing observations; this page does not require a privileged shell, a configuration change or a load test.
- Define the symptom. Record the slow operation, affected service or host, request interval and observed latency or errors. “The server is slow” is not enough to select a resource.
- Check the observation scope. Confirm that the resource data covers the relevant workload and interval, and that you understand its units. Host-wide waiting can involve work other than the requests you are investigating. Data from another workload or time window cannot settle this case.
- Compare competing explanations. Align request latency with available CPU, I/O and memory-pressure observations. Then seek the least costly existing trace or queue observation that can connect a wait to the affected operation.
| Candidate | Read-only evidence to seek in the affected interval |
|---|---|
| CPU | CPU waiting plus task, request or queue evidence connecting that wait to the slow operation |
| Storage/I/O | I/O waiting plus evidence that the affected operation was waiting for I/O |
| Memory | Memory-pressure observations for the relevant workload and evidence connecting them to its delay |
| Network or an application dependency | Request-level or network/dependency observations locating the delayed step |
For example, CPU stalls during a slow-request interval and a request waiting on an application dependency are competing explanations until you locate the affected work's delay. The CPU stalls might concern other work. Conversely, evidence linking the request's delayed work to CPU waiting makes CPU contention a better-supported candidate. Time alignment helps narrow the investigation, but coincidence alone does not establish cause.
Verify the proposed explanation against the original slow operation, not just against a busy chart. If the scope does not match, the observations conflict or the delayed step is still unidentified, record the cause as unknown and collect the missing evidence.
STOP before changing capacity, replacing storage or restarting services when the resource has not been linked to the slow path. This page supplies no corrective-change procedure. Any later repair needs an authorized, platform-specific plan with exact scope, preserved before-state, expected observations, STOP conditions, verification of the original symptom and a rollback that can itself be verified.
Limits and unknowns
This Simulation compares synthetic waiting evidence, not utilization, device performance or root causes. It evaluates only the supplied CPU and I/O durations; it has no memory, network, lock, request-identity or dependency measurements. A larger share does not establish a universal saturation threshold or predict how much faster a request would become after a change.
The observation window accepts 1–600000 ms; each stall duration accepts 0–600000 ms and must not exceed the window. A duration greater than the window produces “Impossible time window.” An invalid current input has no calculated outcome until you correct it or use Reset; the previous valid result must not be treated as applying to that input.
When both durations are zero, the result is unknown. Equal nonzero durations produce “Equal stall durations; more evidence needed,” not proof that both resources are bottlenecks. CPU and I/O intervals may overlap, so do not add their percentages as shares of elapsed time.
No real machine, account, request or service is inspected. The baseline and contrast remain readable without JavaScript; the interactive controls require JavaScript and perform no external action.
What to carry forward
Use resource observations to choose the next discriminating observation, not to declare a winner among charts. A stall matters to your diagnosis when you can connect it to work needed by the slow operation. If that connection is missing, keep the cause unknown and seek request-level or queue evidence before choosing a repair.
Frequently asked questions
Editorial questions, answered from the sources listed for this article.
Can CPU and I/O stall percentages be added?
Not as disjoint shares of elapsed time. In a synthetic 1000 ms window, CPU stalls and I/O stalls can each occupy 1000 ms, giving 100.0% for each resource. Adding them produces 200%, but no extra elapsed time exists: both conditions can occur during the same interval.
Is there a universal utilization percentage that proves a bottleneck?
No. A rule such as 80% or 85% can be an investigation trigger for a particular workload, but it does not by itself establish what delayed a request. Interpret utilization alongside workload latency, queues and waiting evidence. This example deliberately applies no universal cutoff.
Does Linux PSI some mean every task was stalled?
No. Linux PSI some tracks time when at least one task is stalled on the resource, including intervals when all relevant tasks are stalled. It does not require every task to be stalled; full measures that narrower condition for the applicable scope. A 20.0% some-stall share describes time, not the percentage of tasks affected or CPU utilization.
Do zero values here prove the system is healthy?
No. First check whether the observation window includes the slow requests and whether the counters cover the relevant workload. Only two supplied stall durations are evaluated here. If requests remain slow, another resource, missing instrumentation or a different interval may still matter; zero values do not resolve the original symptom.
Sources & model boundaries
Primary references used for this explanation. The interactive example is an educational model, not a live check.
- Linux pressure stall information
some/full fields, averages and cumulative stall time; Pressure interface: some and full; time accounting. Waiting evidence has a defined scope; resource stall periods can overlap; Some includes time when at least one task is stalled, including full-stall intervals. It is a time measure, not the percentage of tasks.
Checked .
- Google SRE: monitoring distributed systems
Black-box/white-box monitoring; latency, traffic, errors and saturation. A probe tests specified behavior from a particular observation surface
Checked .