GPU metrics
This group appears when the daemon reports at least one GPU. Each card is titled from vendor and chip (for example NVIDIA H100).
A license tier can allow GPU reporting (every tier watches at least two) without a machine having a GPU. No GPU in inventory means no group — that is expected, not a misconfiguration. A card's temperature and power are not on the GPU row any more — they are physical signals, captioned with the card's name.
Utilization
| Series | Wire name | What it is |
|---|---|---|
| GPU | gpu.utilizationPercent | Share of time the GPU was busy. NVIDIA/AMD: compute engines. Intel iGPU: GT-awake from RC6 residency when the kernel has no engine/*/busy sysfs (the same boards' btop / intel_gpu_top numbers come from the i915 PMU instead) |
| Memory activity | gpu.memoryActivityPercent | Share of time the GPU memory interface was busy. NVIDIA (NVML/DCGM) only — Intel/AMD sysfs do not export this, and the series hides rather than plotting empty |
High compute with low memory activity is arithmetic-bound work. High memory activity with modest compute is bandwidth-bound (large tensors, scans). Both near 100% is a fully used GPU.
Memory used
gpu.memoryUsedBytes — framebuffer / HBM (or equivalent) in use. Hides when the vendor does not export a device-level used-bytes counter (Intel iGPUs typically do not).
GPUs do not overcommit the way Linux RAM does. Approaching the card's capacity fails new allocations (the GPU runtime reports out of memory). This is on-device memory, not host RAM — host pressure is on Memory.
Temperature
A GPU's core temperature and, when the vendor exports it, memory temperature are reported as physical signals, one chart each, captioned with the card's name. They are not fields on the gpu.* row. A VM reports no signals at all, so a passed-through GPU in a VM has no temperature chart.
Rising temperature with a rising throttle line is the card protecting itself — performance drops on purpose. Critical thermal events are listed under Events.
Power draw
Instantaneous board power is likewise a physical signal in watts, not a gpu.* field. It is absent when the vendor does not export power. Intel iGPUs typically have no i915 hwmon wattage; the daemon then uses RAPL PP1 (uncore / energy-gpu) energy over the sampling interval, the same RAPL GPU energy btop reads. Kernel RAPL energy_uj is root-only; TurboPanel grants the daemon group read access on install/converge so those watts appear without running the daemon as root.
A card stuck well below its usual draw while utilization is high can mean a power cap or a throttle. A card at the cap with utilization below 100% is often power-limited, not compute-limited.
PCIe throughput
| Series | Wire name | What it is |
|---|---|---|
| Receive | gpu.pcieReceiveBytesPerSecond | Host → GPU over the PCIe link |
| Transmit | gpu.pcieTransmitBytesPerSecond | GPU → host |
Sustained high PCIe with modest GPU utilization often means the job is bottlenecked on copies, not kernels. Multi-GPU hosts that also have a dedicated GPU interconnect still show host-link traffic here; that interconnect is separate.
Throttle
gpu.throttlePercent — share of time the GPU was throttling (thermal, power, or other clocks-down).
Any sustained value above zero is lost performance. Pair with the card's temperature and power signals to see whether heat or a power limit is the cause. A gpu_thermal_critical event is the discrete “this went past a chart.”
Related
- Physical signals — this card's temperature and power, alongside the board and package sensors
- GPU events — Xid, ECC, retirement, fallen-off-bus
Last updated on