blog.rust.gay

cgroups v2 in practice

Control groups (cgroups) are the Linux kernel mechanism for organizing processes into a hierarchy and distributing resources along that hierarchy in a controlled, configurable way. Cgroup v2 replaces the older multi-hierarchy model with a single unified hierarchy and a clearer set of rules for controllers, process placement, and delegation.

On systems that use cgroup v2, container runtimes and service managers such as systemd rely on it to account for and limit CPU, memory, I/O, and process counts. The kernel’s cgroup v2 documentation is the authoritative reference for interface behavior and terminology.

The unified hierarchy

Unlike cgroup v1, cgroup v2 uses one hierarchy. The kernel documentation shows mounting it with:

mount -t cgroup2 none $MOUNT_POINT

Controllers that support v2 and are not bound to a v1 hierarchy can be attached to the unified hierarchy. Each cgroup exposes control files such as cgroup.controllers, which lists controllers available to enable, and cgroup.subtree_control, which enables controllers for distribution to immediate children.

Cgroups are directories in the cgroup filesystem. A child cgroup can be created with mkdir, and an empty cgroup can be removed with rmdir. Process membership is managed through files such as cgroup.procs: reading it lists PIDs in the cgroup, and writing a PID migrates that process. In cgroup v2, a process’s membership appears in /proc/$PID/cgroup in the 0::$PATH format.

Controllers and core files

A controller regulates a specific kind of resource. The cgroup v2 documentation describes controllers including cpu, memory, io, and pids, along with their interface files and distribution models. For example:

  • The CPU controller regulates CPU cycle distribution and can use weight and bandwidth controls. The kernel documents interface files such as cpu.weight and cpu.max for non-root cgroups.
  • The memory controller provides accounting and controls such as memory.min, memory.low, memory.high, memory.max, and swap-related limits. systemd exposes many of these through unit settings such as MemoryMin=, MemoryLow=, MemoryHigh=, MemoryMax=, and MemorySwapMax=.
  • The I/O controller regulates block I/O. The kernel documents io.weight and io.max among its interfaces; systemd maps related settings such as IOWeight= and bandwidth/IOPS limits to the unified hierarchy.
  • The pids controller limits the number of tasks in a cgroup. The kernel documents pids.max; systemd exposes it as TasksMax=.

Enabling a controller is top-down: a non-root cgroup can enable a controller in cgroup.subtree_control only if that controller is available and enabled appropriately by its parent. When a controller is enabled for a subtree, the controller’s interface files appear in the affected child cgroups.

Delegation and structural rules

Cgroup v2 is designed for safe delegation. The kernel documentation describes delegation as granting a less privileged user access to a cgroup directory and selected control files, or automatically delegating to a cgroup namespace when the nsdelegate mount option is used. A delegatee can create and manage a sub-hierarchy, but resource limits imposed by ancestors still apply.

Two constraints are especially important in practice. First, resource distribution follows a top-down model: children cannot receive or exceed resources that were not distributed to their parent. Second, the kernel documents a no internal process constraint for non-root domain cgroups: a cgroup that has enabled domain controllers in cgroup.subtree_control should distribute resources to children rather than host competing internal processes. In practice, this means a service manager or container supervisor often moves its own process into a leaf or control sub-cgroup before enabling controllers for child workloads.

Systemd’s delegation guidance adds operational detail: setting Delegate= on a service or scope tells systemd not to manage attributes or create/remove cgroups inside that delegated subtree, while systemd continues to manage the delegated unit itself. This boundary matters for container managers and nested supervisors that need direct cgroup access.

Inspecting cgroups on a running system

A quick inspection usually starts with the mount and the active controllers:

mount | grep cgroup2
cat /sys/fs/cgroup/cgroup.controllers
cat /proc/self/cgroup

The kernel documents cgroup.events for populated/frozen notifications and per-controller files for statistics and limits. On systemd systems, unit configuration and runtime status often reflect the same underlying controls: for example, CPU, memory, task, and I/O settings in service or slice units map to cgroup v2 attributes when the unified hierarchy is in use.

When debugging resource behavior, it helps to identify the exact cgroup path first, then inspect the controller files in that cgroup and its ancestors. Because limits and protections are hierarchical, a constraint can come from a parent slice even when a child unit appears loosely configured.

Conclusion

Cgroup v2 makes Linux resource control more regular by using a single hierarchy, explicit controller enablement, and well-defined delegation rules. The practical payoff is easier reasoning about where limits come from and which component owns which part of the tree. For day-to-day work, the most useful habits are to read the controller files directly, understand ancestor constraints, and treat delegation boundaries as ownership boundaries.

Sources


← back to posts