blog.rust.gay

io_uring: async I/O on Linux

2026-08-17 · systems linux io

io_uring is a Linux-specific asynchronous I/O interface built around ring buffers shared between user space and the kernel. Instead of making a separate blocking system call for every operation, an application queues submission queue entries (SQEs), asks the kernel to process them, and reads completion queue entries (CQEs). The design reduces copying and system-call overhead in workloads that issue many I/O operations.

How the rings work

The kernel documentation describes two primary shared structures: the submission queue (SQ) and the completion queue (CQ). User space writes SQEs to the SQ; the kernel consumes them, performs the requested operations, and posts CQEs to the CQ. In the common case, one submitted SQE produces one CQE, but some operations can produce multiple completions (multi-shot requests) or no completion when configured to skip success notifications.

Each SQE describes an operation through fields such as opcode, fd, addr, len, off, and flags. The user_data field is passed through unchanged to the corresponding CQE, which is the usual way to correlate completions with submitted requests. A CQE reports the result in res: on success it contains the value the equivalent synchronous call would have returned; on error it contains -errno rather than setting errno.

The shared-ring model means applications must follow the memory-ordering and queue-indexing rules described in io_uring(7). Heads and tails are updated by different sides, and ordering is not implied by submission order alone.

Core system calls

io_uring is exposed through a small set of system calls:

  • io_uring_setup(entries, params) creates an io_uring instance with at least entries SQ slots and returns a file descriptor. It also reports ring offsets and supported features in struct io_uring_params.
  • io_uring_enter(fd, to_submit, min_complete, flags, sig) tells the kernel how many SQEs to consume and, optionally, waits for completions. With IORING_ENTER_GETEVENTS, it can submit and wait in one call.
  • io_uring_register(fd, opcode, arg, nr_args) registers resources such as fixed buffers, fixed files, eventfds, personalities, restrictions, and other ring-related objects. Registration can reduce per-operation overhead and, in some configurations, is required for certain features.

Most applications use liburing rather than issuing these calls directly, but understanding the syscalls helps when debugging behavior, reading kernel documentation, or reasoning about performance.

What io_uring can do

The opcode space is much broader than basic file reads and writes. io_uring(7) and io_uring_enter(2) document operations for vectored and fixed-buffer I/O, fsync, polling, networking (send, recv, accept, connect, and related variants), timeouts, file operations, splice/tee, and newer features such as multishot requests and zero-copy networking paths where supported. This breadth is why io_uring is often described as an asynchronous interface rather than only an async file-I/O API.

Two setup modes illustrate the performance trade-offs:

  • Submission queue polling (IORING_SETUP_SQPOLL) can create a kernel thread that polls the SQ, allowing submissions without a syscall on every batch in steady state. The kernel may set IORING_SQ_NEED_WAKEUP when the polling thread is idle, and the application must wake it. SQPOLL has configuration and privilege/history constraints described in io_uring_setup(2).
  • I/O polling (IORING_SETUP_IOPOLL) targets busy-waiting for completions on supported storage paths. The setup documentation notes that it is currently applicable to storage devices and, for the classic read/write fixed and direct paths, depends on device support and O_DIRECT usage.

These modes can reduce latency or syscall overhead, but they can also increase CPU use. They are not universal “faster” switches.

Ordering, safety, and operational caveats

io_uring does not guarantee that independent requests complete in submission order. Applications that need ordering must use explicit mechanisms such as linked requests (IOSQE_IO_LINK) or higher-level design choices. The io_uring(7) man page warns that overlapping sends or overlapping receives on the same stream-oriented socket can be unsafe unless the application uses the interface’s ordering or batching facilities appropriately.

Completion results also require careful interpretation. Because operations are asynchronous, errors are reported through CQE res values rather than errno. Applications should check which request produced each CQE via user_data, handle partial progress where relevant, and avoid assuming that a submitted SQE buffer can be freed before the kernel no longer needs it. The pointer-lifetime rules differ by operation and by older-kernel behavior; io_uring(7) documents the general stability requirements.

From an operational perspective, io_uring exposes a large kernel interface. io_uring_setup(2) documents EPERM cases, including systems where /proc/sys/kernel/io_uring_disabled restricts use. In security-sensitive environments, that configurability and the breadth of supported operations are reasons to evaluate whether io_uring should be enabled for untrusted workloads.

Conclusion

io_uring is best understood as a shared-memory submission/completion protocol between user space and the Linux kernel. Its value comes from batching, asynchronous completion, and optional registration/polling features—not from magic. The practical question is whether a workload can use those mechanics correctly: manage buffer lifetimes, correlate CQEs, enforce ordering where needed, and measure the CPU/latency trade-offs of polling modes.

Sources

← back to posts