⚙️
S1 — Core Kernel Subsystems
10 components
1.1
Process Creation Subsystem (`do_fork` / `copy_process`)
The process creation subsystem is the kernel mechanism that creates new processes and threads by duplicating an exist…
▾
Definition
The process creation subsystem is the kernel mechanism that creates new processes and threads by duplicating an existing task's address space, file descriptors, signal handlers, and scheduling state. It is the implementation behind the fork(2), vfork(2), clone(2), and clone3(2) syscalls and is centered in kernel/fork.c around copy_process().
Execution Flow
- Userspace calls
fork(), vfork(), clone(), or clone3().
- The syscall entry (
sys_clone, sys_fork, etc.) collects flags and enters kernel_clone().
copy_process() is called: it allocates a new task_struct via dup_task_struct(), which copies the kernel stack and sets up thread_info.
- Task struct fields are initialized: counters reset, PID assigned from the PID namespace via
alloc_pid().
- Depending on
CLONE_* flags: copy_mm() duplicates or shares the address space; copy_files() duplicates or shares file descriptors; copy_sighand() duplicates signal handlers.
- The new task is inserted into the task list under
tasklist_lock.
wake_up_new_task() places the child on a run queue with TASK_RUNNING state.
- The scheduler may preempt the parent and begin running the child immediately (copy-on-write pages remain shared until first write-fault).
- Both parent and child return from the syscall: parent receives the child's PID, child receives 0.
Edge Cases & Failure Modes
- ENOMEM: If
dup_task_struct() or copy_mm() fails (OOM condition), copy_process() unwinds via bad_fork_* error labels; partial allocations are freed.
- PID namespace exhaustion:
alloc_pid() returns NULL if the namespace's PID range is saturated; the fork fails with EAGAIN.
RLIMIT_NPROC exceeded: If the user's process count exceeds the rlimit, copy_process() returns EAGAIN; this is bypassed for CAP_SYS_ADMIN.
vfork() deadlock: The parent blocks on a completion until the child calls exec() or exits; if the child sleeps indefinitely, the parent is permanently blocked.
- Signal pending during fork: A pending fatal signal to the parent before
wake_up_new_task() causes the entire fork to be unwound.
DEF
The process creation subsystem is the kernel mechanism that creates new processes and threads by duplicating an exist…
FLOW
Userspace calls `fork()`, `vfork()`, `clone()`, or `clo…The syscall entry (`sys_clone`, `sys_fork`, etc.) colle…`copy_process()` is called: it allocates a new `task_st…Task struct fields are initialized: counters reset, PID…Depending on `CLONE_*` flags: `copy_mm()` duplicates or…The new task is inserted into the task list under `task…`wake_up_new_task()` places the child on a run queue wi…The scheduler may preempt the parent and begin running …Both parent and child return from the syscall: parent r…
EDGE
ENOMEMIf `dup_task_struct()` or `copy_mm()` fails (OOM condition), `copy_process()` un
PID namespace exhaustion`alloc_pid()` returns `NULL` if the namespace's PID range is saturated; the fork
`RLIMIT_NPROC` exceededIf the user's process count exceeds the rlimit, `copy_process()` returns `EAGAIN
`vfork()` deadlockThe parent blocks on a completion until the child calls `exec()` or exits; if th
1.2
Interrupt Request (IRQ) Subsystem (`kernel/irq/`)
The IRQ subsystem manages hardware interrupt lines from device discovery through handler dispatch.
▾
Definition
The IRQ subsystem manages hardware interrupt lines from device discovery through handler dispatch. It provides a hardware-abstracted interrupt descriptor table, supports level-triggered and edge-triggered interrupts, and coordinates with APIC/GIC interrupt controllers via the irq_chip abstraction layer.
Execution Flow
- At boot,
irq_init_percpu_irqstack() and architecture-specific init_IRQ() set up interrupt stacks and the IDT/GIC.
- A driver calls
request_irq() → __setup_irq(), which allocates an irqaction and links it to an irq_desc for the given IRQ number.
- Hardware asserts an interrupt line; the CPU saves registers and jumps to the architecture's interrupt entry stub (
entry_64.S).
handle_irq() → generic_handle_irq() → handle_level_irq() / handle_edge_irq() depending on the irq_desc->handle_irq function pointer.
- The irqchip's
irq_mask or irq_ack is called to prevent re-entry.
- The registered
irqaction->handler runs in interrupt context (hardirq).
- The handler returns
IRQ_HANDLED or IRQ_NONE; if none of the handlers claim the IRQ, the kernel logs an unhandled interrupt.
- The irqchip's
irq_unmask / irq_eoi is called.
- The CPU restores registers and resumes the interrupted task via
iret/eret.
Edge Cases & Failure Modes
- IRQ storm: A misbehaving device asserts its line continuously; the kernel detects this via
irq_safe_remove() logic and may disable the IRQ after a threshold (bad IRQ detection in handle_level_irq()).
- Shared IRQ conflict: Multiple devices on the same line (legacy PCI); if one device's handler fails to claim
IRQ_HANDLED, spurious IRQ warnings accumulate.
- NMI nesting: Non-Maskable Interrupts cannot be disabled; nested NMIs on x86 require the
nmi_nesting_preprocess guard in arch/x86/kernel/nmi.c to prevent stack corruption.
- Interrupt affinity migration race: Changing
/proc/irq/N/smp_affinity while an interrupt is in-flight can send EOI to the wrong CPU's APIC, causing the IRQ to be silently lost.
DEF
The IRQ subsystem manages hardware interrupt lines from device discovery through handler dispatch.
FLOW
At boot, `irq_init_percpu_irqstack()` and architecture-…A driver calls `request_irq()` → `__setup_irq()`, which…Hardware asserts an interrupt line; the CPU saves regis…S`).`handle_irq()` → `generic_handle_irq()` → `handle_level…The irqchip's `irq_mask` or `irq_ack` is called to prev…The registered `irqaction->handler` runs in interrupt c…The handler returns `IRQ_HANDLED` or `IRQ_NONE`; if non…The irqchip's `irq_unmask` / `irq_eoi` is called.The CPU restores registers and resumes the interrupted …
EDGE
IRQ stormA misbehaving device asserts its line continuously; the kernel detects this via
Shared IRQ conflictMultiple devices on the same line (legacy PCI); if one device's handler fails to
NMI nestingNon-Maskable Interrupts cannot be disabled; nested NMIs on x86 require the `nmi_
Interrupt affinity migration raceChanging `/proc/irq/N/smp_affinity` while an interrupt is in-flight can send EOI
1.3
System Call Interface (`arch/x86/entry/`)
The system call interface is the controlled gate through which userspace transitions to kernel privilege to request k…
▾
Definition
The system call interface is the controlled gate through which userspace transitions to kernel privilege to request kernel services. On x86-64 it uses the SYSCALL/SYSRET instructions; on ARM64, SVC #0. The interface dispatches through a syscall table (sys_call_table) indexed by syscall number.
Execution Flow
- Userspace loads the syscall number into
rax (x86-64) and arguments into rdi, rsi, rdx, r10, r8, r9, then executes syscall.
- The CPU switches to ring 0, saves
rip/rflags to per-CPU state, loads the kernel rsp, and jumps to entry_SYSCALL_64 in arch/x86/entry/entry_64.S.
- The entry stub saves all registers to the kernel stack (
struct pt_regs).
do_syscall_64() is called; it fetches the syscall number from regs->orig_ax.
- Bounds-checks the number against
NR_syscalls; looks up sys_call_table[nr].
- Runs any active
seccomp filters (BPF programs) before dispatch.
- Calls the syscall handler (e.g.,
sys_read, sys_mmap); handler returns a long result.
- The return value is placed in
regs->ax.
- The entry stub restores saved registers and executes
sysretq to return to userspace at the saved rip.
Edge Cases & Failure Modes
- Invalid syscall number: Numbers ≥
NR_syscalls return −ENOSYS.
seccomp SIGSYS kill: A BPF filter returning SECCOMP_RET_KILL_PROCESS delivers SIGSYS and kills the process before the syscall handler is reached.
- Signal pending at syscall exit:
exit_to_user_mode_prepare() checks for pending signals; if found, the syscall returns with −EINTR or −ERESTARTSYS before userspace sees the result.
SMAP/SMEP violations: If the kernel accidentally dereferences a userspace pointer without copy_from_user(), SMAP raises a fault; the kernel oopses rather than allowing privilege escalation.
- Speculation side-channels (Spectre v2 / Retpoline): The indirect
call through sys_call_table is protected by retpoline or IBRS; disabling mitigations can re-open the attack surface.
DEF
The system call interface is the controlled gate through which userspace transitions to kernel privilege to request k…
FLOW
Userspace loads the syscall number into `rax` (x86-and arguments into `rdi`, `rsi`, `rdx`, `r10`, `r8`, `r…The CPU switches to ring 0, saves `rip`/`rflags` to per…S`.The entry stub saves all registers to the kernel stack …`do_syscall_64()` is called; it fetches the syscall num…Bounds-checks the number against `NR_syscalls`; looks u…Runs any active `seccomp` filters (BPF programs) before…Calls the syscall handler (e.g., `sys_read`, `sys_mmap`…The return value is placed in `regs->ax`.The entry stub restores saved registers and executes `s…
EDGE
Invalid syscall numberNumbers ≥ `NR_syscalls` return `−ENOSYS`.
`seccomp` SIGSYS killA BPF filter returning `SECCOMP_RET_KILL_PROCESS` delivers `SIGSYS` and kills th
Signal pending at syscall exit`exit_to_user_mode_prepare()` checks for pending signals; if found, the syscall
`SMAP`/`SMEP` violationsIf the kernel accidentally dereferences a userspace pointer without `copy_from_u
1.4
Workqueue Subsystem (`kernel/workqueue.c`)
The workqueue subsystem provides a kernel-thread pool for deferred, process-context work.
▾
Definition
The workqueue subsystem provides a kernel-thread pool for deferred, process-context work. Unlike tasklets (softirq-based), workqueue callbacks can sleep. It manages worker threads (kworkers), work items (struct work_struct), and thread pools organized as worker_pool objects per CPU and per NUMA node.
Execution Flow
- A kernel subsystem defines a
work_struct and initializes it with INIT_WORK(&work, handler_fn).
queue_work() → __queue_work() selects the appropriate worker_pool (bound to the current CPU for normal workqueues, or unbound for UNBOUND workqueues).
- The work item is added to the pool's
worklist.
- If no idle worker is available and the pool is below the concurrency target,
create_worker() spawns a new kworker kernel thread.
- The kworker thread calls
worker_thread() → process_one_work().
process_one_work() calls the handler function: work->func(work).
- After the handler returns, the worker checks for more pending work; if none, it goes idle.
- Idle workers are reaped after
IDLE_WORKER_TIMEOUT (5 minutes) to prevent thread proliferation.
Edge Cases & Failure Modes
- Work item requeue during flush: Calling
queue_work() from within a handler re-enqueues for the next iteration; calling flush_work() from within the same workqueue deadlocks.
- Worker pool stall (watchdog): If all workers are blocked (e.g., waiting on a mutex held by work in the same pool),
wq_watchdog_timer_fn() fires after wq_watchdog_thresh seconds and dumps a stack trace.
- Memory allocation failure in
create_worker(): If kthread_create() fails during worker creation, the pool falls back to existing workers; if zero workers exist, work is never processed.
WQ_MEM_RECLAIM ordering: Non-WQ_MEM_RECLAIM workqueues must not be called from memory-reclaim paths; doing so can cause deadlock when kswapd tries to reclaim memory held by a worker waiting on that same workqueue.
DEF
The workqueue subsystem provides a kernel-thread pool for deferred, process-context work.
FLOW
A kernel subsystem defines a `work_struct` and initiali…`queue_work()` → `__queue_work()` selects the appropria…The work item is added to the pool's `worklist`.If no idle worker is available and the pool is below th…The kworker thread calls `worker_thread()` → `process_o…`process_one_work()` calls the handler function: `work-…After the handler returns, the worker checks for more p…Idle workers are reaped after `IDLE_WORKER_TIMEOUT` (5 …
EDGE
Work item requeue during flushCalling `queue_work()` from within a handler re-enqueues for the next iteration;
Worker pool stall (watchdog)If all workers are blocked (e.g., waiting on a mutex held by work in the same po
Memory allocation failure in `create_worker()`If `kthread_create()` fails during worker creation, the pool falls back to exist
`WQ_MEM_RECLAIM` orderingNon-`WQ_MEM_RECLAIM` workqueues must not be called from memory-reclaim paths; do
1.5
Kernel Timer Subsystem (`kernel/time/`)
The timer subsystem provides two classes of kernel timers: `hrtimer` (high-resolution timers backed by CLOCK_MONOTONI…
▾
Definition
The timer subsystem provides two classes of kernel timers: hrtimer (high-resolution timers backed by CLOCK_MONOTONIC/CLOCK_REALTIME, using hardware timer interrupts down to nanosecond resolution) and legacy timer_list (low-resolution jiffies-based timers). It also exposes POSIX timers to userspace via timer_create(2).
Execution Flow
- A caller initializes a timer:
hrtimer_init(&ht, CLOCK_MONOTONIC, HRTIMER_MODE_REL) and sets ht.function = callback.
hrtimer_start() calls hrtimer_start_range_ns(), which inserts the timer into the per-CPU hrtimer_clock_base red-black tree ordered by expiry time.
- The hardware timer (TSC-deadline, HPET, or ARM architected timer) is programmed to fire at the earliest expiry in the tree.
- When the hardware fires, the interrupt handler calls
hrtimer_interrupt().
hrtimer_interrupt() iterates expired timers from the rbtree front: calls timer->function(timer).
- The callback returns
HRTIMER_RESTART (re-arms) or HRTIMER_NORESTART (removes from tree).
- The hardware timer is reprogrammed for the next expiry.
Edge Cases & Failure Modes
- Cascading timer overload: If hundreds of timers expire simultaneously,
hrtimer_interrupt() processes them all in one IRQ context, extending interrupt latency significantly.
- Clock discontinuity: Calling
settimeofday() or adjtimex() while CLOCK_REALTIME-based hrtimers are pending can cause incorrect future expiry; CLOCK_MONOTONIC timers are immune.
timer_list overflows: Jiffies-based timers use timer_wheels; a timer set for more than MAX_JIFFY_OFFSET jiffies into the future wraps silently and fires immediately.
nohz_full interactions: On CPUs in nohz_full mode, the tick is suppressed; timers are only re-evaluated when a wake-up event occurs, adding up to one-tick jitter to expected expiry.
DEF
The timer subsystem provides two classes of kernel timers: `hrtimer` (high-resolution timers backed by CLOCK_MONOTONI…
FLOW
A caller initializes a timer: `hrtimer_init(&ht, CLOCK_…`hrtimer_start()` calls `hrtimer_start_range_ns()`, whi…The hardware timer (TSC-deadline, HPET, or ARM architec…When the hardware fires, the interrupt handler calls `h…`hrtimer_interrupt()` iterates expired timers from the …The callback returns `HRTIMER_RESTART` (re-arms) or `HR…The hardware timer is reprogrammed for the next expiry.
EDGE
Cascading timer overloadIf hundreds of timers expire simultaneously, `hrtimer_interrupt()` processes the
Clock discontinuityCalling `settimeofday()` or `adjtimex()` while `CLOCK_REALTIME`-based hrtimers a
`timer_list` overflowsJiffies-based timers use `timer_wheels`; a timer set for more than `MAX_JIFFY_OF
`nohz_full` interactionsOn CPUs in `nohz_full` mode, the tick is suppressed; timers are only re-evaluate
1.6
Power Management Framework (`drivers/base/power/`)
The kernel power management framework orchestrates system-wide and per-device power state transitions.
▾
Definition
The kernel power management framework orchestrates system-wide and per-device power state transitions. It supports runtime PM (device-level power gating), system sleep states (suspend-to-RAM, suspend-to-disk / hibernate), and CPU idle management (via the cpuidle and cpufreq subsystems).
Execution Flow
- A userspace tool writes
mem or disk to /sys/power/state.
enter_state() → pm_prepare() calls each device driver's .prepare() callback in topological order (children before parents).
dpm_suspend() calls .suspend() callbacks on all devices (device PM list traversal).
dpm_suspend_late() / dpm_suspend_noirq() follow, allowing drivers to cut clocks/power after IRQs are disabled.
- CPU-specific suspend: each CPU is taken offline except the last; the last CPU calls
arch_suspend_cpu() which issues the platform-specific sleep instruction (e.g., ACPI _GTS, ARM PSCI).
- On wake: the platform restores CPU state;
arch_resume_cpu() runs; IRQs re-enabled.
dpm_resume_noirq() → dpm_resume_early() → dpm_resume() calls .resume() in reverse topological order.
pm_finish() calls .complete() on all devices; the system returns to S0.
Edge Cases & Failure Modes
- Driver suspend failure: A driver returning non-zero from
.suspend() aborts the suspend; dpm_resume() is called on already-suspended devices to restore state.
- Firmware / ACPI AML bugs: BIOS
_PTS or _WAK methods can crash or wedge the system; acpi_suspend_error paths attempt to recover but sometimes leave hardware in a broken state.
- Hibernation image corruption: If the swap partition is modified between snapshot and restore, the kernel detects a checksum mismatch and refuses to restore; the system cold-boots instead.
- Runtime PM reference count leak: A driver calling
pm_runtime_get() without a matching pm_runtime_put() prevents the device from ever entering a low-power state.
DEF
The kernel power management framework orchestrates system-wide and per-device power state transitions.
FLOW
A userspace tool writes `mem` or `disk` to `/sys/power/…`enter_state()` → `pm_prepare()` calls each device driv…`dpm_suspend()` calls `.suspend()` callbacks on all dev…`dpm_suspend_late()` / `dpm_suspend_noirq()` follow, al…CPU-specific suspend: each CPU is taken offline except …On wake: the platform restores CPU state; `arch_resume_…`dpm_resume_noirq()` → `dpm_resume_early()` → `dpm_resu…`pm_finish()` calls `.complete()` on all devices; the s…
EDGE
Driver suspend failureA driver returning non-zero from `.suspend()` aborts the suspend; `dpm_resume()`
Firmware / ACPI AML bugsBIOS `_PTS` or `_WAK` methods can crash or wedge the system; `acpi_suspend_error
Hibernation image corruptionIf the swap partition is modified between snapshot and restore, the kernel detec
Runtime PM reference count leakA driver calling `pm_runtime_get()` without a matching `pm_runtime_put()` preven
1.7
Device Model and sysfs (`drivers/base/`, `fs/sysfs/`)
The kernel device model provides a unified hierarchical representation of all hardware devices and their relationship…
▾
Definition
The kernel device model provides a unified hierarchical representation of all hardware devices and their relationships (buses, devices, drivers). It is exposed to userspace via sysfs (a virtual filesystem mounted at /sys), enabling enumeration, configuration, and udev event delivery for hotplug.
Execution Flow
- A bus driver (e.g., PCI) scans for devices and calls
device_register() for each discovered device.
device_register() → kobject_add() creates a kobject node in the sysfs hierarchy under /sys/devices/.
bus_probe_device() iterates registered drivers; for each, calls driver_match_device() (uses device IDs or DT compatible strings).
- On match,
driver_probe_device() calls driver->probe().
- The driver claims the device; attributes are created via
device_create_file(), populating sysfs attribute files.
- A
uevent is sent via kobject_uevent() → netlink_broadcast() to inform udevd of the new device.
udevd processes the event, creates /dev nodes, loads firmware, runs udev rules.
Edge Cases & Failure Modes
- Driver probe failure: If
probe() returns an error, driver_probe_device() decrements the driver's reference count; if no other driver matches, the device sits unbound.
- Probe deferral (
-EPROBE_DEFER): A driver may return -EPROBE_DEFER if a dependency (e.g., a GPIO chip) is not yet available; the kernel re-queues the probe after any new device is registered.
- sysfs attribute race: Reading a sysfs attribute while
device_unregister() is running can reference freed memory if the driver did not hold the device reference count correctly.
kobject reference count underflow: Calling kobject_put() more times than kobject_get() triggers a BUG() in kobject_release().
DEF
The kernel device model provides a unified hierarchical representation of all hardware devices and their relationship…
FLOW
A bus driver (e.g., PCI) scans for devices and calls `d…`device_register()` → `kobject_add()` creates a `kobjec…`bus_probe_device()` iterates registered drivers; for e…On match, `driver_probe_device()` calls `driver->probe(…The driver claims the device; attributes are created vi…A `uevent` is sent via `kobject_uevent()` → `netlink_br…`udevd` processes the event, creates `/dev` nodes, load…
EDGE
Driver probe failureIf `probe()` returns an error, `driver_probe_device()` decrements the driver's r
Probe deferral (`-EPROBE_DEFER`)A driver may return `-EPROBE_DEFER` if a dependency (e.g., a GPIO chip) is not y
sysfs attribute raceReading a sysfs attribute while `device_unregister()` is running can reference f
`kobject` reference count underflowCalling `kobject_put()` more times than `kobject_get()` triggers a `BUG()` in `k
1.8
Kernel Module Loader (`kernel/module/`)
The module loader implements `insmod`/`modprobe` support: it verifies ELF64 `.ko` files for valid `vermagic` (kernel …
▾
Definition
The module loader implements insmod/modprobe support: it verifies ELF64 .ko files for valid vermagic (kernel version + config hash), applies relocations, resolves symbol dependencies against the kernel's exported symbol table (ksymtab), runs module initialization, and later handles safe removal.
Execution Flow
- Userspace calls
finit_module(fd, params, flags) or init_module(buf, len, params).
load_module() reads the ELF sections into kernel memory via vmalloc_exec().
check_modinfo() validates the vermagic string; mismatches abort immediately.
simplify_symbols() resolves undefined symbols against ksymtab and ksymtab_gpl; unresolved symbols abort.
apply_relocations() patches the module's code/data for the final load address (KASLR randomizes this).
module_init_layout_section() sets up __init and core sections; __init is marked executable only during do_one_initcall().
do_one_initcall(mod->init) runs the module's init_module() function.
- On success, the module is added to
modules list; __init sections are freed.
- On
rmmod: module_exit() is called; free_module() unmaps the ELF image; symbols are removed from the lookup table.
Edge Cases & Failure Modes
- Symbol version mismatch: Even if the symbol name matches, a CRC checksum mismatch (
CONFIG_MODVERSIONS) causes ENOEXEC.
init() failure: If init_module() returns non-zero, the module is immediately unloaded; partial initialization (e.g., registered but un-initialized hardware) must be cleaned up by the module itself.
- Tainted kernel: Loading an unsigned or out-of-tree module sets the
TAINT_OOT_MODULE flag; this affects crash dump analysis.
- Unload race: If a module is in use (reference count > 0),
rmmod returns EWOULDBLOCK; force-removal (rmmod -f) bypasses this but can crash the kernel.
DEF
The module loader implements `insmod`/`modprobe` support: it verifies ELF64 `.ko` files for valid `vermagic` (kernel …
FLOW
Userspace calls `finit_module(fd, params, flags)` or `i…`load_module()` reads the ELF sections into kernel memo…`check_modinfo()` validates the `vermagic` string; mism…`simplify_symbols()` resolves undefined symbols against…`apply_relocations()` patches the module's code/data fo…`module_init_layout_section()` sets up `__init` and `co…`do_one_initcall(mod->init)` runs the module's `init_mo…On success, the module is added to `modules` list; `__i…On `rmmod`: `module_exit()` is called; `free_module()` …
EDGE
Symbol version mismatchEven if the symbol name matches, a CRC checksum mismatch (`CONFIG_MODVERSIONS`)
`init()` failureIf `init_module()` returns non-zero, the module is immediately unloaded; partial
Tainted kernelLoading an unsigned or out-of-tree module sets the `TAINT_OOT_MODULE` flag; this
Unload raceIf a module is in use (reference count > 0), `rmmod` returns `EWOULDBLOCK`; forc
1.9
Memory Management Subsystem (`mm/`)
The `mm/` subsystem is the kernel's unified memory management layer: it manages physical page frames (via the buddy a…
▾
Definition
The mm/ subsystem is the kernel's unified memory management layer: it manages physical page frames (via the buddy allocator), virtual address spaces (vm_area_struct / mmap), page tables, demand paging, swap, and memory compaction/reclaim. It is the largest single subsystem in the kernel.
Execution Flow
mmap(2) calls do_mmap() → mmap_region(), which allocates a vm_area_struct and inserts it into the process's mm_struct maple tree / red-black tree.
- No physical pages are allocated yet (lazy allocation).
- When the process accesses the mapped region, the CPU raises a page fault (
#PF / ESR_ELx_EC_DABT_LOW).
do_page_fault() → handle_mm_fault() → handle_pte_fault() determines the fault type: anonymous page, file-backed page, copy-on-write, or swap-in.
- For anonymous faults:
do_anonymous_page() calls alloc_zeroed_user_highpage_movable() (buddy allocator), sets up the PTE.
- For file-backed:
do_fault() calls the VMA's vm_ops->fault() to load the page from the page cache.
update_mmu_cache() flushes the TLB entry.
kswapd and vmscan run asynchronously to reclaim pages under memory pressure, writing dirty pages to swap or disk.
Edge Cases & Failure Modes
- OOM (Out-of-Memory): When both physical memory and swap are exhausted,
out_of_memory() invokes the OOM killer, which selects and kills a process based on oom_score.
- Huge page splinting: A 2 MiB THP can be split back to 4 KiB pages under memory pressure; the split operation must pause all CPUs accessing the page (via
split_huge_page()).
- Stack overflow: The stack VMA grows down; if a process exhausts its
RLIMIT_STACK, the page fault is not on a valid VMA and delivers SIGSEGV.
mmap_lock contention: The mmap_lock (formerly mmap_sem) is a per-process read-write semaphore; under high contention (many threads faulting simultaneously), it becomes a scalability bottleneck.
DEF
The `mm/` subsystem is the kernel's unified memory management layer: it manages physical page frames (via the buddy a…
FLOW
`mmap(` calls `do_mmap()` → `mmap_region()`, which allocates …No physical pages are allocated yet (lazy allocation).When the process accesses the mapped region, the CPU ra…`do_page_fault()` → `handle_mm_fault()` → `handle_pte_f…For anonymous faults: `do_anonymous_page()` calls `allo…For file-backed: `do_fault()` calls the VMA's `vm_ops->…`update_mmu_cache()` flushes the TLB entry.`kswapd` and `vmscan` run asynchronously to reclaim pag…
EDGE
OOM (Out-of-Memory)When both physical memory and swap are exhausted, `out_of_memory()` invokes the
Huge page splintingA 2 MiB THP can be split back to 4 KiB pages under memory pressure; the split op
Stack overflowThe stack VMA grows down; if a process exhausts its `RLIMIT_STACK`, the page fau
`mmap_lock` contentionThe `mmap_lock` (formerly `mmap_sem`) is a per-process read-write semaphore; und
1.10
VFS (Virtual Filesystem Switch) Core Layer
The VFS is the kernel's filesystem-independent abstraction layer.
▾
Definition
The VFS is the kernel's filesystem-independent abstraction layer. It defines generic inode, dentry, file, and superblock objects with operation tables (inode_operations, file_operations, super_operations) that concrete filesystems implement. This allows uniform syscall handling (open, read, write, stat) regardless of the underlying filesystem type.
Execution Flow
- A syscall like
open(path, flags) → do_sys_open() → do_filp_open() enters the VFS.
path_openat() performs pathname resolution via walk_component() and lookup_fast() / lookup_slow() against the dentry cache.
- On cache miss, the filesystem's
inode_operations->lookup() is called to load the dentry from storage.
vfs_open() checks permissions via inode_permission() (which invokes LSM hooks).
- A
struct file is allocated and f->f_op is set from inode->i_fop.
f->f_op->open(inode, file) is called (filesystem-specific open handler).
- A file descriptor is allocated and placed in the process's
fdtable.
- Subsequent
read(fd) / write(fd) calls are dispatched through f->f_op->read_iter / write_iter.
Edge Cases & Failure Modes
- Dentry cache poisoning: If a filesystem returns a negative dentry for a path, cached "file not found" answers can persist even after the file is created on-disk (requires a
d_invalidate() call).
O_NOFOLLOW + symlink race (TOCTOU): Between the permission check and the open, a symlink can be replaced; O_PATH and openat2(RESOLVE_NO_SYMLINKS) mitigate this.
- File descriptor table exhaustion: Opening files without closing them exhausts the per-process
RLIMIT_NOFILE limit; the open() call returns EMFILE.
- Stale NFS handles: For NFS-backed filesystems, a dentry can reference a file that has been deleted on the server; operations on it return
ESTALE, which must be handled by the application.
DEF
The VFS is the kernel's filesystem-independent abstraction layer.
FLOW
A syscall like `open(path, flags)` → `do_sys_open()` → …`path_openat()` performs pathname resolution via `walk_…On cache miss, the filesystem's `inode_operations->look…`vfs_open()` checks permissions via `inode_permission()…A `struct file` is allocated and `f->f_op` is set from …`f->f_op->open(inode, file)` is called (filesystem-spec…A file descriptor is allocated and placed in the proces…Subsequent `read(fd)` / `write(fd)` calls are dispatche…
EDGE
Dentry cache poisoningIf a filesystem returns a negative dentry for a path, cached "file not found" an
`O_NOFOLLOW` + symlink race (TOCTOU)Between the permission check and the open, a symlink can be replaced; `O_PATH` a
File descriptor table exhaustionOpening files without closing them exhausts the per-process `RLIMIT_NOFILE` limi
Stale NFS handlesFor NFS-backed filesystems, a dentry can reference a file that has been deleted
🔒
S2 — Synchronization Primitives
10 components
2.1
Spinlock (`arch/x86/include/asm/spinlock.h`, `kernel/locking/spinlock.c`)
A spinlock is a busy-wait mutual exclusion primitive for use in contexts where sleeping is prohibited (interrupt hand…
▾
Definition
A spinlock is a busy-wait mutual exclusion primitive for use in contexts where sleeping is prohibited (interrupt handlers, softirqs, NMI handlers). The lock holder holds the CPU in a tight loop until the lock is released. On SMP systems, it uses an atomic exchange or compare-and-swap instruction; on x86, the queued spinlock (qspinlock) uses a ticket-based protocol to ensure fairness.
Execution Flow
spin_lock(&lk) calls raw_spin_lock().
- On x86 with
CONFIG_QUEUED_SPINLOCKS: the lock byte is tested; if zero, an atomic CAS sets it to the current CPU's ticket.
- If locked: the CPU enters a
PAUSE-instruction loop, reading the lock byte until it matches the CPU's ticket (MCS/qspinlock queue position).
- Once acquired, preemption is implicitly disabled (or explicitly via
spin_lock_irq() which also disables local IRQs).
- The critical section executes.
spin_unlock(&lk) atomically increments the lock's serving counter, releasing the next waiter in the queue.
Edge Cases & Failure Modes
- Deadlock on same CPU: Attempting to acquire a spinlock already held on the same CPU (without
spin_trylock) causes an infinite loop; CONFIG_DEBUG_SPINLOCK detects this.
- Lock held across sleep: If code holding a spinlock calls any function that may sleep (e.g.,
kmalloc(GFP_KERNEL)), it triggers a might_sleep() BUG if CONFIG_DEBUG_ATOMIC_SLEEP is set.
- Interrupt nesting: Failing to use
spin_lock_irqsave() in interrupt-context code can allow IRQ re-entry that tries to re-acquire the same spinlock, causing a deadlock.
- Cache line bouncing: Under high contention on NUMA systems, the lock byte bounces between CPU caches, dramatically reducing throughput; use per-CPU structures or fine-grained locking to avoid.
DEF
A spinlock is a busy-wait mutual exclusion primitive for use in contexts where sleeping is prohibited (interrupt hand…
FLOW
`spin_lock(&lk)` calls `raw_spin_lock()`.On x86 with `CONFIG_QUEUED_SPINLOCKS`: the lock byte is…If locked: the CPU enters a `PAUSE`-instruction loop, r…Once acquired, preemption is implicitly disabled (or ex…The critical section executes.`spin_unlock(&lk)` atomically increments the lock's ser…
EDGE
Deadlock on same CPUAttempting to acquire a spinlock already held on the same CPU (without `spin_try
Lock held across sleepIf code holding a spinlock calls any function that may sleep (e.g., `kmalloc(GFP
Interrupt nestingFailing to use `spin_lock_irqsave()` in interrupt-context code can allow IRQ re-
Cache line bouncingUnder high contention on NUMA systems, the lock byte bounces between CPU caches,
2.2
Mutex (`kernel/locking/mutex.c`)
A mutex (MUTual EXclusion) is a sleeping lock for process context.
▾
Definition
A mutex (MUTual EXclusion) is a sleeping lock for process context. Unlike spinlocks, a mutex allows the blocked thread to sleep, releasing the CPU. It enforces strict ownership: only the acquiring task may release it. It uses an atomic_long_t state combined with an MCS-node-based wait queue for fairness.
Execution Flow
mutex_lock(&m) calls __mutex_lock().
- An optimistic fast-path: atomic CAS on
m->owner; if the mutex is unlocked (owner = 0), it is set to the current task pointer, and the lock is acquired immediately.
- Mid-path (OSQ — Optimistic Spinning Queue): if the owner is running on another CPU, the task spins on an MCS node for a brief window (
mutex_optimistic_spin()), avoiding a full sleep if the lock is released quickly.
- Slow-path: the task is added to
m->wait_list, sets its state to TASK_UNINTERRUPTIBLE (or TASK_INTERRUPTIBLE for mutex_lock_interruptible()), and calls schedule().
- When the current owner calls
mutex_unlock(), it clears m->owner and wakes the first waiter via wake_up_process().
- The woken task re-attempts the fast-path; on success, it clears TASK_UNINTERRUPTIBLE and continues.
Edge Cases & Failure Modes
- Priority inversion: A low-priority task holding a mutex blocks a high-priority task; the kernel's PI-mutex (
mutex_lock_pi() via rt_mutex) implements priority inheritance to mitigate this.
- Self-deadlock detection:
CONFIG_DEBUG_MUTEXES enables lock validation; a task acquiring a mutex it already holds triggers a BUG.
mutex_lock_interruptible() early return: If a signal arrives while waiting, the call returns -EINTR; the caller must handle the unacquired-lock case.
- Livelock (OSQ): In degenerate cases, the optimistic spinning phase can consume excessive CPU on a system with many threads contending the same mutex; the kernel caps the spin duration with
need_resched() checks.
DEF
A mutex (MUTual EXclusion) is a sleeping lock for process context.
FLOW
`mutex_lock(&m)` calls `__mutex_lock()`.An optimistic fast-path: atomic CAS on `m->owner`; if t…, it is set to the current task pointer, and the lock i…Mid-path (OSQ — Optimistic Spinning Queue): if the owne…Slow-path: the task is added to `m->wait_list`, sets it…When the current owner calls `mutex_unlock()`, it clear…The woken task re-attempts the fast-path; on success, i…
EDGE
Priority inversionA low-priority task holding a mutex blocks a high-priority task; the kernel's PI
Self-deadlock detection`CONFIG_DEBUG_MUTEXES` enables lock validation; a task acquiring a mutex it alre
`mutex_lock_interruptible()` early returnIf a signal arrives while waiting, the call returns `-EINTR`; the caller must ha
Livelock (OSQ)In degenerate cases, the optimistic spinning phase can consume excessive CPU on
2.3
Read-Write Semaphore (`kernel/locking/rwsem.c`)
An `rwsem` allows multiple concurrent readers or a single exclusive writer.
▾
Definition
An rwsem allows multiple concurrent readers or a single exclusive writer. It is a sleeping lock (readers and writers may block). Used heavily for the mmap_lock, i_rwsem (inode), and s_umount (superblock) locks. Modern kernels use rwsem with an OSQ (Optimistic Spin Queue) for writers.
Execution Flow
- Read lock:
down_read(&sem) → fast-path: atomically increments the reader count field if no writer is active or waiting.
- If a writer is pending, the reader enqueues to
wait_list with RWSEM_WAITING_FOR_READ state and calls schedule().
- Write lock:
down_write(&sem) → sets the RWSEM_WRITER_LOCKED bit via atomic CAS.
- If readers are active, the writer OSQ-spins briefly, then enqueues with
RWSEM_WAITING_FOR_WRITE.
- The writer sleeps until all active readers call
up_read().
- Last active reader calling
up_read() checks for waiting writers; if any, wakes the first writer.
- When the writer calls
up_write(), it clears the writer-locked bit; if readers are queued, it wakes them all simultaneously (batch wake-up).
Edge Cases & Failure Modes
- Writer starvation: A continuous stream of readers can prevent a writer from ever acquiring the lock; the kernel's
RWSEM_FLAG_WAITERS bit causes new readers to queue once a writer is waiting.
- Downgrade race:
downgrade_write() converts a held write lock to a read lock; if done incorrectly (e.g., while in an interrupt-disabled section), it can release the lock to a waiting writer prematurely.
mmap_lock scale issue: The mmap_lock is an rwsem; on heavily multithreaded workloads (e.g., many concurrent page faults), it becomes a scalability cliff. The kernel is actively replacing it with range-locked alternatives in long-term development.
DEF
An `rwsem` allows multiple concurrent readers or a single exclusive writer.
FLOW
**Read lock**: `down_read(&sem)` → fast-path: atomicall…If a writer is pending, the reader enqueues to `wait_li…**Write lock**: `down_write(&sem)` → sets the `RWSEM_WR…If readers are active, the writer OSQ-spins briefly, th…The writer sleeps until all active readers call `up_rea…Last active reader calling `up_read()` checks for waiti…When the writer calls `up_write()`, it clears the write…
EDGE
Writer starvationA continuous stream of readers can prevent a writer from ever acquiring the lock
Downgrade race`downgrade_write()` converts a held write lock to a read lock; if done incorrect
`mmap_lock` scale issueThe `mmap_lock` is an `rwsem`; on heavily multithreaded workloads (e.g., many co
2.4
RCU (Read-Copy-Update) (`kernel/rcu/`)
RCU is a synchronization mechanism that allows multiple readers to access shared data concurrently with writers, with…
▾
Definition
RCU is a synchronization mechanism that allows multiple readers to access shared data concurrently with writers, without locking on the read side. Readers are protected by rcu_read_lock() / rcu_read_unlock(), which are essentially preemption-disable markers. Writers update data by creating a new copy, replacing the pointer atomically, and then waiting for a "grace period" (all pre-existing readers to complete) before freeing the old version.
Execution Flow
- Reader:
rcu_read_lock() disables preemption (or marks an RCU read-side critical section in PREEMPT_RCU).
- Reader dereferences an RCU-protected pointer via
rcu_dereference(ptr) (which includes a data dependency barrier / smp_read_barrier_depends()).
- Reader reads the data;
rcu_read_unlock() re-enables preemption.
- Writer: allocates a new version of the data, copies and modifies it, then calls
rcu_assign_pointer(ptr, new_data) (which includes a write memory barrier).
- Writer calls
synchronize_rcu() (or call_rcu(&old_data->rcu_head, free_fn) for async).
synchronize_rcu() waits for a grace period: it blocks until all currently-executing RCU read-side critical sections on all CPUs complete.
- After the grace period, the old data is safe to free.
Edge Cases & Failure Modes
- Grace period stall: A CPU stuck in a long RCU read-side critical section (e.g., with preemption disabled for too long) delays
synchronize_rcu() indefinitely; the RCU watchdog timer fires and prints a stall warning.
rcu_dereference() outside read-side: Dereferencing an RCU pointer outside rcu_read_lock() can read a pointer to freed memory; CONFIG_PROVE_RCU detects this.
- Expedited RCU side effects:
synchronize_rcu_expedited() forcibly preempts CPUs to accelerate the grace period; this increases interrupt latency and should not be used in hot paths.
- SRCU (Sleepable RCU) deadlock: Unlike regular RCU,
srcu_read_lock() can sleep; calling blocking operations inside an SRCU critical section is legal, but holding an SRCU lock and then calling synchronize_srcu() on the same srcu_struct deadlocks.
DEF
RCU is a synchronization mechanism that allows multiple readers to access shared data concurrently with writers, with…
FLOW
Reader: `rcu_read_lock()` disables preemption (or marks…Reader dereferences an RCU-protected pointer via `rcu_d…Reader reads the data; `rcu_read_unlock()` re-enables p…Writer: allocates a new version of the data, copies and…Writer calls `synchronize_rcu()` (or `call_rcu(&old_dat…`synchronize_rcu()` waits for a grace period: it blocks…After the grace period, the old data is safe to free.
EDGE
Grace period stallA CPU stuck in a long RCU read-side critical section (e.g., with preemption disa
`rcu_dereference()` outside read-sideDereferencing an RCU pointer outside `rcu_read_lock()` can read a pointer to fre
Expedited RCU side effects`synchronize_rcu_expedited()` forcibly preempts CPUs to accelerate the grace per
SRCU (Sleepable RCU) deadlockUnlike regular RCU, `srcu_read_lock()` can sleep; calling blocking operations in
2.5
Seqlock (`include/linux/seqlock.h`)
A seqlock (sequence lock) is an optimistic concurrency primitive designed for data that is written rarely but read ve…
▾
Definition
A seqlock (sequence lock) is an optimistic concurrency primitive designed for data that is written rarely but read very frequently, and where the reader can tolerate retrying. Writers increment a sequence counter before and after the update; readers check the counter before and after reading — if the counter changed (or is odd, indicating a writer is active), the read is retried.
Execution Flow
- Reader: calls
read_seqbegin(&sl) → reads and returns the current sequence counter value seq.
- Reader reads the shared data (non-atomically, without locking).
- Reader calls
read_seqretry(&sl, seq) → reads the counter again; if it differs from seq or is odd (write in progress), returns true → reader retries from step 1.
- Writer: calls
write_seqlock(&sl) → acquires the embedded spinlock, then atomically increments the sequence counter (making it odd).
- Writer modifies the shared data.
- Writer calls
write_sequnlock(&sl) → atomically increments the counter again (making it even), then releases the spinlock.
Edge Cases & Failure Modes
- Livelock: Under a very high writer rate, readers may never see a stable (even) sequence value and loop indefinitely; this is by design but pathological in practice.
- Non-retryable operations in reader: If a reader performs side-effecting operations (e.g., memory allocation) inside the seqlock read window and must retry, those effects are not undone; seqlocks must only wrap pure reads.
- Large critical sections: The kernel's
jiffies and timekeeper use seqlocks; if the writer (clock interrupt handler) holds the lock for too long, readers on other CPUs spin repeatedly.
- Pointer dereference hazard: Reading a pointer under a seqlock and then dereferencing it after the read window is unsafe; if the writer freed the pointed-to object between the read and the dereference, a use-after-free results.
DEF
A seqlock (sequence lock) is an optimistic concurrency primitive designed for data that is written rarely but read ve…
FLOW
Reader: calls `read_seqbegin(&sl)` → reads and returns …Reader reads the shared data (non-atomically, without l…Reader calls `read_seqretry(&sl, seq)` → reads the coun…4. Writer: calls `write_seqlock(&sl)` → acquires the em…Writer modifies the shared data.Writer calls `write_sequnlock(&sl)` → atomically increm…
EDGE
LivelockUnder a very high writer rate, readers may never see a stable (even) sequence va
Non-retryable operations in readerIf a reader performs side-effecting operations (e.g., memory allocation) inside
Large critical sectionsThe kernel's `jiffies` and `timekeeper` use seqlocks; if the writer (clock inter
Pointer dereference hazardReading a pointer under a seqlock and then dereferencing it after the read windo
2.6
Completion (`kernel/sched/completion.c`)
A completion is a one-shot or multi-shot synchronization object that allows one task to wait for another to signal "d…
▾
Definition
A completion is a one-shot or multi-shot synchronization object that allows one task to wait for another to signal "done." It is conceptually a simplified semaphore initialized to zero, used to synchronize between a producer task (which calls complete()) and a consumer task (which calls wait_for_completion()). Used extensively in driver probe/unprobe sequences and kernel thread lifecycle management.
Execution Flow
- A completion is initialized:
init_completion(&done) → sets done.done = 0, done.wait initialized.
- The waiting task calls
wait_for_completion(&done):
- Atomically checks
done.done; if non-zero, decrements and returns immediately.
- Otherwise, creates a
wait_queue_entry, sets the task state to TASK_UNINTERRUPTIBLE, and adds itself to done.wait.
- Calls
schedule(); the task sleeps.
- The signaling task calls
complete(&done):
- Increments
done.done.
- Calls
wake_up_process() on the first waiter in done.wait.
- The sleeping task resumes from
schedule(), re-checks done.done, and returns.
- For broadcast wakeup:
complete_all(&done) sets done.done = UINT_MAX / 2 and wakes all waiters.
Edge Cases & Failure Modes
complete() before wait_for_completion(): This is safe; the done counter records the signal and the later wait_for_completion() returns immediately.
complete() without a waiter: The done counter accumulates; if wait_for_completion() is never called, the completion leaks but causes no crash (unless the memory is freed).
wait_for_completion_killable() early return: A fatal signal causes the function to return -ERESTARTSYS; the caller must handle the case where the waited-for event has not occurred.
- Re-initialization race: Calling
init_completion() while another task is blocked in wait_for_completion() corrupts the wait queue; completions must not be re-initialized while in use.
DEF
A completion is a one-shot or multi-shot synchronization object that allows one task to wait for another to signal "d…
FLOW
A completion is initialized: `init_completion(&done)` →…The waiting task calls `wait_for_completion(&done)`:The signaling task calls `complete(&done)`:The sleeping task resumes from `schedule()`, re-checks …For broadcast wakeup: `complete_all(&done)` sets `done.…
EDGE
`complete()` before `wait_for_completion()`This is safe; the `done` counter records the signal and the later `wait_for_comp
`complete()` without a waiterThe `done` counter accumulates; if `wait_for_completion()` is never called, the
`wait_for_completion_killable()` early returnA fatal signal causes the function to return `-ERESTARTSYS`; the caller must han
Re-initialization raceCalling `init_completion()` while another task is blocked in `wait_for_completio
2.7
Atomic Operations (`arch/x86/include/asm/atomic.h`)
Atomic operations provide CPU-guaranteed read-modify-write operations on integer values (`atomic_t`, `atomic64_t`, `a…
▾
Definition
Atomic operations provide CPU-guaranteed read-modify-write operations on integer values (atomic_t, atomic64_t, atomic_long_t) without requiring locks. They map to CPU-native instructions: on x86, LOCK ADD, LOCK CMPXCHG, XCHG; on ARM64, LDADD, CAS (ARMv8.1+). They serve as the foundation for reference counting, lock implementation, and wait-free counters.
Execution Flow
- A variable is declared:
atomic_t refcount = ATOMIC_INIT(1).
atomic_inc(&refcount) compiles to a LOCK ADDL $1, (mem) on x86 — the LOCK prefix causes the CPU to assert the bus lock (or use cache coherency) guaranteeing the increment is indivisible.
atomic_dec_and_test(&refcount) compiles to LOCK DECL (mem) followed by reading the zero flag — returns true if the result is zero.
atomic_cmpxchg(&v, old, new) → LOCK CMPXCHG — atomically compares v to old; if equal, stores new and returns old; otherwise returns the current value.
- Memory ordering variants:
atomic_read() is a plain load with a compiler barrier; smp_load_acquire() / smp_store_release() add CPU-level acquire/release semantics.
Edge Cases & Failure Modes
- ABA problem:
atomic_cmpxchg() succeeds even if the value was changed to A, then back to A between the load and the CAS; use tagged pointers or DCAS where this matters.
- Overflow/underflow:
atomic_t is a 32-bit counter; overflow wraps silently. Use refcount_t (which uses CONFIG_REFCOUNT_FULL protection) for reference counts to catch accidental underflow to zero.
refcount_t saturation: The kernel's refcount_t saturates at INT_MAX to prevent integer overflow bugs from causing use-after-free; operations on a saturated counter warn but do not crash.
- Non-atomic accesses on 64-bit counters (32-bit CPUs): On 32-bit kernels,
atomic64_t operations require a lock (emulated) because 64-bit memory accesses are not natively atomic; this is a performance cliff.
DEF
Atomic operations provide CPU-guaranteed read-modify-write operations on integer values (`atomic_t`, `atomic64_t`, `a…
FLOW
A variable is declared: `atomic_t refcount = ATOMIC_INI…`.`atomic_inc(&refcount)` compiles to a `LOCK ADDL $1, (m…`atomic_dec_and_test(&refcount)` compiles to `LOCK DECL…`atomic_cmpxchg(&v, old, new)` → `LOCK CMPXCHG` — atomi…Memory ordering variants: `atomic_read()` is a plain lo…
EDGE
ABA problem`atomic_cmpxchg()` succeeds even if the value was changed to `A`, then back to `
Overflow/underflow`atomic_t` is a 32-bit counter; overflow wraps silently. Use `refcount_t` (which
`refcount_t` saturationThe kernel's `refcount_t` saturates at INT_MAX to prevent integer overflow bugs
Non-atomic accesses on 64-bit counters (32-bit CPUs)On 32-bit kernels, `atomic64_t` operations require a lock (emulated) because 64-
2.8
Per-CPU Variables (`include/linux/percpu.h`, `mm/percpu.c`)
Per-CPU variables are instances of data duplicated once per logical CPU, stored in CPU-local memory regions, and acce…
▾
Definition
Per-CPU variables are instances of data duplicated once per logical CPU, stored in CPU-local memory regions, and accessed locklessly because each CPU always accesses only its own copy. They eliminate cache line bouncing on frequently updated statistics counters, scheduler run queues, and allocator caches (SLUB cpu_slab).
Execution Flow
- A per-CPU variable is declared:
DEFINE_PER_CPU(unsigned long, mycounter).
- At boot,
setup_per_cpu_areas() allocates a contiguous memory region for each CPU, copies the initial values, and sets per-CPU base pointers.
- Access:
get_cpu_var(mycounter) disables preemption and returns a reference to the current CPU's copy via this_cpu_ptr(&mycounter) (which uses the %gs or TPIDR_EL1 base register offset on x86/ARM64).
- The variable is modified.
put_cpu_var(mycounter) re-enables preemption.
- For lockless access (safe only for statistics):
this_cpu_add(mycounter, 1) uses a LOCK-free ADD via the this_cpu_* family of operations, which rely on the fact that only the local CPU writes this copy.
Edge Cases & Failure Modes
- Preemption not disabled: If preemption is enabled when
this_cpu_ptr() is used (not inside get_cpu_var()), the task can be migrated to another CPU between reading the pointer and dereferencing it, accessing the wrong CPU's copy.
- Interrupt context access: An interrupt handler can preempt code using
this_cpu_ptr() on the same CPU; both access the same per-CPU variable without coordination, causing a data race if the interrupt handler also modifies it.
- Dynamic per-CPU allocation (
alloc_percpu()): The returned pointer must be freed with free_percpu(); a leak causes the per-CPU allocator's chunk pool to grow without bound.
- Hotplug CPU offline: When a CPU goes offline, its per-CPU data is not automatically migrated; subsystems must flush/drain per-CPU data in their
CPU_DEAD notifier.
DEF
Per-CPU variables are instances of data duplicated once per logical CPU, stored in CPU-local memory regions, and acce…
FLOW
A per-CPU variable is declared: `DEFINE_PER_CPU(unsigne…At boot, `setup_per_cpu_areas()` allocates a contiguous…Access: `get_cpu_var(mycounter)` disables preemption an….The variable is modified.`put_cpu_var(mycounter)` re-enables preemption.For lockless access (safe only for statistics): `this_c…` uses a `LOCK`-free `ADD` via the `this_cpu_*` family …
EDGE
Preemption not disabledIf preemption is enabled when `this_cpu_ptr()` is used (not inside `get_cpu_var(
Interrupt context accessAn interrupt handler can preempt code using `this_cpu_ptr()` on the same CPU; bo
Dynamic per-CPU allocation (`alloc_percpu()`)The returned pointer must be freed with `free_percpu()`; a leak causes the per-C
Hotplug CPU offlineWhen a CPU goes offline, its per-CPU data is not automatically migrated; subsyst
2.9
Futex (`kernel/futex/`)
A futex (Fast Userspace muTEX) is a hybrid synchronization primitive: the fast path is a userspace atomic operation o…
▾
Definition
A futex (Fast Userspace muTEX) is a hybrid synchronization primitive: the fast path is a userspace atomic operation on a shared integer (no syscall required); only on contention does the thread make a futex(2) syscall to block in the kernel. The kernel maintains a hash table of futex_q (futex queue) entries keyed by the physical address of the futex word.
Execution Flow
- Lock attempt: userspace atomically tries CAS on the futex word (
uaddr); if it succeeds (word was 0, now 1), the lock is acquired without entering the kernel.
- On contention (word was already 1): the thread sets a "has waiters" bit (word = 2) and calls
futex(FUTEX_WAIT, uaddr, expected_val, timeout).
- Kernel:
do_futex() → futex_wait() → computes the hash key (futex_key) from the futex word's physical page + offset.
futex_wait_setup() verifies the futex value still equals expected_val; if it changed (racing unlock), returns immediately with EAGAIN.
- The task is added to
hb->chain (hash bucket wait queue) and calls schedule_hrtimeout().
- Unlock: the unlocking thread atomically clears the word; if "has waiters" bit was set, calls
futex(FUTEX_WAKE, uaddr, count).
- Kernel:
futex_wake() removes up to count waiters from the hash bucket and wakes them via wake_up_q().
Edge Cases & Failure Modes
- Priority inversion: A low-priority holder blocks a high-priority waiter;
FUTEX_LOCK_PI / FUTEX_UNLOCK_PI implement priority inheritance via RT-mutexes to prevent this.
- Spurious wakeup: A signal delivery wakes the futex waiter before the futex word changes; userspace must re-check the futex word in a loop.
FUTEX_WAIT / value race: Between the userspace CAS failure and the kernel futex_wait_setup() check, another thread may have already unlocked; the check catches this and returns EAGAIN, preventing a missed wakeup.
- Robust futex list corruption: If a process is killed while holding a robust futex, the kernel walks the robust list (
set_robust_list(2)) to wake waiters with FUTEX_OWNER_DIED; a corrupt list pointer causes a kernel read of a bad address, triggering a fault in handle_futex_death().
DEF
A futex (Fast Userspace muTEX) is a hybrid synchronization primitive: the fast path is a userspace atomic operation o…
FLOW
Lock attempt: userspace atomically tries CAS on the fut…, the lock is acquired without entering the kernel.On contention (word was already: the thread sets a "has waiters" bit (word =and calls `futex(FUTEX_WAIT, uaddr, expected_val, timeo…Kernel: `do_futex()` → `futex_wait()` → computes the ha…`futex_wait_setup()` verifies the futex value still equ…The task is added to `hb->chain` (hash bucket wait queu…Unlock: the unlocking thread atomically clears the word…Kernel: `futex_wake()` removes up to `count` waiters fr…
EDGE
Priority inversionA low-priority holder blocks a high-priority waiter; `FUTEX_LOCK_PI` / `FUTEX_UN
Spurious wakeupA signal delivery wakes the futex waiter before the futex word changes; userspac
`FUTEX_WAIT` / value raceBetween the userspace CAS failure and the kernel `futex_wait_setup()` check, ano
Robust futex list corruptionIf a process is killed while holding a robust futex, the kernel walks the robust
2.10
Semaphore (`kernel/locking/semaphore.c`)
A counting semaphore (`struct semaphore`) allows a specified number of concurrent holders, unlike a mutex (which allo…
▾
Definition
A counting semaphore (struct semaphore) allows a specified number of concurrent holders, unlike a mutex (which allows exactly one). It is a sleeping primitive: down() decrements the count and sleeps if the result is negative; up() increments and wakes a waiter. Used for resource pools (e.g., limiting concurrent hardware accesses).
Execution Flow
- Initialization:
sema_init(&sem, N) sets sem.count = N and initializes sem.wait_list.
down(&sem) → __down():
- Atomically decrements
sem.count.
- If result ≥ 0, returns immediately (resource acquired).
- If result < 0, adds the current task to
sem.wait_list with state TASK_UNINTERRUPTIBLE, calls schedule().
up(&sem) → __up():
- Atomically increments
sem.count.
- If
wait_list is non-empty, calls wake_up_process() on the first waiter.
- The woken waiter re-runs from
schedule() return, verifies it was woken by an up() (not spuriously), and returns.
Edge Cases & Failure Modes
- Semaphore released without being acquired: Calling
up() more times than down() raises the counter above its initial value, allowing more concurrent holders than intended; the kernel does not check for this.
down_interruptible() early return: Returns -EINTR on signal; the caller must handle the case where the resource was not acquired, to avoid resource leaks.
down_trylock() busy-wait abuse: Using down_trylock() in a tight loop is equivalent to spinning and wastes CPU; the caller should use the blocking down() or a semaphore-appropriate data structure.
- Priority inversion (no inheritance): Unlike
rt_mutex-based locks, the kernel semaphore has no priority inheritance; real-time tasks blocked on a semaphore held by a low-priority task may miss their deadlines.
DEF
A counting semaphore (`struct semaphore`) allows a specified number of concurrent holders, unlike a mutex (which allo…
FLOW
Initialization: `sema_init(&sem, N)` sets `sem.count = …`down(&sem)` → `__down()`:`up(&sem)` → `__up()`:The woken waiter re-runs from `schedule()` return, veri…
EDGE
Semaphore released without being acquiredCalling `up()` more times than `down()` raises the counter above its initial val
`down_interruptible()` early returnReturns `-EINTR` on signal; the caller must handle the case where the resource w
`down_trylock()` busy-wait abuseUsing `down_trylock()` in a tight loop is equivalent to spinning and wastes CPU;
Priority inversion (no inheritance)Unlike `rt_mutex`-based locks, the kernel `semaphore` has no priority inheritanc
🧠
S3 — Memory Allocators
10 components
3.1
Buddy Allocator (`mm/page_alloc.c`)
The buddy allocator is the kernel's physical page frame manager.
▾
Definition
The buddy allocator is the kernel's physical page frame manager. It organizes free pages into 11 lists of power-of-two blocks (order 0 = 1 page to order 10 = 1024 pages = 4 MiB). When a block is freed, the allocator checks whether its "buddy" (the adjacent block of the same size) is also free; if so, the two are merged into a block of double the size (recursively up to order 10).
Execution Flow
alloc_pages(gfp_flags, order) → __alloc_pages().
get_page_from_freelist() iterates memory zones (ZONE_DMA32, ZONE_NORMAL, ZONE_HIGHMEM) in preference order.
- For the target zone,
rmqueue() checks the free list at the requested order.
- If a block of that order is available, it is removed from the free list, split if the allocation is for a lower order (the remainder is added back to the appropriate free list as a buddy), and returned.
- If no block at the requested order is available,
rmqueue() tries the next higher order and splits.
- If all higher orders are also empty: watermark check fails,
wake_up_kswapd() is called to reclaim memory, and __alloc_pages_slowpath() tries OOM killer / migration.
- Physical page address returned as
struct page *; virtual mapping via page_to_virt().
Edge Cases & Failure Modes
- External fragmentation: Over time, free memory becomes fragmented into non-contiguous blocks; large-order allocations fail even when total free memory is sufficient.
CONFIG_COMPACTION and kcompactd migrate movable pages to coalesce free space.
- Zone imbalance: If
ZONE_DMA32 is exhausted but ZONE_NORMAL has free pages, allocations requiring GFP_DMA32 fail; devices that require DMA below 4 GiB report errors.
- Atomic allocation failures (
GFP_ATOMIC): Atomic allocations cannot sleep or reclaim; they draw from a reserved "emergency" pool (min_free_kbytes). If that pool is exhausted, the allocation fails and the caller must handle NULL.
- High-order allocation under pressure: A request for order-10 (4 MiB) pages may fail even with gigabytes of free memory if no contiguous region is available; drivers should use scatter-gather DMA rather than requiring contiguous buffers.
DEF
The buddy allocator is the kernel's physical page frame manager.
FLOW
`alloc_pages(gfp_flags, order)` → `__alloc_pages()`.`get_page_from_freelist()` iterates memory zones (ZONE_…For the target zone, `rmqueue()` checks the free list a…If a block of that order is available, it is removed fr…If no block at the requested order is available, `rmque…If all higher orders are also empty: watermark check fa…Physical page address returned as `struct page *`; virt…
EDGE
External fragmentationOver time, free memory becomes fragmented into non-contiguous blocks; large-orde
Zone imbalanceIf `ZONE_DMA32` is exhausted but `ZONE_NORMAL` has free pages, allocations requi
Atomic allocation failures (`GFP_ATOMIC`)Atomic allocations cannot sleep or reclaim; they draw from a reserved "emergency
High-order allocation under pressureA request for order-10 (4 MiB) pages may fail even with gigabytes of free memory
3.2
SLAB Allocator (`mm/slab.c`)
The SLAB allocator (Bonwick 1994, Linux port ~2.0) manages pools of fixed-size kernel objects.
▾
Definition
The SLAB allocator (Bonwick 1994, Linux port ~2.0) manages pools of fixed-size kernel objects. Each pool (kmem_cache) holds multiple slabs (pages containing a homogeneous array of objects). Freed objects are cached rather than immediately returned to the buddy allocator, amortizing initialization cost. SLAB is the original allocator; most modern kernels use SLUB by default.
Execution Flow
kmem_cache_create("name", object_size, align, flags, ctor) creates a new cache; calculates optimal objects-per-slab and slab order.
kmem_cache_alloc(cache, gfp) → checks the per-CPU cache (array_cache); if non-empty, returns the top-of-stack object (fast path, no locking).
- If per-CPU cache is empty:
cache_alloc_refill() fills it from the shared array (requires cache_lock); if shared array is empty, cache_grow_begin() allocates new pages from the buddy allocator and divides them into objects.
- Object initialization: if a constructor (
ctor) was provided, it is called on first use.
kmem_cache_free(cache, obj) returns the object to the per-CPU cache top-of-stack.
- When the per-CPU cache overflows its
batchcount, objects are moved to the shared array; when that overflows, full slabs are returned to the buddy allocator.
Edge Cases & Failure Modes
- Slab poisoning detection: With
CONFIG_DEBUG_SLAB, freed objects are filled with 0x6b ("k"); a later allocation that reads a non-poisoned value indicates a use-after-free.
- Cache aliasing: Multiple
kmem_cache_create() calls with the same size may or may not share an underlying slab pool depending on CONFIG_SLAB_MERGE_DEFAULT; unintended sharing can lead to type confusion bugs.
- Per-CPU cache under/overflow in hot-unplug: When a CPU goes offline while its per-CPU cache holds objects,
free_block() must flush those objects to the shared list; failure to do so leaks memory.
- Constructor called on reuse (SLAB only): SLAB's constructor is called only at slab creation, not on each allocation; a constructor that initializes lock state may leave stale lock state if not re-initialized at alloc time.
DEF
The SLAB allocator (Bonwick 1994, Linux port ~2.0) manages pools of fixed-size kernel objects.
FLOW
`kmem_cache_create("name", object_size, align, flags, c…`kmem_cache_alloc(cache, gfp)` → checks the per-CPU cac…If per-CPU cache is empty: `cache_alloc_refill()` fills…Object initialization: if a constructor (`ctor`) was pr…`kmem_cache_free(cache, obj)` returns the object to the…When the per-CPU cache overflows its `batchcount`, obje…
EDGE
Slab poisoning detectionWith `CONFIG_DEBUG_SLAB`, freed objects are filled with `0x6b` ("k"); a later al
Cache aliasingMultiple `kmem_cache_create()` calls with the same size may or may not share an
Per-CPU cache under/overflow in hot-unplugWhen a CPU goes offline while its per-CPU cache holds objects, `free_block()` mu
Constructor called on reuse (SLAB only)SLAB's constructor is called only at slab creation, not on each allocation; a co
3.3
SLUB Allocator (`mm/slub.c`)
SLUB (SLab Unqueued Bookkeeping) is the default slab allocator since Linux 2.6.23.
▾
Definition
SLUB (SLab Unqueued Bookkeeping) is the default slab allocator since Linux 2.6.23. It reduces metadata overhead by storing bookkeeping in unused parts of objects themselves rather than in separate management arrays. It uses per-CPU "cpu_slab" pointers for lockless fast-path allocation and a per-node partial-slab list for the slow path.
Execution Flow
kmem_cache_alloc(s, gfp) → reads s->cpu_slab->freelist (locklessly, using this_cpu_ptr).
- Fast path: if
freelist is non-NULL, pops the first free object from the freelist (a pointer stored inside the object itself), returns it.
- Slow path (
__slab_alloc()): if the CPU slab is empty, tries the next partial slab on s->node[nid]->partial.
- If no partial slabs: allocates a new slab of pages from the buddy allocator, initializes the freelist chain, and assigns it as the new CPU slab.
kmem_cache_free(s, obj) → do_slab_free(): pushes obj onto the CPU slab's freelist (lockless CAS using this_cpu_cmpxchg_double).
- When the CPU slab becomes fully free, it is returned to the partial list or freed to the buddy allocator (based on
min_partial threshold).
Edge Cases & Failure Modes
- Double-free detection: SLUB stores a "freelist pointer" XOR'd with a secret in the object itself (
CONFIG_SLAB_FREELIST_HARDENED); a double-free corrupts the XOR value and is detected on the next allocation with a BUG_ON.
- Redzone overwrite:
CONFIG_SLUB_DEBUG adds redzones around objects; buffer overflows into the redzone are detected on free.
- Quarantine (KASAN): Kernel Address Sanitizer delays reuse of freed SLUB objects; a use-after-free that accesses a quarantined object triggers an immediate KASAN report.
- NUMAism: SLUB maintains per-node partial lists; on NUMA systems, allocating from a remote node's slab incurs a NUMA penalty; the
kmem_cache_alloc_node() API enforces local allocation.
DEF
SLUB (SLab Unqueued Bookkeeping) is the default slab allocator since Linux 2.6.23.
FLOW
`kmem_cache_alloc(s, gfp)` → reads `s->cpu_slab->freeli…Fast path: if `freelist` is non-NULL, pops the first fr…Slow path (`__slab_alloc()`): if the CPU slab is empty,…If no partial slabs: allocates a new slab of pages from…`kmem_cache_free(s, obj)` → `do_slab_free()`: pushes `o…When the CPU slab becomes fully free, it is returned to…
EDGE
Double-free detectionSLUB stores a "freelist pointer" XOR'd with a secret in the object itself (`CONF
Redzone overwrite`CONFIG_SLUB_DEBUG` adds redzones around objects; buffer overflows into the redz
Quarantine (KASAN)Kernel Address Sanitizer delays reuse of freed SLUB objects; a use-after-free th
NUMAismSLUB maintains per-node partial lists; on NUMA systems, allocating from a remote
3.4
SLOB Allocator (`mm/slob.c`)
SLOB (Simple List Of Blocks) is a compact allocator designed for systems with very limited memory (embedded, tiny IoT…
▾
Definition
SLOB (Simple List Of Blocks) is a compact allocator designed for systems with very limited memory (embedded, tiny IoT kernels). It uses three first-fit free lists for objects ≤ 256 B, ≤ 1024 B, and larger, and stores metadata inline as a small header per block. It has no per-CPU caches or object pools, trading performance for minimal code and metadata size.
Execution Flow
kmalloc(size, gfp) with SLOB selects one of three free lists by size bracket.
slob_alloc(size, gfp, align, node) acquires slob_lock (global spinlock) and walks the selected free list.
- First-fit: finds the first free block ≥ requested size.
- If the block is significantly larger, it is split: the remainder becomes a new free block inserted after the allocated portion.
- If no fit found:
slob_new_pages() allocates a new page from the buddy allocator, initializes it as a SLOB page, and retries.
kfree(obj) → slob_free() acquires slob_lock, inserts the block back into the appropriate free list in address order, and coalesces with adjacent free blocks.
Edge Cases & Failure Modes
- Global lock bottleneck: The single
slob_lock serializes all allocations and frees; SLOB is entirely unsuitable for SMP workloads above trivial scale.
- Fragmentation (no compaction): SLOB has no per-slab concept; fragmentation accumulates and is never compacted, leading to wasted memory over time.
- No NUMA awareness: SLOB is unaware of NUMA topology; all allocations come from the same free list regardless of NUMA node.
- Deprecation path: SLOB was removed from mainline Linux in kernel 6.4 (
CONFIG_SLOB removed); code relying on it must migrate to SLUB.
DEF
SLOB (Simple List Of Blocks) is a compact allocator designed for systems with very limited memory (embedded, tiny IoT…
FLOW
`kmalloc(size, gfp)` with SLOB selects one of three fre…`slob_alloc(size, gfp, align, node)` acquires `slob_loc…First-fit: finds the first free block ≥ requested size.If the block is significantly larger, it is split: the …If no fit found: `slob_new_pages()` allocates a new pag…`kfree(obj)` → `slob_free()` acquires `slob_lock`, inse…
EDGE
Global lock bottleneckThe single `slob_lock` serializes all allocations and frees; SLOB is entirely un
Fragmentation (no compaction)SLOB has no per-slab concept; fragmentation accumulates and is never compacted,
No NUMA awarenessSLOB is unaware of NUMA topology; all allocations come from the same free list r
Deprecation pathSLOB was removed from mainline Linux in kernel 6.4 (`CONFIG_SLOB` removed); code
3.5
`vmalloc` / `vmap` (`mm/vmalloc.c`)
`vmalloc()` allocates virtually contiguous but physically discontiguous memory by mapping individual (potentially non…
▾
Definition
vmalloc() allocates virtually contiguous but physically discontiguous memory by mapping individual (potentially non-contiguous) physical pages into a contiguous range in the kernel's vmalloc area (above VMALLOC_START). It is used when large allocations are needed but physically contiguous pages cannot be guaranteed.
Execution Flow
vmalloc(size) → __vmalloc_node().
- A
vm_struct is allocated (describes the virtual range) and inserted into the vmalloc Red-Black tree.
get_vm_area_caller() finds a gap in the vmalloc address space for the requested size (aligned to page boundaries).
- For each page in the allocation:
alloc_page(GFP_KERNEL) obtains a physical page (order 0) from the buddy allocator.
map_kernel_range() / vmap_pages_range() establishes page table entries mapping the virtual range to the allocated physical pages.
flush_cache_vmap() flushes TLB entries for the new mapping.
- Returns the virtual address of the start of the mapping.
vfree(addr) unmaps page tables, frees each physical page to the buddy allocator, and removes the vm_struct from the tree.
Edge Cases & Failure Modes
- vmalloc space exhaustion: The vmalloc area is finite (128 TiB on x86-64 by default); a leak of
vm_struct entries or large vmalloc allocations without vfree() can exhaust it.
- TLB shootdown cost: Each
vfree() requires an IPI to all CPUs to flush TLB entries for the freed range; this is expensive under high allocation/free rates and can cause latency spikes.
- Non-contiguous physical pages: Drivers that use
vmalloc()-allocated buffers for DMA must use scatter-gather lists; passing a vmalloc address directly to a DMA engine will cause silent corruption or a machine check.
vmap() on highmem pages (32-bit kernels): Mapping ZONE_HIGHMEM pages via vmap() is the standard way to access them on 32-bit kernels; kunmap_atomic() must be called promptly to avoid kmap slot exhaustion.
DEF
`vmalloc()` allocates virtually contiguous but physically discontiguous memory by mapping individual (potentially non…
FLOW
`vmalloc(size)` → `__vmalloc_node()`.A `vm_struct` is allocated (describes the virtual range…`get_vm_area_caller()` finds a gap in the vmalloc addre…For each page in the allocation: `alloc_page(GFP_KERNEL…from the buddy allocator.`map_kernel_range()` / `vmap_pages_range()` establishes…`flush_cache_vmap()` flushes TLB entries for the new ma…Returns the virtual address of the start of the mapping…`vfree(addr)` unmaps page tables, frees each physical p…
EDGE
vmalloc space exhaustionThe vmalloc area is finite (128 TiB on x86-64 by default); a leak of `vm_struct`
TLB shootdown costEach `vfree()` requires an IPI to all CPUs to flush TLB entries for the freed ra
Non-contiguous physical pagesDrivers that use `vmalloc()`-allocated buffers for DMA must use scatter-gather l
`vmap()` on highmem pages (32-bit kernels)Mapping `ZONE_HIGHMEM` pages via `vmap()` is the standard way to access them on
3.6
`kmalloc` / `kfree` (`include/linux/slab.h`)
`kmalloc(size, gfp_flags)` is the primary general-purpose kernel small-object allocator.
▾
Definition
kmalloc(size, gfp_flags) is the primary general-purpose kernel small-object allocator. It is a wrapper around the active slab allocator (SLUB by default) that selects a pre-existing kmem_cache whose object size is the smallest power-of-two ≥ size. The returned memory is physically and virtually contiguous (unlike vmalloc).
Execution Flow
kmalloc(size, gfp) → __kmalloc().
kmalloc_slab(size, gfp) looks up the appropriate kmem_cache from kmalloc_caches[type][index] where index = ilog2(roundup_pow_of_two(size)).
- If size is 0, returns
ZERO_SIZE_PTR (a non-NULL magic constant that is safe to pass to kfree()).
- If size >
KMALLOC_MAX_CACHE_SIZE (typically 8 KiB on 4 KiB page systems): falls through to kmalloc_large() → alloc_pages() for order-based allocation.
- Otherwise:
kmem_cache_alloc(cache, gfp) is called on the selected cache (SLUB fast path: per-CPU freelist pop).
kfree(ptr) → __kfree(): uses virt_to_head_page() to find the struct page → identifies the kmem_cache → calls kmem_cache_free().
Edge Cases & Failure Modes
- Size rounding waste: An allocation of 33 bytes uses the 64-byte cache, wasting 31 bytes; for performance-critical code with many such objects,
kmem_cache_create() with an exact size is preferable.
- GFP flag misuse: Using
GFP_KERNEL (may sleep/reclaim) in atomic context (IRQ handler, spinlock held) triggers a might_sleep() warning or, worse, deadlock if the allocator tries to reclaim memory.
kfree() on a stack pointer or non-kmalloc memory: SLUB's kfree() uses virt_to_head_page() to verify the pointer is slab-managed; a pointer to a stack variable or static data causes a BUG.
- Speculative failure under pressure: Even
GFP_KERNEL allocations can fail under extreme memory pressure if the OOM killer has already killed all candidates; callers must check for NULL and handle it gracefully.
DEF
`kmalloc(size, gfp_flags)` is the primary general-purpose kernel small-object allocator.
FLOW
`kmalloc(size, gfp)` → `__kmalloc()`.`kmalloc_slab(size, gfp)` looks up the appropriate `kme…If size is 0, returns `ZERO_SIZE_PTR` (a non-NULL magic…If size > `KMALLOC_MAX_CACHE_SIZE` (typically 8 KiB on …Otherwise: `kmem_cache_alloc(cache, gfp)` is called on …`kfree(ptr)` → `__kfree()`: uses `virt_to_head_page()` …
EDGE
Size rounding wasteAn allocation of 33 bytes uses the 64-byte cache, wasting 31 bytes; for performa
GFP flag misuseUsing `GFP_KERNEL` (may sleep/reclaim) in atomic context (IRQ handler, spinlock
`kfree()` on a stack pointer or non-kmalloc memorySLUB's `kfree()` uses `virt_to_head_page()` to verify the pointer is slab-manage
Speculative failure under pressureEven `GFP_KERNEL` allocations can fail under extreme memory pressure if the OOM
3.7
Memory Pool (`mm/mempool.c`)
A mempool is a reserved pool of pre-allocated objects that guarantees allocation success even under extreme memory pr…
▾
Definition
A mempool is a reserved pool of pre-allocated objects that guarantees allocation success even under extreme memory pressure, as long as the pool size is not exhausted. It is used in I/O paths (bio allocations, DM) where allocation failure would deadlock the system (the memory reclaim path itself triggers I/O). The pool maintains a minimum number of elements allocated at initialization time.
Execution Flow
mempool_create(min_nr, alloc_fn, free_fn, pool_data) calls alloc_fn() min_nr times and stores the objects in pool->elements[].
mempool_alloc(pool, gfp) first tries the underlying allocator directly: calls pool->alloc(gfp | __GFP_NOMEMALLOC, pool->pool_data).
- If the allocator succeeds, the reserved pool is untouched.
- If the allocator fails and
gfp & __GFP_DIRECT_RECLAIM: the caller may wait on pool->wait (a wait_queue) if the pool is empty.
- If the pool's
elements[] array is non-empty, an element is popped (under pool->lock) and returned.
- If both the allocator and pool are empty and
GFP_NOIO: returns NULL (fatal for some paths; must be sized conservatively).
mempool_free(element, pool):
- If pool is below
min_nr, the element is returned to pool->elements[] and a waiter in pool->wait is woken.
- Otherwise,
pool->free() is called to return the element to the underlying allocator.
Edge Cases & Failure Modes
- Pool size insufficient for deadlock avoidance: If the pool size is too small and all elements are in-flight simultaneously, a new allocation blocks even from the reserved pool, causing deadlock in the reclaim path.
mempool_alloc with GFP_NOIO returning NULL: Callers in the I/O stack must handle NULL explicitly; most DM/bio code treats this as a fatal condition and returns an I/O error.
- Pool element leak: If code acquires an element from a mempool and returns an error without calling
mempool_free(), the pool shrinks below min_nr; the pool replenishes lazily when elements are freed but may be temporarily undersized.
DEF
A mempool is a reserved pool of pre-allocated objects that guarantees allocation success even under extreme memory pr…
FLOW
`mempool_create(min_nr, alloc_fn, free_fn, pool_data)` …`mempool_alloc(pool, gfp)` first tries the underlying a…If the allocator succeeds, the reserved pool is untouch…If the allocator fails and `gfp & __GFP_DIRECT_RECLAIM`…If the pool's `elements[]` array is non-empty, an eleme…If both the allocator and pool are empty and `GFP_NOIO`…`mempool_free(element, pool)`:
EDGE
Pool size insufficient for deadlock avoidanceIf the pool size is too small and all elements are in-flight simultaneously, a n
`mempool_alloc` with `GFP_NOIO` returning NULLCallers in the I/O stack must handle NULL explicitly; most DM/bio code treats th
Pool element leakIf code acquires an element from a mempool and returns an error without calling
3.8
CMA (Contiguous Memory Allocator) (`mm/cma.c`)
CMA reserves contiguous physical memory regions at boot time (via `cma=` kernel parameter or device tree `reserved-me…
▾
Definition
CMA reserves contiguous physical memory regions at boot time (via cma= kernel parameter or device tree reserved-memory nodes). Unlike the buddy allocator, CMA regions are still used for movable page allocations during normal operation and are reclaimed (by migrating those pages) when a CMA allocation is requested, providing contiguous physical memory for DMA-capable devices without permanently wasting RAM.
Execution Flow
- At boot:
cma_declare_contiguous() marks a physical range as a CMA region; the range is managed by the buddy allocator as MIGRATE_CMA pages.
dma_alloc_contiguous() → cma_alloc(cma, count, align, no_warn).
alloc_contig_range() tries to isolate the requested page range from the buddy allocator.
start_isolate_page_range() changes the migration type of the target pages to MIGRATE_ISOLATE, preventing new allocations into them.
migrate_pages() moves all currently-allocated movable pages out of the range to other memory zones.
- If migration succeeds, the range is allocated contiguously and returned as a
struct page *.
cma_release() → free_contig_range() returns the pages to the buddy allocator as MIGRATE_CMA.
Edge Cases & Failure Modes
- Migration failure: If a page in the requested range cannot be migrated (pinned, locked, or of a non-movable type),
alloc_contig_range() fails with -EBUSY; the caller receives NULL.
- CMA fragmentation over time: Long-running systems may pin non-movable pages inside the CMA region (via
get_user_pages()), making contiguous allocation progressively harder.
- CMA size reservation vs. DMA size: If the CMA region is smaller than the device's DMA buffer requirement, the allocation always fails regardless of total free memory.
- Early CMA allocation before migration is ready: During early boot (before
mem_init()), page migration is not available; CMA requests during this phase use a simple early allocator that permanently removes pages from the buddy pool.
DEF
CMA reserves contiguous physical memory regions at boot time (via `cma=` kernel parameter or device tree `reserved-me…
FLOW
At boot: `cma_declare_contiguous()` marks a physical ra…`dma_alloc_contiguous()` → `cma_alloc(cma, count, align…`alloc_contig_range()` tries to isolate the requested p…`start_isolate_page_range()` changes the migration type…`migrate_pages()` moves all currently-allocated movable…If migration succeeds, the range is allocated contiguou…`cma_release()` → `free_contig_range()` returns the pag…
EDGE
Migration failureIf a page in the requested range cannot be migrated (pinned, locked, or of a non
CMA fragmentation over timeLong-running systems may pin non-movable pages inside the CMA region (via `get_u
CMA size reservation vs. DMA sizeIf the CMA region is smaller than the device's DMA buffer requirement, the alloc
Early CMA allocation before migration is readyDuring early boot (before `mem_init()`), page migration is not available; CMA re
3.9
Transparent Huge Pages (THP) (`mm/huge_memory.c`)
THP automatically promotes 4 KiB pages to 2 MiB huge pages (on x86-64) when a process's anonymous mappings are large …
▾
Definition
THP automatically promotes 4 KiB pages to 2 MiB huge pages (on x86-64) when a process's anonymous mappings are large and contiguous, reducing TLB pressure and page table size. It operates transparently: userspace sees no change. khugepaged is a background kernel thread that scans for small-page regions eligible for promotion.
Execution Flow
- A process
mmap()s a large anonymous region.
- Initial page faults allocate 4 KiB pages normally.
khugepaged wakes periodically, scans mm_struct page tables for 2 MiB-aligned regions where all 512 small pages are present and have no special flags (e.g., not locked, not shared beyond the PMD level).
collapse_huge_page() allocates a 2 MiB physically contiguous block (order-9 huge page from the buddy allocator).
copy_page() copies all 512 small pages into the new huge page.
- The PMD entry is replaced with a huge-page PMD pointing to the new 2 MiB block.
- The 512 original small pages are released to the buddy allocator.
- Future page faults in the region use the PMD-level huge page, skipping 512 PTE lookups.
- On split (e.g., due to
mprotect(), mremap(), or memory pressure): split_huge_page() rebuilds the 512 PTE entries and returns the huge page to the buddy allocator.
Edge Cases & Failure Modes
- THP allocation failure: The order-9 allocation may fail under memory pressure;
khugepaged retries later, so the page table remains small until a contiguous block becomes available.
- THP split under swap pressure: If the system needs to swap out part of a THP region, the huge page must be split first; splitting is expensive (requires locking all 512 PTEs and an IPI).
- THP in
mlock()'d regions: Locking a region prevents khugepaged from collapsing it; the interaction between VM_LOCKED and THP requires care in memory-pinning code.
- False sharing amplification: A 2 MiB THP means a write to any page in the region dirtifies the entire 2 MiB; under high write rates with sparse access patterns, this causes more writeback I/O than small pages would.
DEF
THP automatically promotes 4 KiB pages to 2 MiB huge pages (on x86-64) when a process's anonymous mappings are large …
FLOW
A process `mmap()`s a large anonymous region.Initial page faults allocate 4 KiB pages normally.`khugepaged` wakes periodically, scans `mm_struct` page…`collapse_huge_page()` allocates a 2 MiB physically con…`copy_page()` copies all 512 small pages into the new h…The PMD entry is replaced with a huge-page PMD pointing…The 512 original small pages are released to the buddy …Future page faults in the region use the PMD-level huge…On split (e.g., due to `mprotect()`, `mremap()`, or mem…
EDGE
THP allocation failureThe order-9 allocation may fail under memory pressure; `khugepaged` retries late
THP split under swap pressureIf the system needs to swap out part of a THP region, the huge page must be spli
THP in `mlock()`'d regionsLocking a region prevents `khugepaged` from collapsing it; the interaction betwe
False sharing amplificationA 2 MiB THP means a write to any page in the region dirtifies the entire 2 MiB;
3.10
Per-CPU Allocator (`mm/percpu.c`)
The per-CPU allocator manages the per-CPU memory areas created at boot time (and dynamically via `alloc_percpu()`).
▾
Definition
The per-CPU allocator manages the per-CPU memory areas created at boot time (and dynamically via alloc_percpu()). It segments memory into chunks aligned to CPU cache lines and allocates variable-sized per-CPU regions using a first-fit allocator within each chunk. The resulting __percpu-annotated pointers are offsets relative to each CPU's base address.
Execution Flow
- At boot:
pcpu_setup_first_chunk() carves out per-CPU memory from the kernel image and sets __per_cpu_offset[cpu] for each CPU.
alloc_percpu(type) → __alloc_percpu(size, align).
pcpu_alloc() acquires pcpu_lock and searches for a free slot in existing chunks of appropriate group (small/large).
- If no space in existing chunks:
pcpu_create_chunk() allocates new pages from the buddy allocator and maps them into each CPU's per-CPU address space.
- The allocated offset is stored; the returned
__percpu * pointer is an offset, not a real address.
per_cpu_ptr(ptr, cpu) → (void *)((unsigned long)(ptr) + __per_cpu_offset[cpu]) → produces the real per-CPU address for a specific CPU.
free_percpu(ptr) returns the offset to the chunk's free list; fully free chunks are returned to the buddy allocator.
Edge Cases & Failure Modes
- Early boot allocation before
__per_cpu_offset is set: Accessing per-CPU data before setup_per_cpu_areas() completes reads garbage; architecture-specific boot code must be careful about ordering.
- Module per-CPU allocation failure: If
alloc_percpu() fails during module load, the module's init() must free all previously allocated per-CPU areas before returning the error; partial cleanup causes permanent per-CPU space leakage.
- CPU offline drain: When a CPU goes offline, any data accumulated in its per-CPU allocator slab caches (SLUB
cpu_slab) must be drained to the per-node list; the CPU_DEAD notifier handles this but only if properly registered.
- Cache line alignment: The per-CPU allocator aligns allocations to
SMP_CACHE_BYTES (typically 64 B); structures smaller than a cache line waste space but prevent false sharing.
DEF
The per-CPU allocator manages the per-CPU memory areas created at boot time (and dynamically via `alloc_percpu()`).
FLOW
At boot: `pcpu_setup_first_chunk()` carves out per-CPU …`alloc_percpu(type)` → `__alloc_percpu(size, align)`.`pcpu_alloc()` acquires `pcpu_lock` and searches for a …If no space in existing chunks: `pcpu_create_chunk()` a…The allocated offset is stored; the returned `__percpu …`per_cpu_ptr(ptr, cpu)` → `(void *)((unsigned long)(ptr…`free_percpu(ptr)` returns the offset to the chunk's fr…
EDGE
Early boot allocation before `__per_cpu_offset` is setAccessing per-CPU data before `setup_per_cpu_areas()` completes reads garbage; a
Module per-CPU allocation failureIf `alloc_percpu()` fails during module load, the module's `init()` must free al
CPU offline drainWhen a CPU goes offline, any data accumulated in its per-CPU allocator slab cach
Cache line alignmentThe per-CPU allocator aligns allocations to `SMP_CACHE_BYTES` (typically 64 B);
📅
S4 — Process & CPU Schedulers
10 components
4.1
CFS (Completely Fair Scheduler) (`kernel/sched/fair.c`)
CFS is the default scheduler for normal (non-real-time) tasks in Linux, implementing SCHED_NORMAL and SCHED_BATCH pol…
▾
Definition
CFS is the default scheduler for normal (non-real-time) tasks in Linux, implementing SCHED_NORMAL and SCHED_BATCH policies. It models a "perfect multitasking processor" where each task receives equal CPU time proportional to its weight (nice value). It uses a Red-Black tree keyed by vruntime (virtual runtime: actual CPU time scaled by task weight) to select the next task.
Execution Flow
- Task enqueueing:
enqueue_task_fair() adds the sched_entity to the per-CPU RB tree and updates cfs_rq->nr_running.
- Task selection:
pick_next_task_fair() returns the sched_entity with the minimum vruntime (leftmost node of the RB tree).
- The selected task's
vruntime is advanced by calc_delta_fair(delta, se) during each scheduler tick, proportional to actual runtime divided by its weight.
update_curr() is called on each tick (via task_tick_fair()) to update vruntime and check if the current task should be preempted.
- Preemption check: if the minimum
vruntime on the RB tree is more than sysctl_sched_latency / nr_running less than the current task's vruntime, resched_curr() sets TIF_NEED_RESCHED.
- On the next kernel exit or
schedule() call, pick_next_task_fair() dequeues the minimum-vruntime task and context-switches to it.
Edge Cases & Failure Modes
- vruntime overflow: After long uptime, 64-bit
vruntime could theoretically overflow; it doesn't in practice on 64-bit kernels (overflow in ~584 years at 1 GHz), but on 32-bit kernels it can overflow in days if the system is under heavy load.
- Newly woken task vruntime placement: A sleeping task's
vruntime is not advanced; on wake, it is set to max(task->vruntime, cfs_rq->min_vruntime - sysctl_sched_wakeup_granularity) to prevent "vruntime regression" (a just-woken task unfairly dominating the CPU).
- Group scheduling (cgroups) complexity: With CPU cgroups, CFS uses a hierarchical scheduler entity (
sched_entity trees); incorrect group weight configuration can starve entire cgroups.
- Latency target vs. throughput: With many tasks,
sysctl_sched_latency is spread over nr_running tasks; each task's timeslice shrinks below sysctl_sched_min_granularity, causing excessive context-switch overhead.
DEF
CFS is the default scheduler for normal (non-real-time) tasks in Linux, implementing SCHED_NORMAL and SCHED_BATCH pol…
FLOW
Task enqueueing: `enqueue_task_fair()` adds the `sched_…Task selection: `pick_next_task_fair()` returns the `sc…The selected task's `vruntime` is advanced by `calc_del…`update_curr()` is called on each tick (via `task_tick_…Preemption check: if the minimum `vruntime` on the RB t…On the next kernel exit or `schedule()` call, `pick_nex…
EDGE
vruntime overflowAfter long uptime, 64-bit `vruntime` could theoretically overflow; it doesn't in
Newly woken task vruntime placementA sleeping task's `vruntime` is not advanced; on wake, it is set to `max(task->v
Group scheduling (cgroups) complexityWith CPU cgroups, CFS uses a hierarchical scheduler entity (`sched_entity` trees
Latency target vs. throughputWith many tasks, `sysctl_sched_latency` is spread over `nr_running` tasks; each
4.2
RT Scheduler (`kernel/sched/rt.c`)
The real-time scheduler implements `SCHED_FIFO` and `SCHED_RR` policies.
▾
Definition
The real-time scheduler implements SCHED_FIFO and SCHED_RR policies. All real-time tasks have a static priority (1–99 for SCHED_FIFO/SCHED_RR) and preempt any CFS or idle task. SCHED_FIFO runs until blocked or preempted by a higher-priority RT task. SCHED_RR adds a timeslice (100 ms default): when exhausted, the task is moved to the back of the run queue at the same priority.
Execution Flow
enqueue_task_rt() adds the task to the per-priority FIFO queue in rt_rq->active.queue[priority].
pick_next_task_rt() calls pick_next_rt_entity() → finds the highest non-empty priority queue using the rt_rq->active.bitmap (a bitmask of populated priority levels).
- The highest-priority task is returned; if multiple tasks share the same priority, the head of the FIFO queue is selected.
task_tick_rt() runs on each tick; for SCHED_RR, decrements the timeslice counter; on expiry, calls requeue_task_rt() to move the task to the tail of its priority queue.
- For
SCHED_FIFO, the task runs until it blocks, yields, or is preempted by a higher-priority task.
dequeue_task_rt() removes the task from the bitmap/queue on block or completion.
Edge Cases & Failure Modes
- RT bandwidth throttling: Without
CONFIG_RT_GROUP_SCHED, a single SCHED_FIFO task at priority 99 can starve all other tasks (including the kernel's watchdog thread). sysctl_sched_rt_runtime / sysctl_sched_rt_period enforce a bandwidth cap (default: 95% CPU for RT tasks).
- Priority inversion in RT: An RT task blocked waiting for a resource held by a CFS task may be delayed indefinitely without priority inheritance;
SCHED_FIFO with pthread_mutexattr_setprotocol(PTHREAD_PRIO_INHERIT) mitigates this.
- Missed SCHED_RR timeslice reset: On
fork(), the child inherits the parent's remaining timeslice, not a fresh 100 ms; this can cause the child to run for a very short time before being rotated.
- Migration to wrong CPU: Under
RT_GROUP_SCHED, pull/push migrations of RT tasks must respect CPU affinity masks; a migration that violates cpus_allowed causes a kernel BUG in set_task_cpu().
DEF
The real-time scheduler implements `SCHED_FIFO` and `SCHED_RR` policies.
FLOW
`enqueue_task_rt()` adds the task to the per-priority F…`pick_next_task_rt()` calls `pick_next_rt_entity()` → f…The highest-priority task is returned; if multiple task…`task_tick_rt()` runs on each tick; for `SCHED_RR`, dec…For `SCHED_FIFO`, the task runs until it blocks, yields…`dequeue_task_rt()` removes the task from the bitmap/qu…
EDGE
RT bandwidth throttlingWithout `CONFIG_RT_GROUP_SCHED`, a single SCHED_FIFO task at priority 99 can sta
Priority inversion in RTAn RT task blocked waiting for a resource held by a CFS task may be delayed inde
Missed SCHED_RR timeslice resetOn `fork()`, the child inherits the parent's remaining timeslice, not a fresh 10
Migration to wrong CPUUnder `RT_GROUP_SCHED`, pull/push migrations of RT tasks must respect CPU affini
4.3
Deadline Scheduler (`kernel/sched/deadline.c`)
The deadline scheduler implements `SCHED_DEADLINE` policy based on the Earliest Deadline First (EDF) algorithm with C…
▾
Definition
The deadline scheduler implements SCHED_DEADLINE policy based on the Earliest Deadline First (EDF) algorithm with Constant Bandwidth Server (CBS) for admission control. Each task specifies three parameters: runtime (worst-case execution time per period), deadline (relative deadline), and period. The scheduler guarantees that a task meeting its parameters will never miss its deadline, as long as the total utilization (Σ runtime/period) ≤ 1.
Execution Flow
- Task admission:
dl_overflow() checks that adding the new task's utilization (runtime/period) would not exceed the CPU's capacity; rejects with EBUSY if it would.
enqueue_task_dl() inserts the task's sched_dl_entity into the per-CPU RB tree keyed by absolute deadline.
pick_next_task_dl() returns the leftmost node (earliest absolute deadline).
- During execution:
update_dl_task_contention() depletes the CBS bandwidth; task_tick_dl() decrements remaining runtime.
- When runtime is exhausted before the deadline, the task is throttled:
start_dl_timer() sets a replenishment timer for the next period start.
- At period start, the runtime is replenished and the task re-enters the RB tree with a new deadline.
- When the absolute deadline passes (task completes):
dequeue_task_dl() removes the entity; the next period's parameters are set.
Edge Cases & Failure Modes
- Admission control race: Two simultaneous
sched_setattr() calls for two tasks may each pass the capacity check individually but together exceed 100%; the kernel serializes via dl_b->lock per root domain but a race across NUMA nodes can still occur.
- Missed deadline (overrun): CBS guarantees no deadline miss for tasks meeting their parameters; if a task's actual execution time exceeds
runtime, it is throttled (not preempted), which causes a soft deadline miss for that period.
SCHED_DEADLINE task on a cgroup-constrained CPU: If the CPU's available capacity is reduced by cpuset/cgroup constraints, the utilization bound is tighter; the kernel doesn't automatically re-admit tasks after capacity reduction.
- Priority boosting interaction: A deadline task boosting a mutex-holding CFS task (via PI) adds unbounded latency to the DL task; this is an open research problem in the PREEMPT_RT community.
DEF
The deadline scheduler implements `SCHED_DEADLINE` policy based on the Earliest Deadline First (EDF) algorithm with C…
FLOW
Task admission: `dl_overflow()` checks that adding the …`enqueue_task_dl()` inserts the task's `sched_dl_entity…`pick_next_task_dl()` returns the leftmost node (earlie…During execution: `update_dl_task_contention()` deplete…When runtime is exhausted before the deadline, the task…At period start, the runtime is replenished and the tas…When the absolute deadline passes (task completes): `de…
EDGE
Admission control raceTwo simultaneous `sched_setattr()` calls for two tasks may each pass the capacit
Missed deadline (overrun)CBS guarantees no deadline miss for tasks meeting their parameters; if a task's
`SCHED_DEADLINE` task on a cgroup-constrained CPUIf the CPU's available capacity is reduced by cpuset/cgroup constraints, the uti
Priority boosting interactionA deadline task boosting a mutex-holding CFS task (via PI) adds unbounded latenc
4.4
Idle Scheduler (`kernel/sched/idle.c`)
The idle scheduler runs the special `idle` task (PID 0, one per CPU) when no other runnable task exists.
▾
Definition
The idle scheduler runs the special idle task (PID 0, one per CPU) when no other runnable task exists. The idle task calls the architecture's idle loop (cpu_idle_loop()), which calls cpuidle_idle_call() to enter the deepest available C-state (CPU power state) using the cpuidle framework. The idle scheduler is the last-resort scheduler class with priority below all others.
Execution Flow
pick_next_task_idle() is reached only when all other scheduler classes (stop, dl, rt, fair) have no runnable tasks.
- The idle task is returned; it cannot be dequeued.
cpu_idle_loop() starts: sets current->state = TASK_IDLE.
tick_nohz_idle_enter() stops the scheduling tick if NO_HZ_IDLE is configured (tickless idle).
cpuidle_idle_call() queries the cpuidle governor (menu or ladder) for the best C-state.
- The cpuidle driver executes
state->enter() (e.g., mwait_idle_with_hints() on x86 → MWAIT instruction).
- When a hardware interrupt or IPI arrives, the CPU exits the C-state and re-runs the scheduler.
tick_nohz_idle_exit() restores the tick; schedule_idle() checks for new runnable tasks.
Edge Cases & Failure Modes
- C-state entry latency vs. wakeup latency: Deep C-states (C6, C7) have high exit latency (100s of µs); a real-time task requiring sub-100 µs response must set a
PM_QOS_CPU_DMA_LATENCY constraint to prevent deep idle entry.
- Idle task stack overflow: The idle task has a fixed kernel stack (8 KiB or 16 KiB); a buggy interrupt handler called in idle context that uses excessive stack causes a stack overflow without protection (unless
CONFIG_VMAP_STACK is set).
- Tick granularity in nohz_full: On CPUs in
nohz_full mode, re-entering the tick after long idle periods requires an IPI from another CPU, adding latency.
- Intel C-state hardware bugs: Certain CPU stepping revisions have errata where entering C6 with specific hardware configurations causes system hangs; the kernel applies microcode workarounds or avoids the affected C-state.
DEF
The idle scheduler runs the special `idle` task (PID 0, one per CPU) when no other runnable task exists.
FLOW
`pick_next_task_idle()` is reached only when all other …The idle task is returned; it cannot be dequeued.`cpu_idle_loop()` starts: sets `current->state = TASK_I…`tick_nohz_idle_enter()` stops the scheduling tick if `…`cpuidle_idle_call()` queries the cpuidle governor (men…The cpuidle driver executes `state->enter()` (e.g., `mw…When a hardware interrupt or IPI arrives, the CPU exits…`tick_nohz_idle_exit()` restores the tick; `schedule_id…
EDGE
C-state entry latency vs. wakeup latencyDeep C-states (C6, C7) have high exit latency (100s of µs); a real-time task req
Idle task stack overflowThe idle task has a fixed kernel stack (8 KiB or 16 KiB); a buggy interrupt hand
Tick granularity in nohz_fullOn CPUs in `nohz_full` mode, re-entering the tick after long idle periods requir
Intel C-state hardware bugsCertain CPU stepping revisions have errata where entering C6 with specific hardw
4.5
Stop Scheduler (`kernel/sched/stop_task.c`)
The stop scheduler is the highest-priority scheduler class, running `stop_machine` tasks (threads named `migration/N`).
▾
Definition
The stop scheduler is the highest-priority scheduler class, running stop_machine tasks (threads named migration/N). It is used for operations that must run exclusively on a CPU with all other activity stopped: CPU hotplug, scheduler migration operations, RCU quiescent state processing, and stop_machine() callbacks. Only one stop task exists per CPU.
Execution Flow
stop_one_cpu(cpu, fn, arg) or stop_machine(fn, arg, cpumask) is called.
cpu_stop_work entries are prepared with the callback fn and queued on the target CPU's stopper->works list.
- The stop thread (
migration/N for CPU N) is woken via wake_up_process().
pick_next_task_stop() runs before all other scheduler classes; it preempts any running task immediately.
- The stop task runs
fn(arg) in kernel thread context.
- For
stop_machine(): all CPUs are synchronously stopped (via multi_cpu_stop() using an atomic state machine); only the calling CPU executes fn().
- After
fn() returns, CPUs are released in reverse order; the stop task sleeps by setting its state to TASK_DEAD (stop task goes back to waiting).
Edge Cases & Failure Modes
- Nested
stop_machine() calls: Calling stop_machine() from within a stop_machine() callback deadlocks (all CPUs are already stopped; the inner call cannot stop them again).
- Stop task migration race: The stop task must run on its assigned CPU; if the CPU's affinity is changed while a stop work is pending, the work is never processed.
- Long
stop_machine() callbacks: Any code run inside stop_machine() runs with IRQs disabled on all CPUs; long-running callbacks cause visible system freezes and watchdog timeouts.
stop_one_cpu_nowait() completion ordering: The async variant (_nowait) returns before fn() completes; callers that assume immediate completion introduce subtle ordering bugs.
DEF
The stop scheduler is the highest-priority scheduler class, running `stop_machine` tasks (threads named `migration/N`).
FLOW
`stop_one_cpu(cpu, fn, arg)` or `stop_machine(fn, arg, …`cpu_stop_work` entries are prepared with the callback …The stop thread (`migration/N` for CPU N) is woken via …`pick_next_task_stop()` runs before all other scheduler…The stop task runs `fn(arg)` in kernel thread context.For `stop_machine()`: all CPUs are synchronously stoppe…After `fn()` returns, CPUs are released in reverse orde…
EDGE
Nested `stop_machine()` callsCalling `stop_machine()` from within a `stop_machine()` callback deadlocks (all
Stop task migration raceThe stop task must run on its assigned CPU; if the CPU's affinity is changed whi
Long `stop_machine()` callbacksAny code run inside `stop_machine()` runs with IRQs disabled on all CPUs; long-r
`stop_one_cpu_nowait()` completion orderingThe async variant (`_nowait`) returns before `fn()` completes; callers that assu
4.6
Core Scheduler Loop (`schedule()` / `kernel/sched/core.c`)
`schedule()` is the central context-switch function that runs the scheduler, selects the next task via `pick_next_tas…
▾
Definition
schedule() is the central context-switch function that runs the scheduler, selects the next task via pick_next_task(), and performs the actual CPU context switch. It is the heart of the scheduler; every voluntary or involuntary context switch passes through it.
Execution Flow
schedule() → __schedule(sm) where sm is the preempt mode (voluntary, preemption, idle).
rq_lock(rq) acquires the run queue lock for the current CPU.
deactivate_task() is called if the current task is not runnable (TASK_INTERRUPTIBLE, TASK_UNINTERRUPTIBLE).
pick_next_task(rq, prev) calls the scheduler class chain: stop → deadline → rt → fair → idle; returns the highest-priority runnable task.
clear_tsk_need_resched(prev) clears the TIF_NEED_RESCHED flag.
prepare_task_switch(rq, prev, next) runs pre-switch callbacks (e.g., perf_event_task_sched_out()).
context_switch(rq, prev, next) performs the actual switch:
switch_mm_irqs_off(): switches page tables (CR3 on x86).
switch_to(prev, next, prev): saves/restores CPU registers (rsp, rbp) and FPU state lazily; jumps to next.
- After
switch_to() returns, execution is on the next task's stack.
finish_task_switch() releases the previous task's run queue lock and handles post-switch work.
Edge Cases & Failure Modes
schedule() in IRQ context: Calling schedule() with in_interrupt() returning true causes a BUG; IRQ handlers must not sleep.
TIF_NEED_RESCHED missed: If clear_tsk_need_resched() is called but the new task immediately re-sets the flag (e.g., a high-frequency timer fires during finish_task_switch()), the CPU loops back into __schedule() on the next kernel exit.
- FPU state corruption: Lazy FPU save/restore means the FPU state is only saved when another task uses the FPU; if a task is migrated between CPUs while FPU state is not saved,
migrate_disable() / fpregs_lock() must be used.
- Runqueue lock ordering violations: Locking two run queues simultaneously (required for cross-CPU task migration) must always be done in lowest-address-first order to prevent deadlock.
DEF
`schedule()` is the central context-switch function that runs the scheduler, selects the next task via `pick_next_tas…
FLOW
`schedule()` → `__schedule(sm)` where `sm` is the preem…`rq_lock(rq)` acquires the run queue lock for the curre…`deactivate_task()` is called if the current task is no…`pick_next_task(rq, prev)` calls the scheduler class ch…`clear_tsk_need_resched(prev)` clears the `TIF_NEED_RES…`prepare_task_switch(rq, prev, next)` runs pre-switch c…`context_switch(rq, prev, next)` performs the actual sw….After `switch_to()` returns, execution is on the `next`…`finish_task_switch()` releases the previous task's run…
EDGE
`schedule()` in IRQ contextCalling `schedule()` with `in_interrupt()` returning true causes a BUG; IRQ hand
`TIF_NEED_RESCHED` missedIf `clear_tsk_need_resched()` is called but the new task immediately re-sets the
FPU state corruptionLazy FPU save/restore means the FPU state is only saved when another task uses t
Runqueue lock ordering violationsLocking two run queues simultaneously (required for cross-CPU task migration) mu
4.7
Load Balancer (`kernel/sched/fair.c` — `load_balance()`)
The CFS load balancer periodically redistributes tasks across CPUs and NUMA nodes to prevent imbalanced CPU utilization.
▾
Definition
The CFS load balancer periodically redistributes tasks across CPUs and NUMA nodes to prevent imbalanced CPU utilization. It uses a hierarchical scheduling domain model (SMT ↔ Core ↔ Package ↔ NUMA node) and migrates tasks from the busiest CPU in the busiest domain to the idlest CPU, subject to cache affinity and task migration constraints.
Execution Flow
- Triggered by:
scheduler_tick() detecting CPU imbalance, or newidle_balance() when a CPU becomes idle.
load_balance(this_cpu, this_rq, sd, idle, continue_balancing) is called for the current scheduling domain.
find_busiest_group() examines all CPUs in the scheduling domain, computes per-group load (using PELT: Per-Entity Load Tracking), and returns the group with the highest weighted load.
find_busiest_queue() within the busiest group finds the CPU with the highest cfs_rq->avg.load_avg.
detach_tasks() selects and removes up to imbalance worth of load from the busiest queue, applying migration constraints (CPU affinity, cache hot threshold, numa_preferred_nid).
attach_tasks() migrates the detached tasks to this_rq.
- If migration is blocked (all tasks on busiest CPU are CPU-affinity-pinned),
load_balance() returns a "failed" indicator and the domain is marked for re-check.
- The process repeats for each outer scheduling domain level (Core → Package → NUMA).
Edge Cases & Failure Modes
- Affinity pinning creates permanent imbalance: If all tasks on a CPU are pinned to that CPU, the load balancer cannot fix the imbalance and repeatedly fails;
nr_balance_failed increments until a active_load_balance_cpu_stop() migration is attempted (which also fails, causing a stall).
- Scheduler domain misconfiguration (topology): An incorrect
sched_domain_topology_level array (e.g., after CPU hotplug that doesn't rebuild topology) causes domains to cover wrong CPU sets, leading to migrations that degrade cache performance.
- NUMA Imbalance vs. load imbalance: The NUMA balancer (task_numa_work) may conflict with the load balancer's decisions; a task may be migrated by both, negating each other's work.
- Nohz imbalance: CPUs in
nohz_full mode don't run the scheduler tick; their load is not updated until they exit idle; nohz_kick_needed() / nohz_balancer_kick() asynchronously trigger a rebalance via IPI.
DEF
The CFS load balancer periodically redistributes tasks across CPUs and NUMA nodes to prevent imbalanced CPU utilization.
FLOW
Triggered by: `scheduler_tick()` detecting CPU imbalanc…`load_balance(this_cpu, this_rq, sd, idle, continue_bal…`find_busiest_group()` examines all CPUs in the schedul…`find_busiest_queue()` within the busiest group finds t…`detach_tasks()` selects and removes up to `imbalance` …`attach_tasks()` migrates the detached tasks to `this_r…If migration is blocked (all tasks on busiest CPU are C…The process repeats for each outer scheduling domain le…
EDGE
Affinity pinning creates permanent imbalanceIf all tasks on a CPU are pinned to that CPU, the load balancer cannot fix the i
Scheduler domain misconfiguration (topology)An incorrect `sched_domain_topology_level` array (e.g., after CPU hotplug that d
NUMA Imbalance vs. load imbalanceThe NUMA balancer (task_numa_work) may conflict with the load balancer's decisio
Nohz imbalanceCPUs in `nohz_full` mode don't run the scheduler tick; their load is not updated
4.8
NUMA Balancer (`kernel/sched/numa_balancing.c`)
The NUMA balancer (`CONFIG_NUMA_BALANCING`) scans a process's page mappings periodically, marks pages as NUMA-fault-e…
▾
Definition
The NUMA balancer (CONFIG_NUMA_BALANCING) scans a process's page mappings periodically, marks pages as NUMA-fault-eligible (by clearing PTE access bits), waits for access faults, and uses fault locations to infer which NUMA node the task accesses most. It then migrates both the task and its pages to the preferred NUMA node to reduce remote memory access latency.
Execution Flow
task_tick_fair() calls task_tick_numa(rq, curr, queued) on each scheduler tick.
numa_work_next_cpu() queues a numa_work on the task via task_work_add().
- When the task returns to userspace,
task_numa_work() scans a portion of the task's page tables via change_prot_numa(), converting PTEs to _PAGE_NUMA (access-cleared).
- The next access to such a page triggers a NUMA fault (page fault with
pte_numa() set).
do_numa_page() → mpol_misplaced() determines if the page should be migrated.
migrate_misplaced_page() moves the page to the task's preferred NUMA node.
- NUMA hint statistics (
task->numa_faults[]) are updated; numa_migrate_preferred() checks if the task should be migrated to the node holding most of its pages.
task_numa_migrate() calls migrate_task_to() to move the task itself.
Edge Cases & Failure Modes
- NUMA thrashing: Two tasks repeatedly swap NUMA nodes to follow each other's shared pages, causing continuous migration overhead;
sched_numa_balancing can be disabled per-cgroup.
- False-negative scan: If a task's working set exceeds the scan window, many hot pages are never promoted; the scan rate is adaptive but may lag for memory-intensive workloads.
- Migration stall under memcg limits: Page migration requires allocation of a new page; if the destination NUMA node's memcg limit is reached, migration fails and the fault penalty is wasted.
- Interaction with huge pages: NUMA fault handling on a THP requires splitting the huge page first; this defeats THP's purpose and can cause performance regression.
DEF
The NUMA balancer (`CONFIG_NUMA_BALANCING`) scans a process's page mappings periodically, marks pages as NUMA-fault-e…
FLOW
`task_tick_fair()` calls `task_tick_numa(rq, curr, queu…`numa_work_next_cpu()` queues a `numa_work` on the task…When the task returns to userspace, `task_numa_work()` …The next access to such a page triggers a NUMA fault (p…`do_numa_page()` → `mpol_misplaced()` determines if the…`migrate_misplaced_page()` moves the page to the task's…NUMA hint statistics (`task->numa_faults[]`) are update…`task_numa_migrate()` calls `migrate_task_to()` to move…
EDGE
NUMA thrashingTwo tasks repeatedly swap NUMA nodes to follow each other's shared pages, causin
False-negative scanIf a task's working set exceeds the scan window, many hot pages are never promot
Migration stall under memcg limitsPage migration requires allocation of a new page; if the destination NUMA node's
Interaction with huge pagesNUMA fault handling on a THP requires splitting the huge page first; this defeat
4.9
cgroup CPU Controller (`kernel/cgroup/cpu.c`)
The cgroup CPU controller (v1: `cpu` controller; v2: unified hierarchy) limits and prioritizes CPU access for groups …
▾
Definition
The cgroup CPU controller (v1: cpu controller; v2: unified hierarchy) limits and prioritizes CPU access for groups of tasks. In v2, it provides cpu.weight (CFS nice-equivalent weight), cpu.max (bandwidth limiting: maximum runtime per period), and cpu.uclamp.min/max (frequency scaling hints). It operates by embedding cgroup-level sched_entity objects into the CFS hierarchy.
Execution Flow
- A cgroup is created:
mkdir /sys/fs/cgroup/mygroup.
- Tasks are added:
echo PID > /sys/fs/cgroup/mygroup/cgroup.procs.
- CFS creates a per-CPU
task_group with a cfs_rq (group-level run queue) and a sched_entity (group entity on the parent run queue).
- When a task in the group is scheduled: CFS selects the group entity from the parent
cfs_rq (based on group vruntime), then selects a task entity from the group's cfs_rq.
- Bandwidth enforcement (
cpu.max): cfs_bandwidth_work runs periodically; when the group has used its quota within the period, throttle_cfs_rq() is called → all tasks in the group are dequeued.
- At the next period start,
unthrottle_cfs_rq() re-enqueues all throttled tasks.
cpu.weight adjusts the group entity's se.load.weight, proportionally scaling vruntime advance rate.
Edge Cases & Failure Modes
- Bandwidth throttling cliff: A cgroup that hits its
cpu.max quota in the first millisecond of a 100 ms period is throttled for the remaining 99 ms, causing application-visible latency spikes even when total CPU is available.
- Hierarchical weight interaction: Nested cgroups accumulate weights multiplicatively; a deeply nested cgroup may have a de facto weight of near zero regardless of its own
cpu.weight setting.
- RT tasks bypass cgroup limits:
SCHED_FIFO/SCHED_RR tasks are not subject to cpu.max; they can exceed the cgroup's intended CPU budget.
- v1 / v2 incompatibility: The
cpu controller cannot be simultaneously mounted in v1 and v2 hierarchies; a system using systemd's cgroup v2 must not also mount the legacy v1 cpu hierarchy.
DEF
The cgroup CPU controller (v1: `cpu` controller; v2: unified hierarchy) limits and prioritizes CPU access for groups …
FLOW
A cgroup is created: `mkdir /sys/fs/cgroup/mygroup`.Tasks are added: `echo PID > /sys/fs/cgroup/mygroup/cgr…CFS creates a per-CPU `task_group` with a `cfs_rq` (gro…When a task in the group is scheduled: CFS selects the …Bandwidth enforcement (`cpu.max`): `cfs_bandwidth_work`…At the next period start, `unthrottle_cfs_rq()` re-enqu…`cpu.weight` adjusts the group entity's `se.load.weight…
EDGE
Bandwidth throttling cliffA cgroup that hits its `cpu.max` quota in the first millisecond of a 100 ms peri
Hierarchical weight interactionNested cgroups accumulate weights multiplicatively; a deeply nested cgroup may h
RT tasks bypass cgroup limits`SCHED_FIFO`/`SCHED_RR` tasks are not subject to `cpu.max`; they can exceed the
v1 / v2 incompatibilityThe `cpu` controller cannot be simultaneously mounted in v1 and v2 hierarchies;
4.10
Context Switch / `switch_to()` Mechanism (`arch/x86/include/asm/switch_to.h`)
`context_switch()` and its architecture-specific `switch_to(prev, next, last)` macro implement the CPU register save/…
▾
Definition
context_switch() and its architecture-specific switch_to(prev, next, last) macro implement the CPU register save/restore that transfers execution from one task to another. It saves the current task's stack pointer (rsp), instruction pointer (implicitly via the return address), and optionally FPU state, then restores the next task's saved state.
Execution Flow
context_switch(rq, prev, next) in kernel/sched/core.c is called with rq->lock held.
prepare_task_switch(rq, prev, next) runs perf/cgroup/trace hooks.
- If
next has a different address space: switch_mm_irqs_off(prev->active_mm, next->mm, next) → updates CR3 (x86) with next->mm->pgd physical address, flushing TLB entries (or using PCID/ASID to avoid a full flush).
switch_to(prev, next, prev) (assembly macro):
- Pushes callee-saved registers (
rbp, rbx, r12–r15) onto prev's kernel stack.
- Saves
rsp to prev->thread.sp.
- Loads
rsp from next->thread.sp (now on next's kernel stack).
- Calls
__switch_to_asm() → __switch_to() which updates the TSS.sp0 field (kernel stack pointer for syscall entry), loads next's fsbase/gsbase.
ret on next's stack returns to wherever next last called schedule() (or to ret_from_fork for a new task).
- After
switch_to() returns (now running as next): finish_task_switch(prev) releases prev's lock state and handles task cleanup.
Edge Cases & Failure Modes
- FPU state corruption without lazy save: The kernel uses lazy FPU save/restore; if a task is migrated between CPUs while FPU state is not saved (because no other task used the FPU),
fpregs_assert_state_consistent() will BUG.
- Stack corruption during switch: A stack overflow in
prev that overwrites the saved rsp causes the CPU to jump to a garbage address when switch_to() loads the corrupted stack pointer. CONFIG_VMAP_STACK places a guard page under each stack to catch this.
- CR3 update and speculative execution (Spectre/Meltdown): On CPUs without hardware PCID support, every
switch_mm flushes the entire TLB; this is expensive for applications with large working sets; CONFIG_PAGE_TABLE_ISOLATION (PTI/KPTI) adds further overhead for user/kernel page table separation.
last pointer aliasing: switch_to(prev, next, prev) returns with prev pointing to the task that was running before next woke, which may differ from the task that called switch_to() if another task ran between them; callers must use the returned prev (via the last output parameter) for cleanup, not the caller's local variable.
# Linux Kernel Component Taxonomy — Part 2 (Categories 5–8)
DEF
`context_switch()` and its architecture-specific `switch_to(prev, next, last)` macro implement the CPU register save/…
FLOW
`context_switch(rq, prev, next)` in `kernel/sched/core.…`prepare_task_switch(rq, prev, next)` runs perf/cgroup/…If `next` has a different address space: `switch_mm_irq…with `next->mm->pgd` physical address, flushing TLB ent…`switch_to(prev, next, prev)` (assembly macro):After `switch_to()` returns (now running as `next`): `f…
EDGE
FPU state corruption without lazy saveThe kernel uses lazy FPU save/restore; if a task is migrated between CPUs while
Stack corruption during switchA stack overflow in `prev` that overwrites the saved `rsp` causes the CPU to jum
CR3 update and speculative execution (Spectre/Meltdown)On CPUs without hardware PCID support, every `switch_mm` flushes the entire TLB;
`last` pointer aliasing`switch_to(prev, next, prev)` returns with `prev` pointing to the task that was
💾
S5 — Block & I/O Schedulers
10 components
5.1
blk-mq (Multi-Queue Block Layer) (`block/blk-mq.c`)
blk-mq is the modern block layer infrastructure introduced in Linux 3.13 that replaces the single-queue block layer.
▾
Definition
blk-mq is the modern block layer infrastructure introduced in Linux 3.13 that replaces the single-queue block layer. It maps one software staging queue per CPU (or per NUMA node) to multiple hardware dispatch queues on the storage device, eliminating the single global queue lock bottleneck and enabling high-IOPS NVMe SSDs to use full parallelism.
Execution Flow
- A filesystem or direct I/O caller submits a
bio to submit_bio().
blk_mq_submit_bio() splits the bio if it exceeds queue limits, then calls blk_mq_get_request() to allocate a struct request from the per-CPU request pool.
- The request is inserted into the per-CPU software staging queue (
ctx->rq_list).
blk_mq_run_hw_queue() is called: it drains the software queue into the hardware dispatch queue (hctx->dispatch).
- If an I/O scheduler is attached,
e->type->ops.insert_requests() is called (e.g., BFQ sorts by deadline/priority).
blk_mq_dispatch_rq_list() calls the driver's queue_rq() callback with a batch of requests.
- The driver submits requests to hardware (NVMe SQ doorbell, SCSI command queue, etc.).
- Hardware completes I/O, raises an interrupt;
blk_mq_complete_request() is called, which calls the bio's bi_end_io callback, waking the submitting task.
Edge Cases & Failure Modes
- Request tag exhaustion: Each hardware queue has a fixed tag depth; if all tags are in-flight, new requests block in
blk_mq_get_tag() on the hctx->tags->bitmap_tags wait queue.
- Queue quiesce deadlock:
blk_mq_quiesce_queue() waits for all in-flight requests to complete; if called from a context that holds a resource a completing request needs (e.g., a filesystem lock), it deadlocks.
- CPU hotplug and per-CPU queue migration: When a CPU goes offline, its software queue must be drained to another CPU's queue;
blk_mq_hctx_notify_dead() handles this but races with concurrent submissions.
- Plug/unplug coalescing failure:
blk_start_plug() / blk_finish_plug() batch requests before dispatch; a process that starts a plug and exits without calling blk_finish_plug() leaves requests stranded until the plug is flushed at schedule().
DEF
blk-mq is the modern block layer infrastructure introduced in Linux 3.13 that replaces the single-queue block layer.
FLOW
A filesystem or direct I/O caller submits a `bio` to `s…`blk_mq_submit_bio()` splits the bio if it exceeds queu…The request is inserted into the per-CPU software stagi…`blk_mq_run_hw_queue()` is called: it drains the softwa…If an I/O scheduler is attached, `e->type->ops.insert_r…`blk_mq_dispatch_rq_list()` calls the driver's `queue_r…The driver submits requests to hardware (NVMe SQ doorbe…Hardware completes I/O, raises an interrupt; `blk_mq_co…
EDGE
Request tag exhaustionEach hardware queue has a fixed tag depth; if all tags are in-flight, new reques
Queue quiesce deadlock`blk_mq_quiesce_queue()` waits for all in-flight requests to complete; if called
CPU hotplug and per-CPU queue migrationWhen a CPU goes offline, its software queue must be drained to another CPU's que
Plug/unplug coalescing failure`blk_start_plug()` / `blk_finish_plug()` batch requests before dispatch; a proce
5.2
BFQ (Budget Fair Queueing) I/O Scheduler (`block/bfq-iosched.c`)
BFQ is a proportional-share I/O scheduler that assigns a "budget" (number of sectors) to each process's I/O queue and…
▾
Definition
BFQ is a proportional-share I/O scheduler that assigns a "budget" (number of sectors) to each process's I/O queue and serves them in order of budget, while maintaining low latency for interactive and soft-real-time workloads. It tracks per-process I/O queues (bfq_queue) organized in a hierarchical weight-fair tree with time-slice based service.
Execution Flow
- A request arrives:
bfq_insert_request() places it in the submitting process's bfq_queue, sorted by LBA (for seek optimization).
- The
bfq_queue is inserted into the BFQ service tree (a Red-Black tree ordered by virtual finish time, computed from budget and weight).
bfq_select_queue() selects the queue with the minimum virtual finish time that has pending requests and whose budget has not expired.
- The selected queue is "in service"; its requests are dispatched to the blk-mq hardware queue via
bfq_dispatch_request().
- After
budget sectors are served (or a think-time timeout fires), the queue's virtual finish time is advanced and it is re-inserted into the service tree.
- Idle detection: BFQ waits a brief "slice_idle" period for the in-service process to submit a new request (to maintain request locality); if none arrives, the next queue is selected.
Edge Cases & Failure Modes
- Latency cliff for interactive tasks: If a large sequential background I/O workload gets a large budget, it monopolizes the disk for many milliseconds; BFQ's "low_latency" heuristic attempts to detect and throttle such workloads, but misclassification causes latency spikes.
- Weight starvation in hierarchical scheduling: A cgroup with low
blkio.bfq.weight may starve if a high-weight sibling continuously issues I/O; the starvation is bounded by BFQ's fairness guarantee but can still cause multi-second delays.
- bfq_queue split on shared access: When multiple processes share the same
bfq_queue (cooperative scheduling), a synchronization point split is needed; missing a split leaves one process's I/O blocking another's.
- Think-time estimation error: BFQ uses per-queue think-time statistics to decide whether to wait for more I/O; on SSDs with microsecond latency, the think-time window is too coarse and BFQ may switch queues unnecessarily.
DEF
BFQ is a proportional-share I/O scheduler that assigns a "budget" (number of sectors) to each process's I/O queue and…
FLOW
A request arrives: `bfq_insert_request()` places it in …The `bfq_queue` is inserted into the BFQ service tree (…`bfq_select_queue()` selects the queue with the minimum…The selected queue is "in service"; its requests are di…After `budget` sectors are served (or a think-time time…Idle detection: BFQ waits a brief "slice_idle" period f…
EDGE
Latency cliff for interactive tasksIf a large sequential background I/O workload gets a large budget, it monopolize
Weight starvation in hierarchical schedulingA cgroup with low `blkio.bfq.weight` may starve if a high-weight sibling continu
bfq_queue split on shared accessWhen multiple processes share the same `bfq_queue` (cooperative scheduling), a s
Think-time estimation errorBFQ uses per-queue think-time statistics to decide whether to wait for more I/O;
5.3
mq-deadline I/O Scheduler (`block/mq-deadline.c`)
mq-deadline is the multi-queue adaptation of the classic deadline scheduler.
▾
Definition
mq-deadline is the multi-queue adaptation of the classic deadline scheduler. It maintains two per-direction (read/write) sorted queues: a Red-Black tree ordered by LBA (for sequential merging) and a FIFO queue ordered by submission time (for deadline enforcement). Requests are served from the LBA-sorted tree but are "expired" to the FIFO head if their deadline is reached.
Execution Flow
- A request arrives:
dd_insert_request() inserts it into both the LBA RB-tree and the FIFO queue for its direction (read or write).
dd_dispatch_request() is called by blk-mq when the hardware queue needs requests.
- Deadline check: the front of the read FIFO is examined; if its deadline (
jiffies + read_expire, default 500 ms) has passed, it is dispatched immediately (deadline enforcement).
- Write starvation check: if writes have been starved for
writes_starved (default 2) read batches, force a write dispatch.
- Otherwise,
dd_dispatch_prio_aged_requests() dispatches from the LBA RB-tree to maximize sequential access patterns (elevator-style seek optimization).
- After
fifo_batch (default 16) consecutive requests from one direction, the scheduler switches to the other direction.
Edge Cases & Failure Modes
- Read/write inversion on SSDs: mq-deadline was designed for rotational media where sequential access is faster; on NVMe SSDs, seek optimization is irrelevant, and the LBA sorting adds overhead without benefit. The
none scheduler is often better for NVMe.
- Write starvation under heavy read load: If
writes_starved is set too high, write requests can be delayed far beyond their deadline; dirty-page writeback stalls and VFS write syscalls block.
- FIFO queue ordering on multi-queue devices: With multiple hardware queues, the FIFO ordering guarantee breaks across queues; a request submitted earlier may be on a different queue than a later request, violating the deadline ordering assumption.
- Deadline expiry on power-loss-safe SSDs: Power-loss-safe NVMe drives buffer writes in DRAM; a "written" request may not reach flash until a later flush; the deadline scheduler does not account for this, so apparent completion time differs from physical persistence time.
DEF
mq-deadline is the multi-queue adaptation of the classic deadline scheduler.
FLOW
A request arrives: `dd_insert_request()` inserts it int…`dd_dispatch_request()` is called by blk-mq when the ha…Deadline check: the front of the read FIFO is examined;…Write starvation check: if writes have been starved for…read batches, force a write dispatch.Otherwise, `dd_dispatch_prio_aged_requests()` dispatche…After `fifo_batch` (defaultconsecutive requests from one direction, the scheduler …
EDGE
Read/write inversion on SSDsmq-deadline was designed for rotational media where sequential access is faster;
Write starvation under heavy read loadIf `writes_starved` is set too high, write requests can be delayed far beyond th
FIFO queue ordering on multi-queue devicesWith multiple hardware queues, the FIFO ordering guarantee breaks across queues;
Deadline expiry on power-loss-safe SSDsPower-loss-safe NVMe drives buffer writes in DRAM; a "written" request may not r
5.4
Kyber I/O Scheduler (`block/kyber-iosched.c`)
Kyber is a lightweight I/O scheduler designed for fast storage (NVMe, optane).
▾
Definition
Kyber is a lightweight I/O scheduler designed for fast storage (NVMe, optane). Rather than sorting by LBA, it uses token-based rate limiting across two priority domains: "synchronous read" (latency-sensitive) and "other" (background). It targets specific per-domain latency goals and throttles the lower-priority domain when target latencies are exceeded.
Execution Flow
- Requests are classified into domains:
KYBER_READ (sync reads) or KYBER_WRITE (writes/async).
kyber_bio_merge() attempts to merge incoming bios with pending requests.
kyber_dispatch_request() acquires a token from the per-domain sbitmap_queue (a semaphore-like token bucket per hardware queue context).
- If tokens are available for the high-priority domain, its requests are dispatched first.
- The latency sampler (
kyber_latency_timer_fn()) runs periodically: measures recent per-domain completion latencies against read_lat_nsec and write_lat_nsec targets.
- If a domain's latency exceeds its target,
kyber_adjust_rq_dispatches() increases that domain's token budget; if latency is well below target, tokens are reduced.
- This feedback loop maintains target latencies dynamically without explicit request sorting.
Edge Cases & Failure Modes
- Token starvation feedback loop: If the latency target is set too aggressively (e.g., 100 µs for a device with 200 µs baseline), the latency controller continuously over-provisions tokens, overloading the device queue and worsening latency.
- Domain mis-classification: Writes issued as
REQ_SYNC (e.g., fsync metadata writes) are classified as synchronous but share a token bucket with background writes, defeating prioritization.
- Low queue depth devices: On rotational or emulated NVMe devices with small queue depths (QD=1), Kyber's token system does not improve over
none; the hardware serializes everything regardless.
- Multi-actuator drive unsupported: New multi-actuator HDDs (e.g., Seagate MACH.2) present multiple independent actuators; Kyber has no actuator-aware dispatch path, treating both actuators as a single queue.
DEF
Kyber is a lightweight I/O scheduler designed for fast storage (NVMe, optane).
FLOW
Requests are classified into domains: `KYBER_READ` (syn…`kyber_bio_merge()` attempts to merge incoming bios wit…`kyber_dispatch_request()` acquires a token from the pe…If tokens are available for the high-priority domain, i…The latency sampler (`kyber_latency_timer_fn()`) runs p…If a domain's latency exceeds its target, `kyber_adjust…This feedback loop maintains target latencies dynamical…
EDGE
Token starvation feedback loopIf the latency target is set too aggressively (e.g., 100 µs for a device with 20
Domain mis-classificationWrites issued as `REQ_SYNC` (e.g., `fsync` metadata writes) are classified as sy
Low queue depth devicesOn rotational or emulated NVMe devices with small queue depths (QD=1), Kyber's t
Multi-actuator drive unsupportedNew multi-actuator HDDs (e.g., Seagate MACH.2) present multiple independent actu
5.5
None / NOOP I/O Scheduler (`block/noop-iosched.c` / `block/mq-deadline.c`)
The `none` scheduler (blk-mq era) passes requests directly from the blk-mq software queues to the hardware dispatch q…
▾
Definition
The none scheduler (blk-mq era) passes requests directly from the blk-mq software queues to the hardware dispatch queues in FIFO order with minimal overhead. It performs no sorting, no deadline enforcement, and no prioritization. Historically called noop in the single-queue era. Optimal for devices with their own sophisticated internal schedulers (NVMe controllers, SSDs with FTL).
Execution Flow
bio → blk_mq_submit_bio() → request allocation.
noop_insert_request() appends the request to the per-hardware-queue dispatch list (hctx->dispatch).
noop_dispatch_request() dequeues from the front of the list (pure FIFO).
- The request is passed to the driver's
queue_rq().
- No latency tracking, no reordering, no merging decisions beyond bio merging at the blk-mq layer.
- Completion:
blk_mq_complete_request() → bio->bi_end_io().
Edge Cases & Failure Modes
- Request ordering on rotational media: Using
none on a spinning hard disk causes random-seek workloads with severe throughput degradation (e.g., 300 IOPS vs. 150 IOPS for sequential); mq-deadline or BFQ should be used instead.
- Write ordering for fsync correctness: The
none scheduler does not enforce write ordering beyond what the driver provides; on devices without FUA (Force Unit Access) support, fsync may complete before data is persistent.
- Starvation of low-priority requests: Without any fairness mechanism, a high-throughput process issuing many requests can delay latency-sensitive requests indefinitely; the
none scheduler has no mitigation.
- Hot-swap scheduler (online change): The scheduler can be changed at runtime via
/sys/block/sdX/queue/scheduler; changing from BFQ to none on a busy disk flushes in-flight BFQ queues immediately, which can cause a brief I/O pause.
DEF
The `none` scheduler (blk-mq era) passes requests directly from the blk-mq software queues to the hardware dispatch q…
FLOW
`bio` → `blk_mq_submit_bio()` → request allocation.`noop_insert_request()` appends the request to the per-…`noop_dispatch_request()` dequeues from the front of th…The request is passed to the driver's `queue_rq()`.No latency tracking, no reordering, no merging decision…Completion: `blk_mq_complete_request()` → `bio->bi_end_…
EDGE
Request ordering on rotational mediaUsing `none` on a spinning hard disk causes random-seek workloads with severe th
Write ordering for fsync correctnessThe `none` scheduler does not enforce write ordering beyond what the driver prov
Starvation of low-priority requestsWithout any fairness mechanism, a high-throughput process issuing many requests
Hot-swap scheduler (online change)The scheduler can be changed at runtime via `/sys/block/sdX/queue/scheduler`; ch
5.6
`bio` Layer (`block/bio.c`)
A `bio` (Block I/O) is the fundamental unit of I/O in the Linux block layer.
▾
Definition
A bio (Block I/O) is the fundamental unit of I/O in the Linux block layer. It describes a scatter-gather I/O operation: a list of bio_vec entries (each specifying a page, offset, and length), a target block device and start sector, and direction flags. The bio layer handles splitting, merging, and cloning of I/O operations before they reach the I/O scheduler.
Execution Flow
bio_alloc(gfp, nr_vecs) allocates a bio with space for nr_vecs page vectors from the bio slab or mempool.
bio_add_page(bio, page, len, offset) adds a page vector entry; if the new entry is physically contiguous with the last, they are merged into one bio_vec.
submit_bio(bio) enters the block layer: blk_mq_submit_bio() checks queue limits (max sectors, max segments).
- If the bio exceeds limits:
bio_split() creates two bios; the second is resubmitted recursively.
- Bio merging:
blk_attempt_plug_merge() tries to merge the new bio with an existing request on the plug list (front/back merge by sector adjacency).
- After merging decisions, the bio is converted to a
request and enters the I/O scheduler.
- On completion: the driver calls
bio_endio(bio, error) → bio->bi_end_io(bio).
- For split bios:
bio_chain() links parent/child; the parent's bi_end_io fires only after all children complete.
Edge Cases & Failure Modes
- bio_vec segment limit overflow: A bio is bounded by
queue_max_segments(); exceeding it requires splitting. Drivers that don't check the bio segment count before DMA mapping will produce DMA errors or memory corruption.
- High-memory page mapping: On 32-bit kernels,
bio_vec pages in ZONE_HIGHMEM require kmap_atomic() to access; failing to unmap causes permanent kmap slot exhaustion.
- bio cloning for mirroring (DM/MD):
bio_clone_fast() creates a bio that shares the original's page vector but has an independent completion; mishandling the clone's reference counting causes use-after-free in the original bio's page data.
- Partial completion handling: A device that completes only part of a bio (e.g., a device with a sector boundary error) calls
bio_advance() to update the remaining range; callers that don't handle partial completions may submit the already-completed range again.
DEF
A `bio` (Block I/O) is the fundamental unit of I/O in the Linux block layer.
FLOW
`bio_alloc(gfp, nr_vecs)` allocates a bio with space fo…`bio_add_page(bio, page, len, offset)` adds a page vect…`submit_bio(bio)` enters the block layer: `blk_mq_submi…If the bio exceeds limits: `bio_split()` creates two bi…Bio merging: `blk_attempt_plug_merge()` tries to merge …After merging decisions, the bio is converted to a `req…On completion: the driver calls `bio_endio(bio, error)`…For split bios: `bio_chain()` links parent/child; the p…
EDGE
bio_vec segment limit overflowA bio is bounded by `queue_max_segments()`; exceeding it requires splitting. Dri
High-memory page mappingOn 32-bit kernels, `bio_vec` pages in `ZONE_HIGHMEM` require `kmap_atomic()` to
bio cloning for mirroring (DM/MD)`bio_clone_fast()` creates a bio that shares the original's page vector but has
Partial completion handlingA device that completes only part of a bio (e.g., a device with a sector boundar
5.7
Request Queue (`block/blk-core.c`, `include/linux/blkdev.h`)
The `request_queue` (`struct request_queue`) is the central data structure of the block layer for a given block device.
▾
Definition
The request_queue (struct request_queue) is the central data structure of the block layer for a given block device. It holds the blk-mq tag sets, hardware queue contexts (blk_mq_hw_ctx), queue limits (max sectors, max segments, logical/physical block size), and the I/O scheduler state. All block I/O operations for a device are funneled through its request_queue.
Execution Flow
- At device registration:
blk_mq_init_queue() allocates a request_queue and sets its limits from blk_mq_tag_set (driver-provided: queue depth, number of hardware queues, etc.).
blk_queue_max_hw_sectors(), blk_queue_physical_block_size(), etc., configure the queue's operational limits.
- An I/O scheduler is attached:
elevator_init_mq() sets q->elevator.
- I/O submission:
submit_bio() → blk_mq_submit_bio() → routes through q->mq_ops->queue_rq().
- Queue freeze (
blk_freeze_queue()): stops new I/O submission and waits for in-flight I/O to drain (used during driver suspend, error recovery, scheduler change).
- Queue drain (
blk_drain_queue()): waits for all requests to complete.
- Queue teardown (
blk_cleanup_queue()): on device removal, drains the queue, destroys tag sets, frees the request_queue.
Edge Cases & Failure Modes
- Queue limits violation: A filesystem that caches
queue_max_sectors() at mount time and doesn't re-query after a live queue limits change (e.g., after a hardware reconfiguration) may submit oversized bios that require splitting, reducing performance.
- Request queue stall on tag exhaustion: When all blk-mq tags are in-flight, new requests block; if the device never completes a request (firmware hang), the queue stalls permanently.
blk_abort_request() and device reset paths exist but are complex.
- Queue freeze deadlock: Calling
blk_freeze_queue() from within a bio completion callback deadlocks: completion callbacks run while the queue is active; a freeze waiting for active I/O to drain is blocked by the caller holding the callback context.
- Rotational flag and elevator selection: Setting
q->rotational = 0 signals that the device is non-rotational; the kernel uses this to select a default scheduler (none for NVMe, mq-deadline for rotational). An incorrect flag causes the wrong default scheduler.
DEF
The `request_queue` (`struct request_queue`) is the central data structure of the block layer for a given block device.
FLOW
At device registration: `blk_mq_init_queue()` allocates…`blk_queue_max_hw_sectors()`, `blk_queue_physical_block…An I/O scheduler is attached: `elevator_init_mq()` sets…I/O submission: `submit_bio()` → `blk_mq_submit_bio()` …Queue freeze (`blk_freeze_queue()`): stops new I/O subm…Queue drain (`blk_drain_queue()`): waits for all reques…Queue teardown (`blk_cleanup_queue()`): on device remov…
EDGE
Queue limits violationA filesystem that caches `queue_max_sectors()` at mount time and doesn't re-quer
Request queue stall on tag exhaustionWhen all blk-mq tags are in-flight, new requests block; if the device never comp
Queue freeze deadlockCalling `blk_freeze_queue()` from within a bio completion callback deadlocks: co
Rotational flag and elevator selectionSetting `q->rotational = 0` signals that the device is non-rotational; the kerne
5.8
I/O Barriers / Write Ordering (`block/blk-flush.c`)
I/O barriers (now implemented as flush sequences in blk-mq) ensure write ordering and durability.
▾
Definition
I/O barriers (now implemented as flush sequences in blk-mq) ensure write ordering and durability. A "flush" operation forces the device's write cache to persistent storage. A "FUA" (Force Unit Access) write bypasses the write cache for a single request. These are used by filesystems (ext4, XFS, btrfs) and databases to implement journaling and fsync guarantees.
Execution Flow
- A filesystem calls
blkdev_issue_flush() or submits a bio with REQ_PREFLUSH | REQ_FUA.
blk_insert_flush() checks the device's flush/FUA capabilities in q->queue_flags (QUEUE_FLAG_WC = write cache, QUEUE_FLAG_FUA = FUA support).
- If neither is supported: the flush is a no-op (all writes are already cache-less).
- If FUA is supported but
REQ_PREFLUSH is set: a FLUSH command is dispatched first; the data write follows with the FUA bit set in the command.
- The flush sequence state machine (
blk_flush_complete_seq()) processes three phases: REQ_FSEQ_PREFLUSH (send flush), REQ_FSEQ_DATA (send data writes), REQ_FSEQ_POSTFLUSH (optional trailing flush).
- Flush coalescing: multiple concurrent flush requests are coalesced into one hardware flush command to reduce overhead.
- On flush completion:
blk_flush_complete_seq() advances all coalesced requests to the data phase.
Edge Cases & Failure Modes
- Flush coalescing window race: Two
fsync() calls that arrive nanoseconds apart may land in different flush batches; the second process may wait for a full extra flush cycle, adding latency.
- Device reports FUA support but doesn't implement it: Some storage devices (especially in virtual environments) advertise FUA support but handle it as a no-op; writes "completed" with FUA may not be persistent on power loss. Databases have been bitten by this.
REQ_PREFLUSH overhead on tmpfs: Filesystems mounted on tmpfs or ramfs bypass the block layer entirely; blkdev_issue_flush() returns immediately without a syscall. Callers must check the mount type.
- Flush timeout on faulty device: If a device takes too long to complete a cache flush (e.g., a failing SSD with a large write buffer), the block layer timeout fires, the request is aborted, and the filesystem sees an I/O error on
fsync().
DEF
I/O barriers (now implemented as flush sequences in blk-mq) ensure write ordering and durability.
FLOW
A filesystem calls `blkdev_issue_flush()` or submits a …`blk_insert_flush()` checks the device's flush/FUA capa…If neither is supported: the flush is a no-op (all writ…If FUA is supported but `REQ_PREFLUSH` is set: a FLUSH …The flush sequence state machine (`blk_flush_complete_s…Flush coalescing: multiple concurrent flush requests ar…On flush completion: `blk_flush_complete_seq()` advance…
EDGE
Flush coalescing window raceTwo `fsync()` calls that arrive nanoseconds apart may land in different flush ba
Device reports FUA support but doesn't implement itSome storage devices (especially in virtual environments) advertise FUA support
`REQ_PREFLUSH` overhead on tmpfsFilesystems mounted on `tmpfs` or `ramfs` bypass the block layer entirely; `blkd
Flush timeout on faulty deviceIf a device takes too long to complete a cache flush (e.g., a failing SSD with a
5.9
Device Mapper (`drivers/md/dm.c`)
Device Mapper (DM) is a kernel framework that creates virtual block devices by mapping I/O operations to one or more …
▾
Definition
Device Mapper (DM) is a kernel framework that creates virtual block devices by mapping I/O operations to one or more underlying real block devices according to configurable mapping tables. It underpins LVM2 (logical volume management), dm-crypt (full-disk encryption), dm-multipath (multipath I/O), and dm-thin (thin provisioning).
Execution Flow
- A DM device is created:
dm_create() allocates a mapped_device and a gendisk; a mapping table is loaded via ioctl(DM_TABLE_LOAD_CMD).
- The table is a list of
dm_target entries, each covering a sector range and naming a target type (e.g., linear, crypt, thin).
- I/O submission:
dm_make_request() intercepts bios destined for the DM device.
dm_map_bio() looks up the target for the bio's start sector in the table's sector-range tree.
- The target's
.map() callback transforms the bio: remaps the sector number and device pointer (for linear), applies encryption (for crypt), or handles copy-on-write (for snapshot).
- The transformed bio is re-submitted to the underlying device via
submit_bio().
- On completion: the target's optional
.end_io() callback runs; then dm_io_dec_pending() decrements the reference count and completes the original bio.
Edge Cases & Failure Modes
- Table reload race:
dm_swap_table() replaces the active mapping table atomically; in-flight I/O on the old table continues using the old target references until all I/O completes. A dm_suspend() / dm_resume() cycle is needed for a clean swap.
- dm-crypt bio splitting overhead: dm-crypt splits bios to align with its crypto block size; on workloads with many small I/Os, this produces 2× the number of bios and can saturate the dm-crypt workqueue.
- dm-thin provisioning ENOSPC: When the thin pool runs out of space, new writes to thin volumes return
EIO immediately; the filesystem must handle this gracefully (most ext4/XFS implementations do, but applications that assume ENOSPC is impossible may crash).
- dm-multipath path failure during I/O: If a path fails while I/O is in-flight on it, dm-multipath must requeue the in-flight I/O on another path; some storage arrays have windows where both the original and retry I/O complete, causing duplicate writes.
DEF
Device Mapper (DM) is a kernel framework that creates virtual block devices by mapping I/O operations to one or more …
FLOW
A DM device is created: `dm_create()` allocates a `mapp…The table is a list of `dm_target` entries, each coveri…I/O submission: `dm_make_request()` intercepts bios des…`dm_map_bio()` looks up the target for the bio's start …The target's `.map()` callback transforms the bio: rema…The transformed bio is re-submitted to the underlying d…On completion: the target's optional `.end_io()` callba…
EDGE
Table reload race`dm_swap_table()` replaces the active mapping table atomically; in-flight I/O on
dm-crypt bio splitting overheaddm-crypt splits bios to align with its crypto block size; on workloads with many
dm-thin provisioning ENOSPCWhen the thin pool runs out of space, new writes to thin volumes return `EIO` im
dm-multipath path failure during I/OIf a path fails while I/O is in-flight on it, dm-multipath must requeue the in-f
5.10
MD RAID Layer (`drivers/md/raid*.c`, `drivers/md/md.c`)
The MD (Multiple Devices) RAID layer implements software RAID levels (0 through 10 and their variants) entirely in th…
▾
Definition
The MD (Multiple Devices) RAID layer implements software RAID levels (0 through 10 and their variants) entirely in the kernel. Each MD array is a virtual block device with a stripe-based mapping across N physical member devices. MD handles data striping, mirroring, parity computation (for RAID-5/6), resync, reshape, and recovery from device failures.
Execution Flow
- Array assembly:
md_run() reads superblocks from each member device, validates the configuration, and starts the RAID personality (raid5_run(), raid1_run(), etc.).
- I/O submission:
md_make_request() intercepts bios; the RAID personality's make_request() (e.g., raid5_make_request()) determines the stripe number and position.
- For RAID-5: a
stripe_head is allocated for the target stripe; partial writes trigger a read-modify-write cycle (read old data and parity, XOR in new data, write new data + new parity).
- The stripe cache (hash table of
stripe_head objects) coalesces multiple bios targeting the same stripe.
- Completed stripe I/O:
raid5_end_read_request() / raid5_end_write_request() update stripe state; when all required I/O is done, handle_stripe() advances the stripe state machine.
- Device failure:
md_error() marks the failing device; RAID-1/5/6 continues degraded; md_check_recovery() starts a resync to rebuild the missing data on a hot spare.
Edge Cases & Failure Modes
- RAID-5 write hole: If power fails between writing the new data chunk and writing the new parity, the array is inconsistent; the kernel mitigates this with journal devices (
raid5_journal) or by writing a "partial parity log" (PPL).
- Stripe cache deadlock: If the stripe cache is full and all stripes are waiting for I/O that itself requires new stripe cache entries, the array deadlocks; the kernel limits in-flight stripes via the
stripe_hash_locks / conf->max_nr_stripes semaphore.
- Bitmap-assisted resync: The write-intent bitmap tracks which stripe regions need resync after a crash; a corrupted bitmap causes a full resync on every boot, adding hours of latency.
- RAID-6 double-disk failure during resync: If a second disk fails while the array is degraded (one disk already failed), RAID-6 can still recover; RAID-5 cannot and marks the array as "failed," making data inaccessible until the failed disks are restored.
DEF
The MD (Multiple Devices) RAID layer implements software RAID levels (0 through 10 and their variants) entirely in th…
FLOW
Array assembly: `md_run()` reads superblocks from each …I/O submission: `md_make_request()` intercepts bios; th…For RAID-5: a `stripe_head` is allocated for the target…The stripe cache (hash table of `stripe_head` objects) …Completed stripe I/O: `raid5_end_read_request()` / `rai…Device failure: `md_error()` marks the failing device; …
EDGE
RAID-5 write holeIf power fails between writing the new data chunk and writing the new parity, th
Stripe cache deadlockIf the stripe cache is full and all stripes are waiting for I/O that itself requ
Bitmap-assisted resyncThe write-intent bitmap tracks which stripe regions need resync after a crash; a
RAID-6 double-disk failure during resyncIf a second disk fails while the array is degraded (one disk already failed), RA
📂
S6 — Virtual Filesystems (VFS)
10 components
6.1
Dentry Cache (dcache) (`fs/dcache.c`)
The dentry cache is an in-memory cache of directory entry objects (`struct dentry`), each mapping a filename componen…
▾
Definition
The dentry cache is an in-memory cache of directory entry objects (struct dentry), each mapping a filename component to its inode. It implements the VFS name lookup fast path: lookup_fast() checks the dcache hash table before hitting the filesystem. The cache is bounded by memory and uses LRU eviction under pressure.
Execution Flow
- Pathname lookup begins:
path_lookupat() calls walk_component() for each path component.
lookup_fast() hashes the (parent dentry, name) pair and checks dentry_hashtable with RCU read lock.
- Cache hit: a dentry is found and reference-counted via
lockref_get_not_dead(); the lookup returns immediately without filesystem I/O.
- Cache miss:
lookup_slow() acquires parent->d_inode->i_rwsem, calls d_lookup() again under lock (to close a race), then calls the filesystem's inode_operations->lookup().
- The filesystem reads the directory entry and calls
d_add() or d_splice_alias() to insert a new dentry into the cache.
- Dentry eviction: under memory pressure,
shrink_dcache_sb() / dcache_shrinker calls dentry_kill() on unused (zero-refcount) dentries in LRU order.
- Negative dentries (for nonexistent files) are also cached to avoid repeated "file not found" disk lookups; they expire via timer or explicit
d_invalidate().
Edge Cases & Failure Modes
- Dentry leaks: A module that calls
d_lookup() without a matching dput() increments the dentry's reference count permanently; the dentry is never evicted, slowly growing the dcache without bound.
- Stale negative dentry after create: If a negative dentry (caching "file not found") exists and the file is then created on the same mount, the negative dentry must be invalidated; NFS-mounted directories with dcache_invalidate disabled may serve stale negatives for the full
actimeo timeout.
- Alias dentries on hardlinks: Multiple dentries can point to the same inode (hardlinks);
d_find_alias() finds all aliases; operations that must be coherent across aliases (e.g., rename()) must lock all aliases.
- Dentry hash table scalability: Under extreme parallelism (millions of files, many threads doing concurrent lookups), the global dentry hash table lock becomes a scalability bottleneck; RCU-based lookup mitigates but does not eliminate this.
DEF
The dentry cache is an in-memory cache of directory entry objects (`struct dentry`), each mapping a filename componen…
FLOW
Pathname lookup begins: `path_lookupat()` calls `walk_c…`lookup_fast()` hashes the (parent dentry, name) pair a…**Cache hit**: a dentry is found and reference-counted …**Cache miss**: `lookup_slow()` acquires `parent->d_ino…The filesystem reads the directory entry and calls `d_a…Dentry eviction: under memory pressure, `shrink_dcache_…Negative dentries (for nonexistent files) are also cach…
EDGE
Dentry leaksA module that calls `d_lookup()` without a matching `dput()` increments the dent
Stale negative dentry after createIf a negative dentry (caching "file not found") exists and the file is then crea
Alias dentries on hardlinksMultiple dentries can point to the same inode (hardlinks); `d_find_alias()` find
Dentry hash table scalabilityUnder extreme parallelism (millions of files, many threads doing concurrent look
6.2
Inode Cache (`fs/inode.c`)
The inode cache stores `struct inode` objects in memory, representing files and directories independently of their na…
▾
Definition
The inode cache stores struct inode objects in memory, representing files and directories independently of their names. Each inode holds file metadata (size, permissions, timestamps, block pointers) and operation function pointer tables. The cache uses a hash table (keyed by sb + ino) and an LRU list for eviction.
Execution Flow
iget_locked(sb, ino) checks inode_hashtable for an existing inode; if found, returns a locked reference.
- On cache miss:
alloc_inode() → sb->s_op->alloc_inode() allocates a filesystem-specific inode struct (embedded or separate from struct inode).
- The inode is added to the hash table and marked
I_NEW; the caller must call sb->s_op->read_inode() (or its equivalent) to populate the inode from disk.
unlock_new_inode() clears I_NEW, waking any waiters that were blocked on wait_on_inode().
- Normal usage:
iget_locked() returns the locked inode; the caller increments its reference count via ihold() and stores it in a dentry.
- When all references drop:
iput_final() calls sb->s_op->evict_inode() to write back dirty data and release filesystem-specific state.
- The inode struct is then returned to the inode cache's slab (
inode_cachep) or freed if the cache is under pressure.
Edge Cases & Failure Modes
- Inode number reuse race: After
iput() removes an inode from the cache, a new file created on the filesystem may reuse the same inode number; a concurrent iget_locked() call racing with the eviction may return the old, partially-cleared inode.
I_DIRTY writeback ordering: An inode is marked I_DIRTY_DATASYNC on a metadata write; if evict_inode() is called before writeback completes, write_inode_now() must be called to flush dirty metadata.
- Filesystem-specific inode allocation failure: If
s_op->alloc_inode() fails (OOM), iget_locked() returns ERR_PTR(-ENOMEM); callers that don't check the error create a NULL pointer dereference.
- Inode cache over-retention: Large filesystems with millions of inodes can fill the inode cache; the
inode_shrinker evicts inodes but can be slow; a "dentry/inode storm" (rapidly creating and deleting files) can exhaust the slab allocator.
DEF
The inode cache stores `struct inode` objects in memory, representing files and directories independently of their na…
FLOW
`iget_locked(sb, ino)` checks `inode_hashtable` for an …On cache miss: `alloc_inode()` → `sb->s_op->alloc_inode…The inode is added to the hash table and marked `I_NEW`…`unlock_new_inode()` clears `I_NEW`, waking any waiters…Normal usage: `iget_locked()` returns the locked inode;…When all references drop: `iput_final()` calls `sb->s_o…The inode struct is then returned to the inode cache's …
EDGE
Inode number reuse raceAfter `iput()` removes an inode from the cache, a new file created on the filesy
`I_DIRTY` writeback orderingAn inode is marked `I_DIRTY_DATASYNC` on a metadata write; if `evict_inode()` is
Filesystem-specific inode allocation failureIf `s_op->alloc_inode()` fails (OOM), `iget_locked()` returns `ERR_PTR(-ENOMEM)`
Inode cache over-retentionLarge filesystems with millions of inodes can fill the inode cache; the `inode_s
6.3
Page Cache (`mm/filemap.c`)
The page cache (address space cache) stores filesystem pages in memory indexed by `(inode, page_index)` pairs in a pe…
▾
Definition
The page cache (address space cache) stores filesystem pages in memory indexed by (inode, page_index) pairs in a per-inode address_space XArray. It acts as the read/write cache for file I/O: reads check the cache first; writes go into cache (dirty) and are written back asynchronously by pdflush/writeback threads. It is also used for memory-mapped files.
Execution Flow
filemap_get_pages() (called from generic_file_read_iter()) looks up the page in inode->i_mapping XArray.
- Cache hit: the page is returned; its reference count is incremented.
- Cache miss:
page_cache_alloc() allocates a new page; add_to_page_cache_lru() inserts it into the XArray and the LRU list.
a_ops->readpage() / readahead() submits I/O to populate the page from disk.
- The calling task waits on the page's
PG_locked bit via wait_on_page_locked(); when I/O completes, the driver calls unlock_page().
- The page is returned;
copy_page_to_iter() copies data to the userspace buffer.
- Write path:
generic_perform_write() → a_ops->write_begin() locks the page; data is copied from userspace; a_ops->write_end() marks the page PG_dirty.
balance_dirty_pages_ratelimited() throttles the writer if dirty ratio thresholds are exceeded.
- Writeback:
wb_writeback() calls a_ops->writepages() to submit dirty pages to the block layer.
Edge Cases & Failure Modes
- Page cache coherence with direct I/O:
O_DIRECT bypasses the page cache; a file opened with O_DIRECT while another fd has it open with buffered I/O can read stale cached data. The kernel partially invalidates the cache before direct I/O but cannot guarantee coherence during concurrent access.
- Dirty ratio threshold exhaustion: If the dirty ratio (
vm.dirty_ratio) is exceeded, the writing process is throttled in balance_dirty_pages(); under extreme write pressure, all processes writing to disk may be throttled simultaneously, causing application-visible stalls.
- Page writeback error handling: If writeback fails (e.g., network filesystem timeout),
mapping_set_error() records the error; subsequent fsync()/close() calls return the error. But if the error is not consumed between two close() calls, the second call may succeed without seeing the error (Linux 5.1+ errseq_t fixes this).
- Page eviction under mmap: A page that is simultaneously in the page cache and mapped into a process's address space cannot be evicted until the mapping is removed;
try_to_unmap() must succeed before shrink_page_list() can reclaim it.
DEF
The page cache (address space cache) stores filesystem pages in memory indexed by `(inode, page_index)` pairs in a pe…
FLOW
`filemap_get_pages()` (called from `generic_file_read_i…**Cache hit**: the page is returned; its reference coun…**Cache miss**: `page_cache_alloc()` allocates a new pa…`a_ops->readpage()` / `readahead()` submits I/O to popu…The calling task waits on the page's `PG_locked` bit vi…The page is returned; `copy_page_to_iter()` copies data…Write path: `generic_perform_write()` → `a_ops->write_b…`balance_dirty_pages_ratelimited()` throttles the write…Writeback: `wb_writeback()` calls `a_ops->writepages()`…
EDGE
Page cache coherence with direct I/O`O_DIRECT` bypasses the page cache; a file opened with `O_DIRECT` while another
Dirty ratio threshold exhaustionIf the dirty ratio (`vm.dirty_ratio`) is exceeded, the writing process is thrott
Page writeback error handlingIf writeback fails (e.g., network filesystem timeout), `mapping_set_error()` rec
Page eviction under mmapA page that is simultaneously in the page cache and mapped into a process's addr
6.4
VFS `open()` Path (`fs/open.c`, `fs/namei.c`)
The VFS `open()` path is the kernel's implementation of `open(2)` / `openat(2)`: it resolves a pathname to an inode, …
▾
Definition
The VFS open() path is the kernel's implementation of open(2) / openat(2): it resolves a pathname to an inode, checks permissions, allocates a struct file, invokes the filesystem's open handler, and returns a file descriptor. It is one of the most complex kernel paths, involving dentry/inode caches, LSM hooks, O_CREAT file creation, and symlink resolution.
Execution Flow
sys_openat2() → do_sys_open() → do_filp_open() → path_openat().
path_init() establishes the starting dentry (root / or current working directory).
link_path_walk() resolves each component: for each slash-delimited name, calls walk_component() → lookup_fast() → (on miss) lookup_slow().
- Symlink resolution: each symlink encountered calls
pick_link() → recursively resolves the symlink target (up to MAXSYMLINKS = 40 depth).
- At the final component: if
O_CREAT is set, atomic_open() or lookup_open() is called; if the file doesn't exist, vfs_create() → inode_ops->create() creates it.
vfs_open() → do_dentry_open(): checks inode_permission() (DAC + LSM hooks); calls f->f_op->open(inode, file).
- A file descriptor is allocated from the process's
fdtable via get_unused_fd_flags().
fd_install(fd, file) atomically places the struct file * into the fdtable.
Edge Cases & Failure Modes
- TOCTOU (time-of-check/time-of-use) races: Between
path_lookupat() and vfs_open(), the filesystem state can change; O_NOFOLLOW and openat2() with RESOLVE_NO_SYMLINKS reduce the attack surface but cannot eliminate all races.
- Too many open files:
get_unused_fd_flags() returns EMFILE when the process's fdtable is full (up to RLIMIT_NOFILE); the filesystem's inode_ops->create() may have already run, leaving a created-but-unreachable file.
O_CREAT | O_EXCL race: Two processes simultaneously calling open(path, O_CREAT|O_EXCL) on the same path; only one succeeds; the filesystem's atomic_open() or inode_ops->create() must use an atomic create mechanism to avoid creating duplicate files.
- Pathname too long: Pathname components are bounded by
NAME_MAX (255 bytes); the full path by PATH_MAX (4096 bytes); exceeding these returns ENAMETOOLONG before any filesystem I/O.
DEF
The VFS `open()` path is the kernel's implementation of `open(2)` / `openat(2)`: it resolves a pathname to an inode, …
FLOW
`sys_openat2()` → `do_sys_open()` → `do_filp_open()` → …`path_init()` establishes the starting dentry (root `/`…`link_path_walk()` resolves each component: for each sl…Symlink resolution: each symlink encountered calls `pic…At the final component: if `O_CREAT` is set, `atomic_op…`vfs_open()` → `do_dentry_open()`: checks `inode_permis…A file descriptor is allocated from the process's `fdta…`fd_install(fd, file)` atomically places the `struct fi…
EDGE
TOCTOU (time-of-check/time-of-use) racesBetween `path_lookupat()` and `vfs_open()`, the filesystem state can change; `O_
Too many open files`get_unused_fd_flags()` returns `EMFILE` when the process's `fdtable` is full (u
`O_CREAT | O_EXCL` raceTwo processes simultaneously calling `open(path, O_CREAT|O_EXCL)` on the same pa
Pathname too longPathname components are bounded by `NAME_MAX` (255 bytes); the full path by `PAT
6.5
VFS `read()` / `write()` (`fs/read_write.c`)
The VFS `read(2)` and `write(2)` paths dispatch file I/O through the `file->f_op->read_iter` / `write_iter` interface…
▾
Definition
The VFS read(2) and write(2) paths dispatch file I/O through the file->f_op->read_iter / write_iter interface, which all modern kernel filesystems implement. This abstraction supports regular file I/O (page cache), direct I/O, pipe I/O, and character device I/O through the same syscall interface via iov_iter.
Execution Flow
sys_read(fd, buf, count) → ksys_read() → vfs_read().
file_pos_read(f) reads and locks the current file position (f->f_pos).
rw_verify_area(): verifies the file is open for the requested direction; checks mandatory locks; calls security_file_permission() (LSM hook).
call_read_iter(f, &kiocb, &iter) → f->f_op->read_iter(&kiocb, &iter).
- For regular files:
generic_file_read_iter() → page cache lookup → readahead if needed → copy_page_to_iter() to userspace buffer.
- For O_DIRECT:
generic_file_direct_write() submits bios directly, bypassing the page cache.
file_pos_write(f, pos) updates f->f_pos under f->f_lock.
- Returns bytes transferred (may be less than
count for signals, EOF, pipe full, etc.).
Edge Cases & Failure Modes
- Short read: A read can return fewer bytes than requested due to EOF, signal interruption, or pipe underflow; callers (especially library code) must loop until the full count is read or an error occurs.
f_pos race on shared file descriptor: Two threads sharing the same struct file (via fork() without O_CLOEXEC) racing on f->f_pos; the kernel serializes f_pos updates with f->f_lock but the read/update window is not atomic.
O_APPEND semantics: Writes with O_APPEND seek to end-of-file and write atomically under the inode's i_mutex; on NFS, this atomicity guarantee may be lost if the server reboots between the seek and the write.
- Blocking
read() on a pipe: A read() from an empty pipe blocks until data is available or all write ends are closed; if the writer is killed, read() returns 0 (EOF); a signal to the reader returns -EINTR.
DEF
The VFS `read(2)` and `write(2)` paths dispatch file I/O through the `file->f_op->read_iter` / `write_iter` interface…
FLOW
`sys_read(fd, buf, count)` → `ksys_read()` → `vfs_read(…`file_pos_read(f)` reads and locks the current file pos…`rw_verify_area()`: verifies the file is open for the r…`call_read_iter(f, &kiocb, &iter)` → `f->f_op->read_ite…For regular files: `generic_file_read_iter()` → page ca…For O_DIRECT: `generic_file_direct_write()` submits bio…`file_pos_write(f, pos)` updates `f->f_pos` under `f->f…Returns bytes transferred (may be less than `count` for…
EDGE
Short readA read can return fewer bytes than requested due to EOF, signal interruption, or
`f_pos` race on shared file descriptorTwo threads sharing the same `struct file` (via `fork()` without `O_CLOEXEC`) ra
`O_APPEND` semanticsWrites with `O_APPEND` seek to end-of-file and write atomically under the inode'
Blocking `read()` on a pipeA `read()` from an empty pipe blocks until data is available or all write ends a
6.6
Mount Subsystem (`fs/namespace.c`)
The mount subsystem manages filesystem mount points, maintaining a tree of `struct mount` objects (each containing a …
▾
Definition
The mount subsystem manages filesystem mount points, maintaining a tree of struct mount objects (each containing a vfsmount and a dentry binding point). It supports bind mounts, shared/private/slave/unbindable propagation types, mount namespaces (per-process mount views), and pivot_root() / chroot() operations.
Execution Flow
sys_mount(source, target, fstype, flags, data) → do_mount().
lookup_mnt_in_ns() looks up the target path in the current mount namespace.
- For a new filesystem mount:
vfs_kern_mount() → mount_fs() calls fstype->mount(fstype, flags, dev, data) → returns a new dentry and super_block.
graft_tree() attaches the new struct mount to the target dentry in the mount tree.
- For bind mounts (
MS_BIND): copy_tree() duplicates a subtree of the mount tree and attaches it at the target.
- Mount propagation:
propagate_mnt() propagates the new mount to all peer/slave mounts in the same propagation group.
- The new mount is visible in
/proc/self/mounts and in the VFS path walk (intercepted by follow_mount() in walk_component()).
Edge Cases & Failure Modes
- Mount namespace leak: A process that enters a mount namespace and creates mounts without calling
umount() before exiting leaves orphan mounts; they persist until the last process holding a reference to the namespace exits.
- Propagation storm: In a heavily shared mount namespace (e.g., systemd's
MS_SHARED root), every mount() call propagates to all peers; with thousands of containers sharing a namespace, one mount() call triggers thousands of propagation operations.
pivot_root() ordering requirements: pivot_root() requires the new root and old root to be on different filesystems and that the new root be a mount point; violating these constraints returns EINVAL and can leave the filesystem in a partially-pivoted state.
- Unmount with busy mountpoint:
umount() fails with EBUSY if any file or directory on the mount is in use (open files, working directories, nested mounts); MNT_FORCE can unmount NFS even with open files but may cause data loss.
DEF
The mount subsystem manages filesystem mount points, maintaining a tree of `struct mount` objects (each containing a …
FLOW
`sys_mount(source, target, fstype, flags, data)` → `do_…`lookup_mnt_in_ns()` looks up the target path in the cu…For a new filesystem mount: `vfs_kern_mount()` → `mount…`graft_tree()` attaches the new `struct mount` to the t…For bind mounts (`MS_BIND`): `copy_tree()` duplicates a…Mount propagation: `propagate_mnt()` propagates the new…The new mount is visible in `/proc/self/mounts` and in …
EDGE
Mount namespace leakA process that enters a mount namespace and creates mounts without calling `umou
Propagation stormIn a heavily shared mount namespace (e.g., systemd's `MS_SHARED` root), every `m
`pivot_root()` ordering requirements`pivot_root()` requires the new root and old root to be on different filesystems
Unmount with busy mountpoint`umount()` fails with `EBUSY` if any file or directory on the mount is in use (o
6.7
`fsync` / Writeback (`fs/sync.c`, `fs/fs-writeback.c`)
Writeback is the kernel's mechanism for asynchronously flushing dirty pages (modified file data and metadata) from th…
▾
Definition
Writeback is the kernel's mechanism for asynchronously flushing dirty pages (modified file data and metadata) from the page cache to storage. fsync(2) / fdatasync(2) are synchronous variants that guarantee all dirty data for a specific file has been written and acknowledged by the storage device before returning.
Execution Flow
fsync(fd) → vfs_fsync_range() → file->f_op->fsync(file, 0, LLONG_MAX, 0).
- The filesystem's
fsync implementation (e.g., ext4_sync_file()):
- Calls
file_write_and_wait_range() to submit and wait for all dirty data pages in the page cache.
- Logs a journal commit (for journaled filesystems: ext4, XFS, btrfs) to ensure metadata consistency.
- Calls
blkdev_issue_flush() to flush the device's write cache.
- Asynchronous writeback (background):
wb_do_writeback() runs periodically from bdi_writeback kernel threads; it calls writeback_inodes_wb() → writeback_single_inode() → do_writepages() → a_ops->writepages().
- Dirty ratio enforcement:
balance_dirty_pages() throttles the process if dirty pages exceed vm.dirty_ratio.
writeback_inodes_sb_nr() can be triggered by sync(2) or syncfs(2) to flush all dirty inodes on a superblock.
Edge Cases & Failure Modes
- fsync on rename (rename without fsync of parent): Renaming a file over another is atomic at the VFS level but requires fsyncing the parent directory to guarantee durability; many applications call
fsync(new_file_fd) but not fsync(dir_fd), so a crash can leave the directory pointing to the old file.
- Writeback loop stall: Under extreme write load,
pdflush threads cannot drain dirty pages faster than new dirty pages are created; the system enters "writeback throttling" and all writing processes may block indefinitely.
- ext4 delayed allocation + crash: With
data=ordered mode, ext4 delays block allocation until writeback; a crash between data writeback and journal commit leaves allocated but uninitialized blocks exposed; ext4's zeroing-on-allocation prevents data leakage but requires extra I/O.
- btrfs CoW writeback ordering: btrfs writes new data to new extents before updating metadata; a crash during writeback can leave unreferenced extents; the transaction commit log ensures that only fully committed trees are used on remount.
DEF
Writeback is the kernel's mechanism for asynchronously flushing dirty pages (modified file data and metadata) from th…
FLOW
`fsync(fd)` → `vfs_fsync_range()` → `file->f_op->fsync(…`.The filesystem's `fsync` implementation (e.g., `ext4_sy…Asynchronous writeback (background): `wb_do_writeback()…Dirty ratio enforcement: `balance_dirty_pages()` thrott…`writeback_inodes_sb_nr()` can be triggered by `sync(` or `syncfs(` to flush all dirty inodes on a superblock.
EDGE
fsync on rename (rename without fsync of parent)Renaming a file over another is atomic at the VFS level but requires fsyncing th
Writeback loop stallUnder extreme write load, `pdflush` threads cannot drain dirty pages faster than
ext4 delayed allocation + crashWith `data=ordered` mode, ext4 delays block allocation until writeback; a crash
btrfs CoW writeback orderingbtrfs writes new data to new extents before updating metadata; a crash during wr
6.8
`splice` / `sendfile` (`fs/splice.c`)
`splice(2)` and `sendfile(2)` implement zero-copy data transfer between file descriptors, eliminating the need to cop…
▾
Definition
splice(2) and sendfile(2) implement zero-copy data transfer between file descriptors, eliminating the need to copy data through userspace. sendfile() transfers from a file fd to a socket fd using the page cache; splice() moves data between arbitrary fds via an internal pipe buffer without touching userspace memory.
Execution Flow
sendfile(out_fd, in_fd, offset, count) → do_sendfile().
vfs_sendfile() calls in_file->f_op->splice_read() to populate pipe buffers from the source file (page cache pages are added to the pipe without copying).
splice_from_pipe() calls out_file->f_op->splice_write() on the destination socket.
- For socket send:
skb_splice_bits() maps the pipe's pages into sk_buff fragments directly, avoiding a memcpy().
- For
splice(in_fd, in_off, out_fd, out_off, len, flags):
- If
in_fd is a pipe: pages are moved from the in-pipe's buffer to the out-fd.
- If
in_fd is a file: generic_file_splice_read() fills a temporary pipe from the page cache, then drains it into the out-fd.
- Pipe buffer reference counts track when pages can be released back to the page cache.
Edge Cases & Failure Modes
- Non-splice-capable fd: Not all fd types support splice; a device file or procfs file without
f_op->splice_read falls back to default_file_splice_read(), which copies data through a temporary page buffer — no actual zero-copy.
- sendfile to non-socket:
sendfile() is defined only for socket destinations on most Linux versions; using it with a regular file destination returns EINVAL.
- Pipe buffer exhaustion:
splice() requires at least one pipe fd; the pipe's buffer (default 64 KiB, configurable via F_SETPIPE_SZ) limits the in-flight data; a full pipe blocks the writer.
- Page pinning under memory pressure: Pages used in a
splice() transfer are reference-counted to prevent eviction; a long-lived splice() operation on a large file pins many pages, potentially causing memory pressure for other workloads.
DEF
`splice(2)` and `sendfile(2)` implement zero-copy data transfer between file descriptors, eliminating the need to cop…
FLOW
`sendfile(out_fd, in_fd, offset, count)` → `do_sendfile…`vfs_sendfile()` calls `in_file->f_op->splice_read()` t…`splice_from_pipe()` calls `out_file->f_op->splice_writ…For socket send: `skb_splice_bits()` maps the pipe's pa…For `splice(in_fd, in_off, out_fd, out_off, len, flags)…Pipe buffer reference counts track when pages can be re…
EDGE
Non-splice-capable fdNot all fd types support splice; a device file or procfs file without `f_op->spl
sendfile to non-socket`sendfile()` is defined only for socket destinations on most Linux versions; usi
Pipe buffer exhaustion`splice()` requires at least one pipe fd; the pipe's buffer (default 64 KiB, con
Page pinning under memory pressurePages used in a `splice()` transfer are reference-counted to prevent eviction; a
6.9
`epoll` (`fs/eventpoll.c`)
`epoll` is a scalable I/O event notification mechanism for monitoring large numbers of file descriptors simultaneously.
▾
Definition
epoll is a scalable I/O event notification mechanism for monitoring large numbers of file descriptors simultaneously. Unlike poll(2) / select(2) (which scan all fds in O(N)), epoll maintains a Red-Black tree of monitored fds and delivers only ready events, achieving O(1) per-event overhead. It is used by high-performance servers (nginx, Redis, etc.).
Execution Flow
epoll_create1(flags) → creates an eventpoll struct and an anonymous inode/fd for it.
epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &event): adds fd to the RB tree; calls ep_insert() → add_wait_queue() to register a callback on fd's wait queue.
- When
fd becomes readable/writable: the kernel calls ep_poll_callback() → adds the epoll item to the eventpoll->rdllist (ready list) and wakes up any task blocked in epoll_wait().
epoll_wait(epfd, events, maxevents, timeout):
- Checks
ep->rdllist; if events are ready, calls ep_send_events() → copies events to userspace.
- If none ready: adds the calling task to
ep->wq, calls schedule_hrtimeout_range().
- On wake (by
ep_poll_callback()): re-checks rdllist, copies events.
EPOLLET (edge-triggered) mode: an event is delivered only once per state transition (not repeatedly while the fd remains ready); the application must drain the fd completely.
Edge Cases & Failure Modes
- Level-triggered (LT) busy loop: If the application calls
epoll_wait(), receives an EPOLLIN event, reads only part of the available data, and then calls epoll_wait() again, it immediately gets another EPOLLIN; without reading all data, this loops at 100% CPU.
- Edge-triggered missed event: In
EPOLLET mode, if the application fails to drain all data after a wakeup, it will not receive another event until more new data arrives; subsequent data may never be read (missed wakeup).
- Circular
epoll monitoring: Adding an epoll fd to itself via EPOLL_CTL_ADD is detected and returns ELOOP; more complex cycles (epollA monitors epollB which monitors epollA) are also detected via cycle detection in ep_loop_check().
EPOLLHUP and closed sockets: When a socket peer closes the connection, EPOLLHUP is delivered even if the application did not request it; failing to handle EPOLLHUP causes the application to loop on a closed fd.
DEF
`epoll` is a scalable I/O event notification mechanism for monitoring large numbers of file descriptors simultaneously.
FLOW
`epoll_create1(flags)` → creates an `eventpoll` struct …`epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &event)`: adds `fd`…When `fd` becomes readable/writable: the kernel calls `…`epoll_wait(epfd, events, maxevents, timeout)`:`EPOLLET` (edge-triggered) mode: an event is delivered …
EDGE
Level-triggered (LT) busy loopIf the application calls `epoll_wait()`, receives an `EPOLLIN` event, reads only
Edge-triggered missed eventIn `EPOLLET` mode, if the application fails to drain all data after a wakeup, it
Circular `epoll` monitoringAdding an epoll fd to itself via `EPOLL_CTL_ADD` is detected and returns `ELOOP`
`EPOLLHUP` and closed socketsWhen a socket peer closes the connection, `EPOLLHUP` is delivered even if the ap
6.10
inotify (`fs/notify/inotify/`)
inotify is the kernel's filesystem event notification system.
▾
Definition
inotify is the kernel's filesystem event notification system. It allows userspace applications to watch files and directories for events (create, delete, modify, rename, etc.) by registering watches on specific paths. The kernel delivers events via a file descriptor queue. Used by file managers, build systems, and configuration management tools.
Execution Flow
inotify_init1(flags) → creates an inotify_device and returns a fd. The fd is backed by an fsnotify_group.
inotify_add_watch(inotify_fd, path, mask): resolves path to an inode; creates a fsnotify_mark on the inode's fsnotify_marks list; registers the event mask.
- When a monitored filesystem operation occurs (e.g.,
vfs_create(), vfs_unlink()), the VFS calls fsnotify(dir, inode, mask, name).
fsnotify() iterates the inode's marks; for each matching fsnotify_group, calls group->ops->handle_event() → inotify_handle_event().
- An
inotify_event struct is queued to the group's notification_list.
- The inotify fd becomes readable;
read(inotify_fd, buf, len) dequeues events into the userspace buffer.
inotify_rm_watch(inotify_fd, wd) removes the fsnotify_mark from the inode and closes the watch.
Edge Cases & Failure Modes
- IN_Q_OVERFLOW: If events are generated faster than the application reads them, the kernel queue exceeds
inotify_max_queued_events; a single IN_Q_OVERFLOW event is delivered, and subsequent events are dropped; the application must rescan the filesystem.
- IN_IGNORED on watch removal: If a watched file is deleted, inotify delivers
IN_DELETE_SELF and then IN_IGNORED (indicating the watch was automatically removed); applications that don't handle IN_IGNORED accumulate stale watch descriptors.
- Rename across watched directories:
inotify_add_watch() watches an inode, not a path; if a file is renamed out of a watched directory into another, the watch on the old parent fires IN_MOVED_FROM and the new parent fires IN_MOVED_TO with matching cookies; applications must correlate cookies to track renames.
- Hardlink creation: Creating a hardlink to a watched file notifies the directory where the link is created (
IN_CREATE) but not the inode itself; applications tracking all names for a file must monitor both the inode and all parent directories.
DEF
inotify is the kernel's filesystem event notification system.
FLOW
`inotify_init1(flags)` → creates an `inotify_device` an…`inotify_add_watch(inotify_fd, path, mask)`: resolves `…When a monitored filesystem operation occurs (e.g., `vf…`fsnotify()` iterates the inode's marks; for each match…An `inotify_event` struct is queued to the group's `not…The inotify fd becomes readable; `read(inotify_fd, buf,…`inotify_rm_watch(inotify_fd, wd)` removes the fsnotify…
EDGE
IN_Q_OVERFLOWIf events are generated faster than the application reads them, the kernel queue
IN_IGNORED on watch removalIf a watched file is deleted, inotify delivers `IN_DELETE_SELF` and then `IN_IGN
Rename across watched directories`inotify_add_watch()` watches an inode, not a path; if a file is renamed out of
Hardlink creationCreating a hardlink to a watched file notifies the directory where the link is c
🌐
S7 — Network Stack Hooks
10 components
7.1
Netfilter Hook Framework (`net/netfilter/core.c`)
Netfilter is the kernel's packet filtering and manipulation framework.
▾
Definition
Netfilter is the kernel's packet filtering and manipulation framework. It defines five hook points in the IPv4/IPv6 packet processing path (NF_INET_PRE_ROUTING, NF_INET_LOCAL_IN, NF_INET_FORWARD, NF_INET_LOCAL_OUT, NF_INET_POST_ROUTING) where registered hook functions (from iptables, nftables, conntrack, etc.) can inspect, modify, accept, or drop packets.
Execution Flow
- A packet arrives at a network interface:
netif_receive_skb() → ip_rcv() for IPv4.
ip_rcv() calls NF_HOOK(NFPROTO_IPV4, NF_INET_PRE_ROUTING, skb, in, NULL, ip_rcv_finish).
nf_hook_slow() iterates the registered hook list for this protocol+hooknum combination (sorted by priority).
- Each hook function is called:
hook->hook(priv, skb, state). Returns one of: NF_ACCEPT, NF_DROP, NF_STOLEN, NF_QUEUE, NF_REPEAT.
NF_ACCEPT: proceed to next hook or resume processing with ip_rcv_finish().
NF_DROP: kfree_skb() discards the packet; processing ends.
NF_QUEUE: packet is enqueued to nfnetlink_queue for userspace inspection; processing suspends until verdict is returned.
- After all pre-routing hooks:
ip_rcv_finish() performs routing lookup and continues to NF_INET_LOCAL_IN or NF_INET_FORWARD.
Edge Cases & Failure Modes
- Hook registration order: Hooks are sorted by numerical priority at registration time; a module that registers at the wrong priority level sees packets in the wrong order relative to conntrack or NAT, causing incorrect filter decisions.
NF_STOLEN memory leak: A hook returning NF_STOLEN claims ownership of the skb; if the hook function then loses the skb without freeing it, the packet is leaked permanently.
- Hook list modification while in use: A module calling
nf_unregister_net_hook() while packets are traversing the hook list is safe (RCU-protected list), but the unregistered hook may still be executing on another CPU; synchronize_net() must be called before the hook function's module is unloaded.
NF_QUEUE verdict timeout: If a userspace nfnetlink_queue process crashes or hangs, queued packets are never verdicted; they accumulate in memory until the queue limit is reached, after which new packets are dropped with NF_DROP.
DEF
Netfilter is the kernel's packet filtering and manipulation framework.
FLOW
A packet arrives at a network interface: `netif_receive…2. `ip_rcv()` calls `NF_HOOK(NFPROTO_IPV4, NF_INET_PRE_…`nf_hook_slow()` iterates the registered hook list for …Each hook function is called: `hook->hook(priv, skb, st…`NF_ACCEPT`: proceed to next hook or resume processing …`NF_DROP`: `kfree_skb()` discards the packet; processin…`NF_QUEUE`: packet is enqueued to `nfnetlink_queue` for…After all pre-routing hooks: `ip_rcv_finish()` performs…
EDGE
Hook registration orderHooks are sorted by numerical priority at registration time; a module that regis
`NF_STOLEN` memory leakA hook returning `NF_STOLEN` claims ownership of the `skb`; if the hook function
Hook list modification while in useA module calling `nf_unregister_net_hook()` while packets are traversing the hoo
`NF_QUEUE` verdict timeoutIf a userspace `nfnetlink_queue` process crashes or hangs, queued packets are ne
7.2
iptables (`net/ipv4/netfilter/ip_tables.c`)
iptables is the classic packet filter that uses Netfilter hooks to implement stateless and stateful packet filtering,…
▾
Definition
iptables is the classic packet filter that uses Netfilter hooks to implement stateless and stateful packet filtering, NAT, and packet mangling. It organizes rules into tables (filter, nat, mangle, raw, security), each containing chains (hook points) of rules. Each rule contains a match (conditions) and a target (action).
Execution Flow
- At the
NF_INET_LOCAL_IN hook: iptable_filter_hook() → ipt_do_table(skb, state, priv->table).
ipt_do_table() traverses the table's rule array: for each rule, calls all match functions (m->match(skb, par)).
- If all matches return true for the packet, the target function is invoked:
t->target(skb, par).
- Targets return a verdict:
IPT_CONTINUE (try next rule), NF_ACCEPT, NF_DROP, NF_QUEUE, or a chain jump (e.g., LOG target).
- Chain jumps (
-j CHAIN_NAME): push current position to a stack; begin traversing the named chain; RETURN target pops the stack.
- If no rule matches and the chain's policy is reached: return the policy verdict (
ACCEPT or DROP).
- The final verdict is returned to
nf_hook_slow().
Edge Cases & Failure Modes
- Stateful vs. stateless race with conntrack: The
nat table modifies packets based on conntrack state; a packet processed before the conntrack entry is established may bypass NAT if the raw table's NOTRACK target was used.
- Rule table atomicity:
iptables-restore replaces the entire ruleset atomically via setsockopt(IPT_SO_SET_REPLACE); during the brief transition, packets may be processed by an inconsistent partial ruleset.
- Jump stack overflow: The rule traversal maintains a stack for chain jumps; a deeply recursive ruleset (chains jumping to chains) can overflow the stack (bounded by
XT_JUMPSTACK_MULTIPLIER * xt_jumpstack_alloc()).
- Connection tracking table exhaustion: The
conntrack module maintains a hash table of connections; if nf_conntrack_max is exceeded, new connections are dropped; on high-connection-rate servers, this requires tuning nf_conntrack_max and GC parameters.
DEF
iptables is the classic packet filter that uses Netfilter hooks to implement stateless and stateful packet filtering,…
FLOW
At the `NF_INET_LOCAL_IN` hook: `iptable_filter_hook()`…`ipt_do_table()` traverses the table's rule array: for …If all matches return true for the packet, the target f…Targets return a verdict: `IPT_CONTINUE` (try next rule…Chain jumps (`-j CHAIN_NAME`): push current position to…If no rule matches and the chain's policy is reached: r…The final verdict is returned to `nf_hook_slow()`.
EDGE
Stateful vs. stateless race with conntrackThe `nat` table modifies packets based on `conntrack` state; a packet processed
Rule table atomicity`iptables-restore` replaces the entire ruleset atomically via `setsockopt(IPT_SO
Jump stack overflowThe rule traversal maintains a stack for chain jumps; a deeply recursive ruleset
Connection tracking table exhaustionThe `conntrack` module maintains a hash table of connections; if `nf_conntrack_m
7.3
nftables (`net/netfilter/nf_tables_core.c`)
nftables is the modern replacement for iptables, providing a unified framework for packet classification using an in-…
▾
Definition
nftables is the modern replacement for iptables, providing a unified framework for packet classification using an in-kernel virtual machine (nft_expr VM) that evaluates rules as bytecode expressions. It supports all protocols (IPv4, IPv6, ARP, Ethernet) through a single framework with set/map data structures for high-performance lookups.
Execution Flow
nft_do_chain() is called from a Netfilter hook; it receives the nft_chain and skb.
- For each
nft_rule in the chain: nft_rule_for_each_expr() iterates the expression list.
- Each expression (
nft_expr) is evaluated by its ops->eval(expr, regs, pktinfo) function: nft_cmp_eval (compare field), nft_lookup_eval (set/map lookup), nft_meta_eval (metadata extraction), etc.
- Expressions write intermediate results to virtual registers (
nft_regs).
- The final expression in a rule is typically a verdict (
nft_verdict): NFT_CONTINUE, NFT_ACCEPT, NFT_DROP, NFT_JUMP chain, NFT_GOTO chain, NFT_RETURN.
- Set lookups:
nft_lookup_eval() calls nft_set_ops->lookup(set, key, &data) → performs a hash lookup or rbtree/interval tree lookup in the set; on hit, may apply a map action (store data into a register).
- The final verdict from
nft_do_chain() is returned to nf_hook_slow().
Edge Cases & Failure Modes
- Atomic ruleset commit: nftables transactions use
nft_trans objects; nft_commit() atomically installs the new generation; in-flight packets using the old generation continue until synchronize_rcu() completes.
- Set update race with packet processing: Calling
nft_set_insert_elem() while nft_lookup_eval() is running on another CPU is safe for hash sets (RCU-protected), but interval-tree sets require a write lock that blocks packet processing.
- Expression VM register contamination: If an expression writes to a register and a subsequent expression reads a different register, uninitialized register data may cause incorrect decisions; nftables validates register usage at rule load time.
- JIT compilation overhead: nftables rules can be JIT-compiled to native code for performance; a JIT bug in an expression can cause arbitrary kernel code execution; the JIT is disabled by default and can be enabled with
nft jit on.
DEF
nftables is the modern replacement for iptables, providing a unified framework for packet classification using an in-…
FLOW
`nft_do_chain()` is called from a Netfilter hook; it re…For each `nft_rule` in the chain: `nft_rule_for_each_ex…Each expression (`nft_expr`) is evaluated by its `ops->…Expressions write intermediate results to virtual regis…The final expression in a rule is typically a verdict (…Set lookups: `nft_lookup_eval()` calls `nft_set_ops->lo…The final verdict from `nft_do_chain()` is returned to …
EDGE
Atomic ruleset commitnftables transactions use `nft_trans` objects; `nft_commit()` atomically install
Set update race with packet processingCalling `nft_set_insert_elem()` while `nft_lookup_eval()` is running on another
Expression VM register contaminationIf an expression writes to a register and a subsequent expression reads a differ
JIT compilation overheadnftables rules can be JIT-compiled to native code for performance; a JIT bug in
7.4
Socket Buffer (`sk_buff`) (`net/core/skbuff.c`)
The `sk_buff` (socket buffer, skb) is the fundamental data structure for network packets in the Linux kernel.
▾
Definition
The sk_buff (socket buffer, skb) is the fundamental data structure for network packets in the Linux kernel. It is a metadata header describing a packet: pointers to the actual packet data (in a separate data buffer), linked list pointers for queues, protocol headers, timestamps, and device references. The design allows O(1) prepend/append of protocol headers without data copying.
Execution Flow
- Packet reception: the network driver allocates an skb via
netdev_alloc_skb() or napi_alloc_skb(), places DMA'd packet data into the skb's data buffer.
skb_put(skb, len) advances the tail pointer, defining the data length.
- The skb is passed up the protocol stack: each layer calls
skb_pull(skb, header_len) to advance the data pointer past the layer's header, exposing the payload to the next layer.
- Protocol layers add metadata:
skb->protocol (ETH_P_IP), skb->ip_summed (checksum status), skb->mark (for firewall rules), etc.
- For transmission:
skb_push(skb, header_len) prepends headers (TCP → IP → Ethernet) without data copying (the head buffer has headroom reserved for this purpose).
dev_queue_xmit(skb) → queues to the device's Qdisc → driver transmits.
kfree_skb(skb) / consume_skb(skb): decrements reference count; on zero, calls skb_release_all() to free the data buffer and skb header.
Edge Cases & Failure Modes
- skb linear/non-linear confusion: An skb may have its data split between a linear buffer and
skb_shinfo->frags[] (paged fragments); a function that calls skb->data assuming all data is linear when the skb is non-linear causes incorrect reads past the linear portion.
- skb reference count leak: Each call to
skb_get() increments the reference count; a code path that takes a reference and then exits on an error without calling kfree_skb() leaks the skb and its data buffer permanently.
- GRO (Generic Receive Offload) aggregation corruption: GRO merges multiple incoming TCP/UDP segments into one large skb for efficiency; if the GRO logic misidentifies a packet as belonging to an existing flow, data from different flows is merged, causing TCP corruption.
- skb headroom exhaustion: If a driver allocates an skb with insufficient headroom and the protocol stack calls
skb_push() beyond the headroom, pskb_expand_head() must reallocate; under memory pressure this fails, dropping the packet.
DEF
The `sk_buff` (socket buffer, skb) is the fundamental data structure for network packets in the Linux kernel.
FLOW
Packet reception: the network driver allocates an skb v…`skb_put(skb, len)` advances the `tail` pointer, defini…The skb is passed up the protocol stack: each layer cal…Protocol layers add metadata: `skb->protocol` (ETH_P_IP…For transmission: `skb_push(skb, header_len)` prepends …`dev_queue_xmit(skb)` → queues to the device's Qdisc → …`kfree_skb(skb)` / `consume_skb(skb)`: decrements refer…
EDGE
skb linear/non-linear confusionAn skb may have its data split between a linear buffer and `skb_shinfo->frags[]`
skb reference count leakEach call to `skb_get()` increments the reference count; a code path that takes
GRO (Generic Receive Offload) aggregation corruptionGRO merges multiple incoming TCP/UDP segments into one large skb for efficiency;
skb headroom exhaustionIf a driver allocates an skb with insufficient headroom and the protocol stack c
7.5
Traffic Control Qdisc (`net/sched/`)
The Traffic Control (TC) framework implements Quality of Service (QoS) for outgoing packets via a hierarchy of Queuei…
▾
Definition
The Traffic Control (TC) framework implements Quality of Service (QoS) for outgoing packets via a hierarchy of Queueing Disciplines (Qdiscs). A Qdisc is attached to each network device's transmit queue; it classifies, schedules, shapes, and polices outgoing traffic. Examples: pfifo_fast (default FIFO), htb (hierarchical token bucket), fq_codel (fair queuing + CoDel AQM), tbf (token bucket filter).
Execution Flow
dev_queue_xmit(skb) calls __dev_xmit_skb().
q->enqueue(skb, q, &to_free) is called: the Qdisc's enqueue function classifies the packet (via tc_classify() for classful Qdiscs like HTB) and places it in the appropriate queue or band.
qdisc_run(q) calls __qdisc_run(q):
q->dequeue(q) removes the next packet according to the scheduling policy (FIFO, WFQ, token bucket drain rate, etc.).
sch_direct_xmit(skb, q, dev, txq, root_lock, true) calls dev_hard_start_xmit() → driver transmit.
- Loop until the transmit queue is full or no more packets.
- For shaping (
tbf, htb): a software hrtimer is used to defer dequeue until the token bucket refills.
- For AQM (
codel, fq_codel): packets are dropped or ECN-marked if sojourn time (time in queue) exceeds the target delay, signaling congestion to TCP.
Edge Cases & Failure Modes
- Qdisc lock contention: The Qdisc lock (
qdisc_lock(q)) serializes all enqueue/dequeue operations; under high packet rates, this becomes a bottleneck; noqueue or mq (multi-queue) Qdiscs avoid this.
- HTB quantum misconfiguration: If
quantum (burst size) is set too small relative to MTU, HTB cannot transfer a full packet in one dequeue cycle, causing starvation of low-priority classes even when their rate limit allows transmission.
fq_codel under UDP flood: CoDel's AQM relies on delay feedback from TCP congestion control; UDP flows don't respond to ECN or drops; fq_codel's flow hashing may assign UDP flood traffic to a separate flow bucket but cannot reduce the flood rate.
- Shaping timer resolution: Software shaping uses
hrtimers with nanosecond resolution, but the actual wake-up jitter depends on the system's timer resolution and load; at high rates (>1 Gbps), software shaping jitter significantly affects shaped throughput.
DEF
The Traffic Control (TC) framework implements Quality of Service (QoS) for outgoing packets via a hierarchy of Queuei…
FLOW
`dev_queue_xmit(skb)` calls `__dev_xmit_skb()`.`q->enqueue(skb, q, &to_free)` is called: the Qdisc's e…`qdisc_run(q)` calls `__qdisc_run(q)`:For shaping (`tbf`, `htb`): a software `hrtimer` is use…For AQM (`codel`, `fq_codel`): packets are dropped or E…
EDGE
Qdisc lock contentionThe Qdisc lock (`qdisc_lock(q)`) serializes all enqueue/dequeue operations; unde
HTB quantum misconfigurationIf `quantum` (burst size) is set too small relative to MTU, HTB cannot transfer
`fq_codel` under UDP floodCoDel's AQM relies on delay feedback from TCP congestion control; UDP flows don'
Shaping timer resolutionSoftware shaping uses `hrtimers` with nanosecond resolution, but the actual wake
7.6
XDP (eXpress Data Path) (`net/core/xdp.c`, `include/linux/bpf.h`)
XDP is a high-performance, programmable packet processing framework that runs eBPF programs at the earliest possible …
▾
Definition
XDP is a high-performance, programmable packet processing framework that runs eBPF programs at the earliest possible point in the receive path — in the network driver's NAPI receive function, before the skb is allocated. This eliminates the overhead of skb allocation, Netfilter hooks, and protocol stack traversal for packets that should be dropped or redirected.
Execution Flow
- At driver load time:
xdp_rxq_info_reg() registers a receive queue for XDP.
- An eBPF program is loaded with
bpf(BPF_PROG_LOAD, ...) and type BPF_PROG_TYPE_XDP; the verifier checks it.
- The program is attached to the interface:
bpf(BPF_LINK_CREATE, ..., attach_type=BPF_XDP) or via ip link set dev eth0 xdp obj prog.o.
- Packet reception: the driver's NAPI poll function calls
bpf_prog_run_xdp(prog, &xdp) before allocating an skb.
- The XDP program executes, returning one of:
XDP_DROP (discard immediately), XDP_PASS (continue to normal stack), XDP_TX (retransmit on same interface), XDP_REDIRECT (send to another interface/CPU via bpf_redirect()), XDP_ABORTED (drop + trace).
XDP_PASS: the driver proceeds to allocate an skb and pass the packet to netif_receive_skb().
XDP_REDIRECT with AF_XDP: packets are delivered to a userspace memory-mapped ring buffer (UMEM) via xsk_rcv().
Edge Cases & Failure Modes
- Driver XDP support required: XDP requires a driver-specific implementation (
ndo_bpf with XDP_SETUP_PROG); drivers without XDP support fall back to "generic XDP" (skb-based), which is slower and loses most XDP performance benefits.
XDP_TX on a congested interface: Retransmitting a packet on an interface whose transmit queue is full causes the packet to be dropped silently; the XDP program has no feedback mechanism to detect this.
- Multi-buffer XDP packets: Jumbo frames or TSO-segmented packets that span multiple receive descriptors require "multi-buffer XDP" support (added in kernel 5.18); earlier kernels silently drop such packets when XDP is active.
- AF_XDP UMEM alignment requirements: The UMEM frame size must be a power of two aligned to the page size; misaligned frames cause the
xsk_bind() syscall to return EINVAL, and the kernel does not report which alignment constraint was violated.
DEF
XDP is a high-performance, programmable packet processing framework that runs eBPF programs at the earliest possible …
FLOW
At driver load time: `xdp_rxq_info_reg()` registers a r…An eBPF program is loaded with `bpf(BPF_PROG_LOAD, ...)…The program is attached to the interface: `bpf(BPF_LINK…Packet reception: the driver's NAPI poll function calls…The XDP program executes, returning one of: `XDP_DROP` …`XDP_PASS`: the driver proceeds to allocate an skb and …`XDP_REDIRECT` with `AF_XDP`: packets are delivered to …
EDGE
Driver XDP support requiredXDP requires a driver-specific implementation (`ndo_bpf` with `XDP_SETUP_PROG`);
`XDP_TX` on a congested interfaceRetransmitting a packet on an interface whose transmit queue is full causes the
Multi-buffer XDP packetsJumbo frames or TSO-segmented packets that span multiple receive descriptors req
AF_XDP UMEM alignment requirementsThe UMEM frame size must be a power of two aligned to the page size; misaligned
7.7
eBPF Network Hooks (`kernel/bpf/`, `net/core/filter.c`)
eBPF (extended Berkeley Packet Filter) allows safe, sandboxed programs to be loaded into the kernel at various attach…
▾
Definition
eBPF (extended Berkeley Packet Filter) allows safe, sandboxed programs to be loaded into the kernel at various attachment points beyond XDP: socket filters (SO_ATTACH_BPF), traffic classifiers (TC BPF), cgroup socket hooks (BPF_CGROUP_INET_*), sk_msg / sk_skb for stream parsers, bpf_sockmap for socket redirection, and bpf_flow_dissector. Programs are verified by the eBPF verifier before being allowed to run.
Execution Flow
- A BPF program is written in restricted C, compiled with
clang -target bpf, producing eBPF bytecode.
bpf(BPF_PROG_LOAD, attr) submits bytecode to the kernel; the eBPF verifier:
- Checks the control-flow graph for loops (bounded only via the verifier's loop unrolling or bounded loops in 5.3+).
- Verifies type safety: each register's type is tracked; invalid dereferences are rejected.
- Checks that helper calls use valid argument types.
- The program is JIT-compiled by
do_jit() to native machine code.
- Attached to a hook (e.g.,
setsockopt(SO_ATTACH_BPF) for a socket filter).
- On packet arrival at the hook point:
BPF_PROG_RUN(prog, ctx) executes the JIT'd code.
- The program reads packet data, calls BPF helpers (
bpf_skb_load_bytes(), bpf_map_lookup_elem()), and returns a verdict (drop/pass/redirect).
Edge Cases & Failure Modes
- Verifier false negative (complexity limit): Programs exceeding the instruction complexity limit (
BPF_COMPLEXITY_LIMIT_INSNS = 1,000,000 instructions verified) are rejected with EACCES even if semantically safe; refactoring into smaller programs is required.
- JIT spraying via BPF constants: Loading a BPF program with crafted immediate values can place attacker-controlled data in JIT code pages;
BPF_JIT_KALLSYMS and constant blinding (bpf_jit_blind_constant()) mitigate this.
- BPF map race conditions: Two BPF programs updating the same
BPF_MAP_TYPE_HASH map concurrently are protected by per-bucket spinlocks, but a program that reads a value and then writes based on it (check-then-act) is not atomic; use BPF_MAP_TYPE_PERCPU_HASH or atomic helpers.
- Helper function ABI stability: BPF helper functions are part of the stable kernel ABI; a helper removed or changed in a kernel update silently breaks BPF programs that relied on it.
DEF
eBPF (extended Berkeley Packet Filter) allows safe, sandboxed programs to be loaded into the kernel at various attach…
FLOW
A BPF program is written in restricted C, compiled with…`bpf(BPF_PROG_LOAD, attr)` submits bytecode to the kern…3+).The program is JIT-compiled by `do_jit()` to native mac…Attached to a hook (e.g., `setsockopt(SO_ATTACH_BPF)` f…On packet arrival at the hook point: `BPF_PROG_RUN(prog…The program reads packet data, calls BPF helpers (`bpf_…
EDGE
Verifier false negative (complexity limit)Programs exceeding the instruction complexity limit (`BPF_COMPLEXITY_LIMIT_INSNS
JIT spraying via BPF constantsLoading a BPF program with crafted immediate values can place attacker-controlle
BPF map race conditionsTwo BPF programs updating the same `BPF_MAP_TYPE_HASH` map concurrently are prot
Helper function ABI stabilityBPF helper functions are part of the stable kernel ABI; a helper removed or chan
7.8
NAPI (New API) (`net/core/dev.c`)
NAPI is the interrupt-mitigation mechanism for network drivers.
▾
Definition
NAPI is the interrupt-mitigation mechanism for network drivers. Instead of delivering one interrupt per received packet (which at 10 Gbps would be millions per second), NAPI allows a driver to disable interrupts after the first packet arrival and poll for more packets in a single scheduling quantum, then re-enable interrupts. This dramatically reduces interrupt overhead under high load.
Execution Flow
- A packet arrives; the NIC asserts an IRQ; the driver's interrupt handler fires.
- The handler: disables further NIC receive interrupts; calls
napi_schedule(&napi) to add the NAPI structure to the per-CPU softnet_data->poll_list.
- The
NET_RX_SOFTIRQ softirq is raised; net_rx_action() runs in softirq context.
net_rx_action() calls each NAPI's poll() function (driver-specific, e.g., igb_poll()) with a budget (default 64 packets).
- The driver's
poll() function processes up to budget received descriptors; for each: allocates an skb, fills it, calls napi_gro_receive() → netif_receive_skb().
- If fewer than
budget packets were processed: napi_complete_done() re-enables NIC interrupts and removes from poll_list.
- If exactly
budget packets were processed (more may be waiting): the NAPI remains scheduled; napi_schedule_prep() reschedules for the next softirq run.
Edge Cases & Failure Modes
- NAPI budget starvation: Under an extreme packet flood, a single high-traffic NAPI may consume the entire softirq time budget (10 ms) and starve other network devices;
netdev_budget and netdev_budget_usecs can be tuned.
napi_schedule() from non-interrupt context: Calling napi_schedule() from a context where the NIC interrupt is not disabled (e.g., from a timer callback) causes a double-schedule; napi_schedule_prep() uses an atomic bit to prevent this.
- GRO session expiry: GRO (Generic Receive Offload) holds partially merged skbs in a session; if no more segments arrive within
GRO_MAX_BACKOFF, the session flushes via napi_gro_flush(); on a lightly loaded interface, GRO adds per-packet latency equal to the flush timeout.
- NAPI weight fairness: All NICs on the same CPU share the softirq budget; a NIC with 64 packets in one NAPI poll consumes the entire budget, starving other NICs;
ethtool -C can tune interrupt coalescing to redistribute.
DEF
NAPI is the interrupt-mitigation mechanism for network drivers.
FLOW
A packet arrives; the NIC asserts an IRQ; the driver's …The handler: disables further NIC receive interrupts; c…The `NET_RX_SOFTIRQ` softirq is raised; `net_rx_action(…`net_rx_action()` calls each NAPI's `poll()` function (…The driver's `poll()` function processes up to `budget`…If fewer than `budget` packets were processed: `napi_co…If exactly `budget` packets were processed (more may be…
EDGE
NAPI budget starvationUnder an extreme packet flood, a single high-traffic NAPI may consume the entire
`napi_schedule()` from non-interrupt contextCalling `napi_schedule()` from a context where the NIC interrupt is not disabled
GRO session expiryGRO (Generic Receive Offload) holds partially merged skbs in a session; if no mo
NAPI weight fairnessAll NICs on the same CPU share the softirq budget; a NIC with 64 packets in one
7.9
Receive Packet Steering (RPS) (`net/core/dev.c`)
RPS is a software implementation of Receive-Side Scaling (RSS) for NICs that have only a single receive queue.
▾
Definition
RPS is a software implementation of Receive-Side Scaling (RSS) for NICs that have only a single receive queue. It distributes received packets across multiple CPUs by computing a hash over the packet's flow tuple (src/dst IP, src/dst port) and steering the packet to the target CPU's input queue via an IPI, allowing parallel protocol processing.
Execution Flow
- A packet is received; the driver calls
netif_rx() or netif_receive_skb().
- If RPS is enabled on the interface (
/sys/class/net/ethX/queues/rx-N/rps_cpus bitmap non-zero):
get_rps_cpu(dev, skb, &rflow) computes a flow hash from the skb's L3/L4 headers using skb_get_hash().
- The hash is mapped to a CPU index via the
rps_map->cpus[] array.
- If the target CPU differs from the current CPU:
enqueue_to_backlog() adds the skb to the target CPU's input_pkt_queue (a lockless ring buffer).
- An IPI (
raise_softirq_irqoff(NET_RX_SOFTIRQ)) is sent to the target CPU.
- The target CPU's
net_rx_action() softirq dequeues from input_pkt_queue and calls __netif_receive_skb().
- Protocol stack processing continues on the target CPU.
Edge Cases & Failure Modes
- Flow hash collision: Two different flows that hash to the same CPU index are always processed on the same CPU, defeating RPS's distribution for those flows; increasing
rps_map size reduces collisions.
- IPI overhead at low load: At low packet rates, the IPI to steer each packet to a remote CPU costs more than the protocol processing it saves; RPS should only be enabled on systems where the NIC interrupt CPU is a bottleneck.
input_pkt_queue overflow: Each CPU's backlog queue has a maximum depth (netdev_max_backlog, default 1000); packets that arrive faster than the softirq can drain them are dropped with NET_RX_DROP.
- RPS and hardware RSS interaction: If the NIC also has hardware RSS (distributing packets across multiple receive queues), enabling software RPS on top creates double-steering overhead; they should be used exclusively.
DEF
RPS is a software implementation of Receive-Side Scaling (RSS) for NICs that have only a single receive queue.
FLOW
A packet is received; the driver calls `netif_rx()` or …If RPS is enabled on the interface (`/sys/class/net/eth…If the target CPU differs from the current CPU: `enqueu…An IPI (`raise_softirq_irqoff(NET_RX_SOFTIRQ)`) is sent…The target CPU's `net_rx_action()` softirq dequeues fro…Protocol stack processing continues on the target CPU.
EDGE
Flow hash collisionTwo different flows that hash to the same CPU index are always processed on the
IPI overhead at low loadAt low packet rates, the IPI to steer each packet to a remote CPU costs more tha
`input_pkt_queue` overflowEach CPU's backlog queue has a maximum depth (`netdev_max_backlog`, default 1000
RPS and hardware RSS interactionIf the NIC also has hardware RSS (distributing packets across multiple receive q
7.10
Netlink Socket (`net/netlink/`)
Netlink is a socket-based IPC mechanism for communication between the kernel and userspace (and between kernel subsys…
▾
Definition
Netlink is a socket-based IPC mechanism for communication between the kernel and userspace (and between kernel subsystems). It uses a datagram-oriented socket family (AF_NETLINK) with protocol numbers for different subsystems: NETLINK_ROUTE (routing, ARP, links), NETLINK_NETFILTER (iptables/nftables configuration), NETLINK_GENERIC (extensible subsystem for new protocols). Used by ip, ss, iptables, iw, and many daemons.
Execution Flow
- Userspace opens:
socket(AF_NETLINK, SOCK_RAW, NETLINK_ROUTE).
bind(fd, &addr, sizeof(addr)) registers the socket with its PID (or 0 for auto-assign) in the netlink subsystem.
- Userspace sends a request:
sendmsg() with a nlmsghdr header (type, flags, sequence, PID) followed by a protocol-specific payload (e.g., struct ifinfomsg).
- The kernel's
netlink_rcv_skb() dequeues the message; calls the protocol's input() callback (e.g., rtnetlink_rcv_msg() for NETLINK_ROUTE).
- The handler processes the request: for
RTM_GETLINK, it iterates dev_base_head and builds nlmsghdr replies using nlmsg_put() / nla_put_*() helpers.
nlmsg_unicast() or nlmsg_multicast() queues the reply to the requesting socket or all subscribers to a multicast group.
- Userspace reads the reply via
recvmsg().
Edge Cases & Failure Modes
- Dump truncation: For large datasets (many routes, many interfaces), a Netlink dump uses
NLM_F_MULTI + NLMSG_DONE to paginate results; if the socket receive buffer (SO_RCVBUF) is too small, ENOBUFS is returned and the dump is incomplete; the client must retry with a larger buffer.
- Netlink multicast group overflow: A slow userspace subscriber to a multicast group (e.g.,
RTMGRP_LINK) with a full receive buffer causes the kernel to drop events silently; the subscriber may miss link state changes.
- Privileged operations without
CAP_NET_ADMIN: Write operations (adding routes, configuring interfaces) require CAP_NET_ADMIN; a non-privileged process attempting these gets EPERM; read operations (listing routes) are generally unprivileged.
- Netlink attribute parsing errors: Malformed
nlattr payloads (incorrect length, missing required attributes) are caught by nla_validate() before dispatch; older kernels that don't validate nlattr alignment can read unaligned memory.
DEF
Netlink is a socket-based IPC mechanism for communication between the kernel and userspace (and between kernel subsys…
FLOW
Userspace opens: `socket(AF_NETLINK, SOCK_RAW, NETLINK_…`bind(fd, &addr, sizeof(addr))` registers the socket wi…Userspace sends a request: `sendmsg()` with a `nlmsghdr…The kernel's `netlink_rcv_skb()` dequeues the message; …The handler processes the request: for `RTM_GETLINK`, i…`nlmsg_unicast()` or `nlmsg_multicast()` queues the rep…Userspace reads the reply via `recvmsg()`.
EDGE
Dump truncationFor large datasets (many routes, many interfaces), a Netlink dump uses `NLM_F_MU
Netlink multicast group overflowA slow userspace subscriber to a multicast group (e.g., `RTMGRP_LINK`) with a fu
Privileged operations without `CAP_NET_ADMIN`Write operations (adding routes, configuring interfaces) require `CAP_NET_ADMIN`
Netlink attribute parsing errorsMalformed `nlattr` payloads (incorrect length, missing required attributes) are
🛡️
S8 — Linux Security Modules (LSMs)
10 components
8.1
LSM Framework (`security/security.c`)
The Linux Security Module (LSM) framework is the kernel's pluggable MAC (Mandatory Access Control) infrastructure.
▾
Definition
The Linux Security Module (LSM) framework is the kernel's pluggable MAC (Mandatory Access Control) infrastructure. It defines over 200 security hook points throughout the kernel (in VFS, IPC, networking, process management, etc.) that call registered hook implementations. Multiple LSMs can be stacked (since Linux 4.2 for minor LSMs, full stacking since 5.1); each hook calls all registered implementations in sequence.
Execution Flow
- The kernel defines hooks as function calls in security-sensitive paths: e.g.,
security_inode_permission(inode, mask) in inode_permission().
security_inode_permission() in security/security.c calls call_int_hook(inode_permission, 0, inode, mask).
call_int_hook iterates the security_hook_heads.inode_permission list; for each registered hook function, calls it with the same arguments.
- Each LSM hook returns 0 (allow) or a negative error code (deny).
- The first non-zero return causes the loop to stop and the error is returned to the caller (deny-wins policy).
- The caller (e.g.,
vfs_open()) returns the error to userspace (typically EACCES or EPERM).
- LSM registration: each LSM calls
security_add_hooks(hooks, count, name) during __init; hooks are added to security_hook_heads lists.
Edge Cases & Failure Modes
- Hook ordering non-determinism: Multiple LSMs registered for the same hook execute in registration order, which is determined by kernel config; a policy that relies on SELinux running before AppArmor may break if the order changes.
- Performance overhead of
call_int_hook: Each hook traversal iterates all registered LSMs; on systems with many stacked LSMs and performance-critical paths (e.g., security_socket_sendmsg()), this adds measurable latency.
- NULL hook dereference in minor LSMs: LSMs registered as "minor" (like Yama, Lockdown) may implement only a subset of hooks;
security_add_hooks() only registers non-NULL hooks, so stale hook function pointers are never called.
- LSM data stored in kernel objects: Each LSM may attach private security blobs to kernel objects (inodes, tasks, sockets) via the
lsm_blob_sizes mechanism; a bug in blob allocation sizing causes memory corruption in security-sensitive structures.
DEF
The Linux Security Module (LSM) framework is the kernel's pluggable MAC (Mandatory Access Control) infrastructure.
FLOW
The kernel defines hooks as function calls in security-…`security_inode_permission()` in `security/security.c` …`call_int_hook` iterates the `security_hook_heads.inode…Each LSM hook returns 0 (allow) or a negative error cod…The first non-zero return causes the loop to stop and t…The caller (e.g., `vfs_open()`) returns the error to us…LSM registration: each LSM calls `security_add_hooks(ho…
EDGE
Hook ordering non-determinismMultiple LSMs registered for the same hook execute in registration order, which
Performance overhead of `call_int_hook`Each hook traversal iterates all registered LSMs; on systems with many stacked L
NULL hook dereference in minor LSMsLSMs registered as "minor" (like Yama, Lockdown) may implement only a subset of
LSM data stored in kernel objectsEach LSM may attach private security blobs to kernel objects (inodes, tasks, soc
8.2
SELinux (`security/selinux/`)
SELinux (Security-Enhanced Linux) implements mandatory access control via type enforcement: every process and kernel …
▾
Definition
SELinux (Security-Enhanced Linux) implements mandatory access control via type enforcement: every process and kernel object has a security context (user:role:type:level), and access is governed by an in-kernel policy database (the Type Enforcement, Role-Based Access Control, and Multi-Level Security components). It is the dominant LSM on RHEL, Fedora, and Android.
Execution Flow
- At boot (policy load):
security_load_policy() → selinux_set_policy() loads the binary policy from /etc/selinux/*/policy.* via avc_policy_seqno().
- The Access Vector Cache (AVC) is initialized: an in-memory hash table mapping (source SID, target SID, class) →
allowed / denied / auditallow decision bitsets.
- Access check at hook time:
selinux_inode_permission() → avc_has_perm(ssid, tsid, SECCLASS_FILE, perm).
avc_has_perm_noaudit() checks the AVC cache: on hit, returns the cached decision in nanoseconds.
- On AVC miss:
avc_compute_av() → security_compute_av() queries the policy database (DFA transitions).
- The result is cached in the AVC and returned; if
denied: avc_audit() logs a denial message to the audit subsystem.
- In "permissive" mode: denials are logged but access is granted; in "enforcing" mode: denials return
-EACCES.
Edge Cases & Failure Modes
- AVC thrashing: Rapidly creating processes with new type combinations (e.g., container runtimes) floods the AVC with new SID lookups; the AVC's LRU eviction can't keep up, causing repeated policy database queries and performance regression.
- Policy reload race: A
load_policy call during active use atomically swaps the policy via RCU; in-flight permission checks using the old policy SIDs complete with the old policy; a brief window of inconsistency is unavoidable.
- Mislabeled filesystem objects: Files without SELinux extended attributes (
security.selinux) receive the unlabeled_t type; most SELinux policies deny access to unlabeled_t, causing application failures on newly created files; restorecon fixes labels.
- SID overflow: Security IDs are 32-bit integers assigned from an incrementing counter; long-lived systems with many unique contexts can exhaust the SID space;
selinux_sid_to_context() returns ENOMEM when this happens.
DEF
SELinux (Security-Enhanced Linux) implements mandatory access control via type enforcement: every process and kernel …
FLOW
At boot (policy load): `security_load_policy()` → `seli…The Access Vector Cache (AVC) is initialized: an in-mem…Access check at hook time: `selinux_inode_permission()`…`avc_has_perm_noaudit()` checks the AVC cache: on hit, …On AVC miss: `avc_compute_av()` → `security_compute_av(…The result is cached in the AVC and returned; if `denie…In "permissive" mode: denials are logged but access is …
EDGE
AVC thrashingRapidly creating processes with new type combinations (e.g., container runtimes)
Policy reload raceA `load_policy` call during active use atomically swaps the policy via RCU; in-f
Mislabeled filesystem objectsFiles without SELinux extended attributes (`security.selinux`) receive the `unla
SID overflowSecurity IDs are 32-bit integers assigned from an incrementing counter; long-liv
8.3
AppArmor (`security/apparmor/`)
AppArmor implements MAC via path-based profiles: each application is confined by a named profile that specifies allow…
▾
Definition
AppArmor implements MAC via path-based profiles: each application is confined by a named profile that specifies allowed file paths (using globs), network access, capabilities, and mount operations. Unlike SELinux's type-enforcement model, AppArmor labels processes by their executable path and uses a DFA (Deterministic Finite Automaton) for path matching.
Execution Flow
- Profile loading:
aa_replace_profiles() reads profiles from userspace (via securityfs at /sys/kernel/security/apparmor/.replace); the profile text is compiled to a DFA by aa_policy_init() and stored in aa_profile.
- Process confinement: at
exec(), apparmor_bprm_set_creds() looks up the new executable's path in the profile database; if a matching profile exists, it is attached to the task's credential.
- File permission check:
apparmor_inode_permission() calls aa_path_perm() → aa_str_perms(profile->file.dfa, path, mode) runs the DFA against the file's path string.
- The DFA transitions through states for each path character; the final state maps to a permission set (read, write, execute, lock, etc.).
- If the requested access is not in the allowed set: return
-EACCES (enforcing) or log + allow (complain mode).
aa_audit_msg() logs denials to the audit ring buffer with the matched profile name and denied path.
Edge Cases & Failure Modes
- Path vs. inode confusion: AppArmor matches paths at permission-check time, not at exec time; a process can be confined to
/usr/bin/app but if app is bind-mounted to another location, the path check sees the new location, potentially matching a different or no profile.
- Glob pattern explosion: Overly broad globs (e.g.,
/**) in a profile cause the DFA to become very large (exponential state explosion); AppArmor compiles profiles offline and may refuse to load excessively large DFAs.
- Namespace profile inheritance: AppArmor namespaces (for containers) inherit parent profiles; a container that breaks out of its AppArmor namespace can access the parent's profiles, which are typically less restrictive.
- Transition label vs. exec path: If a process
exec()s a new binary while the filesystem path changes between the exec() call and the credential setup (TOCTOU via bind mount), the wrong profile may be attached.
DEF
AppArmor implements MAC via path-based profiles: each application is confined by a named profile that specifies allow…
FLOW
Profile loading: `aa_replace_profiles()` reads profiles…Process confinement: at `exec()`, `apparmor_bprm_set_cr…File permission check: `apparmor_inode_permission()` ca…The DFA transitions through states for each path charac…If the requested access is not in the allowed set: retu…`aa_audit_msg()` logs denials to the audit ring buffer …
EDGE
Path vs. inode confusionAppArmor matches paths at permission-check time, not at exec time; a process can
Glob pattern explosionOverly broad globs (e.g., `/**`) in a profile cause the DFA to become very large
Namespace profile inheritanceAppArmor namespaces (for containers) inherit parent profiles; a container that b
Transition label vs. exec pathIf a process `exec()`s a new binary while the filesystem path changes between th
8.4
Smack (`security/smack/`)
Smack (Simplified Mandatory Access Control Kernel) implements MAC using a simple label-based model: every process and…
▾
Definition
Smack (Simplified Mandatory Access Control Kernel) implements MAC using a simple label-based model: every process and filesystem object has a Smack label (a short string). Access is governed by a small, explicitly defined access matrix: a process with label A can access an object with label B only if an A b rwxatl rule exists in the Smack access list. Designed for embedded and IoT use cases.
Execution Flow
- Labels are assigned: filesystem objects get Smack labels via extended attributes (
security.SMACK64); processes inherit their parent's label at fork, or receive a new one via smack_task_setscheduler().
- Access check:
smack_inode_permission() → smk_access(subject_label, object_label, access_mode, audit).
smk_access() calls smk_find_rule() which searches the Smack access list (a linked list of smack_rule structs) for a matching (subject, object) pair.
- If a rule is found: check whether the requested access mode is in
rule->smk_access; return 0 (allow) or -EACCES (deny).
- If no rule is found: apply the star-rule fallback:
* label (floor label) allows all access; ^ label (hat label) has access only from *.
- Write access to Smack configuration via
securityfs (/sys/fs/smackfs/) requires CAP_MAC_ADMIN.
Edge Cases & Failure Modes
- Rule list scalability: Smack's access list is a simple linked list; with many labels,
smk_find_rule() is O(N) per access check; on systems with large policy files, this is a performance bottleneck.
- Label length limit: Smack labels are limited to
SMK_MAXLEN (23 characters); labels exceeding this are silently truncated; two labels differing only after the 23rd character become the same label.
- Unlabeled filesystem objects: Files without a
security.SMACK64 xattr receive the default_label (initially the floor label _); unlabeled NFS mounts receive a fixed label configured at mount time, which may be more permissive than intended.
- Transmute directories: A directory with the
SMACK64TRANSMUTE xattr causes new files created within it to inherit the directory's label regardless of the creator's label; a misconfigured transmute directory allows label escalation.
DEF
Smack (Simplified Mandatory Access Control Kernel) implements MAC using a simple label-based model: every process and…
FLOW
Labels are assigned: filesystem objects get Smack label…Access check: `smack_inode_permission()` → `smk_access(…`smk_access()` calls `smk_find_rule()` which searches t…If a rule is found: check whether the requested access …If no rule is found: apply the star-rule fallback: `*` …Write access to Smack configuration via `securityfs` (`…
EDGE
Rule list scalabilitySmack's access list is a simple linked list; with many labels, `smk_find_rule()`
Label length limitSmack labels are limited to `SMK_MAXLEN` (23 characters); labels exceeding this
Unlabeled filesystem objectsFiles without a `security.SMACK64` xattr receive the `default_label` (initially
Transmute directoriesA directory with the `SMACK64TRANSMUTE` xattr causes new files created within it
8.5
TOMOYO (`security/tomoyo/`)
TOMOYO Linux is a name-based MAC system that learns and enforces process call graphs: every process's behavior is def…
▾
Definition
TOMOYO Linux is a name-based MAC system that learns and enforces process call graphs: every process's behavior is defined in terms of which executables it can spawn (domain transitions) and which file paths it can access, based on its "domain" (defined by the sequence of programs in its execution history). It operates in learning, permissive, and enforcing modes.
Execution Flow
- Domain creation: each process starts in a domain defined by the full executable path chain (e.g.,
/sbin/init /usr/sbin/sshd /usr/bin/bash).
- Learning mode:
tomoyo_check_file_perm() and other hooks log every access; tomoyo_manager reads logs and auto-generates allow rules.
- Policy enforcement:
tomoyo_path_perm() → tomoyo_check_perm() looks up the current process's domain in the policy database.
- The access request (file path, operation) is checked against the domain's allow list (a sorted list of path patterns stored as
tomoyo_path_acl entries).
tomoyo_path_matches_pattern() performs glob-style matching on the path string against each pattern.
- If no match: in enforcing mode, return
-EACCES; in learning mode, add the path to the domain's policy and allow.
- Domain transitions at
exec(): tomoyo_find_next_domain() looks up the new domain in the policy tree; if found, the process's domain pointer is updated.
Edge Cases & Failure Modes
- Policy database size in learning mode: TOMOYO's learning mode generates one entry per unique (domain, path) pair; on a heavily used system with many unique paths, the policy file can grow to hundreds of megabytes.
- Domain transition loop detection: A process that repeatedly
exec()s different programs generates an ever-deepening domain chain; TOMOYO_MAX_DOMAINNAME_LEN (4096 characters) caps the chain length; at the cap, the domain is truncated, potentially misidentifying the process.
- Symlink indirection: TOMOYO uses the resolved path after all symlinks; a policy that allows access to
/usr/bin/python automatically covers any symlink pointing to it, which may be undesirable for confining wrapper scripts.
- Performance under high syscall rate: Each syscall to a file requires a path match against the domain's allow list; on a system making millions of syscalls per second, the linear list scan is a bottleneck;
CONFIG_TOMOYO is not recommended for high-throughput servers.
DEF
TOMOYO Linux is a name-based MAC system that learns and enforces process call graphs: every process's behavior is def…
FLOW
Domain creation: each process starts in a domain define…Learning mode: `tomoyo_check_file_perm()` and other hoo…Policy enforcement: `tomoyo_path_perm()` → `tomoyo_chec…The access request (file path, operation) is checked ag…`tomoyo_path_matches_pattern()` performs glob-style mat…If no match: in enforcing mode, return `-EACCES`; in le…Domain transitions at `exec()`: `tomoyo_find_next_domai…
EDGE
Policy database size in learning modeTOMOYO's learning mode generates one entry per unique (domain, path) pair; on a
Domain transition loop detectionA process that repeatedly `exec()`s different programs generates an ever-deepeni
Symlink indirectionTOMOYO uses the resolved path after all symlinks; a policy that allows access to
Performance under high syscall rateEach syscall to a file requires a path match against the domain's allow list; on
8.6
Yama (`security/yama/`)
Yama is a simple LSM that restricts `ptrace(2)` and `/proc/<pid>/mem` access between processes without a parental rel…
▾
Definition
Yama is a simple LSM that restricts ptrace(2) and /proc//mem access between processes without a parental relationship, preventing ptrace-based privilege escalation and credential theft. It implements a single policy knob (/proc/sys/kernel/yama/ptrace_scope) with four levels (0–3) controlling which processes can ptrace which others.
Execution Flow
- A process calls
ptrace(PTRACE_ATTACH, target_pid, ...).
- The VFS path invokes
security_ptrace_access_check(task, mode) → yama_ptrace_access_check(task, mode).
- Level 0 (classic): allows any process to ptrace any process it could normally
ptrace (DAC rules only).
- Level 1 (restricted):
task_is_descendant(current, task) checks if current is an ancestor of task; if not, checks for a capability exception (CAP_SYS_PTRACE).
- Level 2 (admin-only): only processes with
CAP_SYS_PTRACE may attach.
- Level 3 (no attach):
ptrace(PTRACE_ATTACH) is denied for all processes (even root); no runtime override.
- Exception grants: a process can explicitly allow another to ptrace it via
prctl(PR_SET_PTRACER, pid), which adds an entry to ptracer_llist.
Edge Cases & Failure Modes
- Level 1 bypass via re-parenting: If a privileged daemon uses
prctl(PR_SET_CHILD_SUBREAPER), it becomes the ancestor of all orphaned processes; this allows it to bypass Level 1 restrictions by being an artificial ancestor.
PR_SET_PTRACER list growth: Each prctl(PR_SET_PTRACER, pid) call allocates a ptracer_node; a process that sets many ptrace permissions without revoking them accumulates nodes; the list is walked on every ptrace check.
- Level 3 and debugger tooling: Enabling Level 3 breaks strace, gdb, and system-wide profiling tools (
perf); it cannot be relaxed at runtime; this makes Level 3 unsuitable for development systems.
- Yama and user namespaces: Within a user namespace,
CAP_SYS_PTRACE is scoped to the namespace; a process with this capability in a user namespace can bypass Level 2 restrictions for processes in the same namespace, potentially unintended in container environments.
DEF
Yama is a simple LSM that restricts `ptrace(2)` and `/proc/<pid>/mem` access between processes without a parental rel…
FLOW
A process calls `ptrace(PTRACE_ATTACH, target_pid, ...)…The VFS path invokes `security_ptrace_access_check(task…Level 0 (classic): allows any process to ptrace any pro…Level 1 (restricted): `task_is_descendant(current, task…Level 2 (admin-only): only processes with `CAP_SYS_PTRA…Level 3 (no attach): `ptrace(PTRACE_ATTACH)` is denied …Exception grants: a process can explicitly allow anothe…
EDGE
Level 1 bypass via re-parentingIf a privileged daemon uses `prctl(PR_SET_CHILD_SUBREAPER)`, it becomes the ance
`PR_SET_PTRACER` list growthEach `prctl(PR_SET_PTRACER, pid)` call allocates a `ptracer_node`; a process tha
Level 3 and debugger toolingEnabling Level 3 breaks strace, gdb, and system-wide profiling tools (`perf`); i
Yama and user namespacesWithin a user namespace, `CAP_SYS_PTRACE` is scoped to the namespace; a process
8.7
Landlock (`security/landlock/`)
Landlock (Linux 5.13+) is a sandboxing LSM that allows unprivileged processes to restrict their own access rights at …
▾
Definition
Landlock (Linux 5.13+) is a sandboxing LSM that allows unprivileged processes to restrict their own access rights at runtime. Unlike SELinux or AppArmor (which require root-managed policies), Landlock lets a process create a ruleset specifying allowed filesystem access, network operations (5.19+), and IPC operations (6.10+), and then self-enforce it via a syscall.
Execution Flow
- A process creates a ruleset:
landlock_create_ruleset(attr, attr_size, flags) → returns a ruleset fd.
- Rules are added:
landlock_add_rule(ruleset_fd, type, attr, flags):
- For
LANDLOCK_RULE_PATH_BENEATH: open the directory fd; specify allowed_access bitset (read, write, exec, etc.).
- The rule is stored in the ruleset's Red-Black tree keyed by inode number.
- The process restricts itself:
landlock_restrict_self(ruleset_fd, flags) → adds the ruleset to the current thread's landlock_ruleset credential field.
- At each LSM hook (e.g.,
landlock_inode_permission()): landlock_get_applicable_ruleset(cred) retrieves the active ruleset.
check_access_path() traverses the inode's ancestor dentries; for each layer, checks if the dentry's inode is in the ruleset's RB tree and whether the requested access is in its allowed_access.
- If no rule covers the access: return
-EACCES.
- The ruleset is inherited by child processes (and survives
exec()) but cannot be extended; it can only be further restricted by adding a new ruleset.
Edge Cases & Failure Modes
- Ruleset inheritance + exec: A child process inherits the parent's Landlock restrictions; a setuid binary exec'd by a Landlock-restricted process inherits the restrictions, which may prevent the setuid program from accessing files it needs; this is intentional (defense in depth) but breaks some use cases.
- Hardlink bypass: Landlock rules are inode-based; a file accessible via a hardlink from a path not covered by the ruleset can be accessed if the hardlink target's inode is not explicitly restricted; this is a known limitation.
- Versioned ABI: Landlock's ABI is versioned via the
abi_version field in landlock_create_ruleset; programs must check the version and downgrade their handled_access_fs to the highest supported version; failing to do so results in EINVAL.
- Network restriction in Landlock < 5.19: The network restriction feature (
LANDLOCK_RULE_NET_PORT) was added in Linux 5.19; programs that check for Landlock support but don't check the ABI version may incorrectly assume network restrictions are available.
DEF
Landlock (Linux 5.13+) is a sandboxing LSM that allows unprivileged processes to restrict their own access rights at …
FLOW
A process creates a ruleset: `landlock_create_ruleset(a…Rules are added: `landlock_add_rule(ruleset_fd, type, a…The process restricts itself: `landlock_restrict_self(r…At each LSM hook (e.g., `landlock_inode_permission()`):…`check_access_path()` traverses the inode's ancestor de…If no rule covers the access: return `-EACCES`.The ruleset is inherited by child processes (and surviv…
EDGE
Ruleset inheritance + execA child process inherits the parent's Landlock restrictions; a setuid binary exe
Hardlink bypassLandlock rules are inode-based; a file accessible via a hardlink from a path not
Versioned ABILandlock's ABI is versioned via the `abi_version` field in `landlock_create_rule
Network restriction in Landlock < 5.19The network restriction feature (`LANDLOCK_RULE_NET_PORT`) was added in Linux 5.
8.8
LoadPin (`security/loadpin/`)
LoadPin is a simple LSM that restricts kernel module loading and firmware loading to files from a single trusted file…
▾
Definition
LoadPin is a simple LSM that restricts kernel module loading and firmware loading to files from a single trusted filesystem (typically the initramfs or the root filesystem). Once the "pinned" filesystem is set (on first module load after boot), no module or firmware may be loaded from any other filesystem. It prevents loading of untrusted kernel code at runtime.
Execution Flow
- At boot: LoadPin is enabled with
loadpin.enforce=1 on the kernel command line.
- The first time the kernel loads a module or firmware (
security_kernel_read_file() or security_kernel_load_data()), loadpin_read_file() is called.
- If
pinned_root is NULL (not yet set): the superblock of the file being loaded is saved as pinned_root; this filesystem is now the trusted root.
- On all subsequent module/firmware loads:
loadpin_read_file() checks whether the file's superblock matches pinned_root.
- If it doesn't match: return
-EPERM; the module/firmware load is blocked.
- The pinned root can optionally be set to a specific list of trusted filesystems via
loadpin.verity_root_digest (dm-verity integration).
Edge Cases & Failure Modes
- Initramfs pivot_root interaction: If the system pivots from an initramfs root to the real root filesystem,
pinned_root was set to the initramfs superblock; subsequent module loads from the real root fail with -EPERM; LoadPin-enabled systems must load all required modules before pivot_root().
- Network-loaded firmware: Firmware blobs fetched from the network (e.g., via
request_firmware() with an FIRMWARE_URL override) have no filesystem superblock; LoadPin rejects them, preventing remote firmware updates; loadpin.exclude=firmware can exempt firmware from the check.
- Hot-plugged device firmware: A USB device inserted after boot requires firmware; if the firmware is on the pinned filesystem, it loads normally; if it's on a removable media filesystem, it fails; this is intentional but may surprise administrators.
- LoadPin with module signing: LoadPin and
CONFIG_MODULE_SIG provide complementary protections; LoadPin prevents loading from an untrusted FS, while module signing prevents loading unsigned code from any FS; both should be used together for maximum hardening.
DEF
LoadPin is a simple LSM that restricts kernel module loading and firmware loading to files from a single trusted file…
FLOW
At boot: LoadPin is enabled with `loadpin.enforce=1` on…The first time the kernel loads a module or firmware (`…If `pinned_root` is NULL (not yet set): the superblock …On all subsequent module/firmware loads: `loadpin_read_…If it doesn't match: return `-EPERM`; the module/firmwa…The pinned root can optionally be set to a specific lis…
EDGE
Initramfs pivot_root interactionIf the system pivots from an initramfs root to the real root filesystem, `pinned
Network-loaded firmwareFirmware blobs fetched from the network (e.g., via `request_firmware()` with an
Hot-plugged device firmwareA USB device inserted after boot requires firmware; if the firmware is on the pi
LoadPin with module signingLoadPin and `CONFIG_MODULE_SIG` provide complementary protections; LoadPin preve
8.9
SafeSetID (`security/safesetid/`)
SafeSetID is a targeted LSM that restricts which user IDs (UIDs) and group IDs (GIDs) a privileged process (one with …
▾
Definition
SafeSetID is a targeted LSM that restricts which user IDs (UIDs) and group IDs (GIDs) a privileged process (one with CAP_SETUID / CAP_SETGID) may transition to. It enforces a whitelist of allowed UID/GID transitions, preventing a compromised privileged daemon from dropping privileges to an arbitrary UID and then re-acquiring them via other means.
Execution Flow
- Policy is loaded via
securityfs at /sys/kernel/security/safesetid/whitelist_policy as a text file of source_uid:target_uid pairs.
safesetid_load_policy() parses the file and builds a hash table of setid_rule entries keyed by (source_uid, target_uid).
- When a process calls
setuid(new_uid) or setresuid(): safesetid_task_fix_setuid() is invoked.
- It checks whether a whitelist entry exists for
(current_uid, new_uid) via safesetid_find_rule(current_uid, new_uid).
- If no entry exists: return
-EACCES; the UID transition is denied even if the process has CAP_SETUID.
- The check applies transitively: if process A transitions to UID B, and B tries to transition to UID C, the rule
(B, C) must also exist in the whitelist.
Edge Cases & Failure Modes
- Policy not loaded on boot: If SafeSetID is compiled in but no policy is loaded, all UID transitions are allowed (the default); the security benefit only activates once a policy is explicitly loaded.
- Wildcard transition blocker: The special rule
source_uid:CAP_SETUID_UNSAFE blocks a UID from ever using CAP_SETUID to transition to a UID not in the whitelist; a misconfigured wildcard rule can lock out legitimate UID transitions.
- GID vs. UID policies: SafeSetID can enforce both GID and UID transitions; a policy that covers UIDs but not GIDs allows a process to change to an arbitrary GID, potentially gaining access to group-protected resources.
- Interaction with PAM: PAM modules that call
setuid()/setgid() as part of login (e.g., pam_unix) may fail silently if the UID transition they attempt is not in the SafeSetID whitelist, causing login failures without obvious error messages.
DEF
SafeSetID is a targeted LSM that restricts which user IDs (UIDs) and group IDs (GIDs) a privileged process (one with …
FLOW
Policy is loaded via `securityfs` at `/sys/kernel/secur…`safesetid_load_policy()` parses the file and builds a …When a process calls `setuid(new_uid)` or `setresuid()`…It checks whether a whitelist entry exists for `(curren…If no entry exists: return `-EACCES`; the UID transitio…The check applies transitively: if process A transition…
EDGE
Policy not loaded on bootIf SafeSetID is compiled in but no policy is loaded, all UID transitions are all
Wildcard transition blockerThe special rule `source_uid:CAP_SETUID_UNSAFE` blocks a UID from ever using `CA
GID vs. UID policiesSafeSetID can enforce both GID and UID transitions; a policy that covers UIDs bu
Interaction with PAMPAM modules that call `setuid()`/`setgid()` as part of login (e.g., `pam_unix`)
8.10
Integrity Measurement Architecture (IMA) (`security/integrity/ima/`)
IMA (Integrity Measurement Architecture) is the kernel's runtime integrity subsystem.
▾
Definition
IMA (Integrity Measurement Architecture) is the kernel's runtime integrity subsystem. It measures (hashes) files as they are opened for execution, reads, or mmap'd, and stores the measurements in an in-kernel log and in a TPM PCR (Platform Configuration Register). The companion EVM (Extended Verification Module) verifies file metadata integrity using HMAC or digital signatures stored in extended attributes. Together they detect unauthorized file modifications.
Execution Flow
- At boot:
ima_init() initializes the measurement list and extends TPM PCR 10 with an initial aggregate hash.
- A file is opened:
security_file_open() → ima_file_check() → evaluates the IMA policy (rules specifying which files to measure/appraise).
- If the policy matches:
ima_collect_measurement() reads the file's hash (SHA-256 by default) using crypto_shash_digest().
- The measurement is added to the IMA measurement list (
ima_add_template_entry()).
ima_pcr_extend() extends TPM PCR 10 with the hash of the measurement, cryptographically chaining it.
- For EVM appraisal:
evm_inode_setxattr() reads security.ima xattr (expected hash or signature); ima_appraise_measurement() compares the measured hash to the expected value.
- If appraisal fails: in
enforce mode, security_inode_permission() returns -EACCES; in fix mode, the xattr is updated.
- The IMA log is accessible from userspace at
/sys/kernel/security/integrity/ima/ascii_runtime_measurements.
Edge Cases & Failure Modes
- TPM unavailability: Without a TPM, IMA measurements are still logged in the kernel's measurement list but cannot be hardware-attested; remote attestation workflows that require TPM-backed PCR values fail.
- IMA policy miss after hot-plug: Files opened before IMA policy is loaded are not measured; a policy that is loaded after system start has no measurements for pre-loaded binaries;
ima_policy must be loaded via initramfs for complete coverage.
- EVM HMAC key compromise: EVM's HMAC key is stored in a kernel keyring (
_evm); if the key is extracted (e.g., via a cold-boot attack or memory dump), an attacker can forge valid EVM signatures for any file, bypassing appraisal.
- Extended attribute loss on filesystem copy: If files are copied without preserving extended attributes (
cp --no-preserve=all, some archive tools), security.ima and security.evm xattrs are lost; IMA appraisal fails for all copied files until xattrs are regenerated.
DEF
IMA (Integrity Measurement Architecture) is the kernel's runtime integrity subsystem.
FLOW
At boot: `ima_init()` initializes the measurement list …A file is opened: `security_file_open()` → `ima_file_ch…If the policy matches: `ima_collect_measurement()` read…The measurement is added to the IMA measurement list (`…`ima_pcr_extend()` extends TPM PCR 10 with the hash of …For EVM appraisal: `evm_inode_setxattr()` reads `securi…If appraisal fails: in `enforce` mode, `security_inode_…The IMA log is accessible from userspace at `/sys/kerne…
EDGE
TPM unavailabilityWithout a TPM, IMA measurements are still logged in the kernel's measurement lis
IMA policy miss after hot-plugFiles opened before IMA policy is loaded are not measured; a policy that is load
EVM HMAC key compromiseEVM's HMAC key is stored in a kernel keyring (`_evm`); if the key is extracted (
Extended attribute loss on filesystem copyIf files are copied without preserving extended attributes (`cp --no-preserve=al
No matching components.
Linux Kernel Component Taxonomy · MDL State-Machine Protocol · 80 items × 4 fields · Generated with Unified Reasoner L3 + GoT→CoT→PVL + Oracle-Anchoring · Linux 5.x/6.x basis