Managing Processors

Jeff Chase
Duke University
The story so far: protected CPU mode
Any kind of machine exception transfers control to a registered
(trusted) kernel handler running in a protected CPU mode.

user
mode
syscall trap fault fault

u-start u-return u-start u-return

kernel “top half” kernel
kernel “bottom half” (interrupt handlers) mode
clock interrupt
interrupt return

Kernel handler manipulates

CPU register context to return
to selected user context.
 The kernel
Every entry to the kernel is the result of a trap, fault, or interrupt. The
core switches to kernel mode and transfers control to a handler routine.

syscall trap/return fault/return

OS kernel code and data for system calls (files, process
fork/exit/wait, pipes, binder IPC, low-level thread support, etc.)
and virtual memory management (page faults, etc.)

I/O completions interrupt/return timer ticks

The handler accesses the core register context to read the details of
the exception (trap, fault, or interrupt). It may call other kernel routines.
 Exceptions: trap, fault, interrupt
intentional unintentional
happens every time contributing factors

synchronous trap: system call fault

open, close, read, invalid or protected
caused by an
write, fork, exec, exit, address or opcode, page
instruction wait, kill, etc. fault, overflow, etc.

asynchronous “software interrupt” interrupt

caused by software requests an caused by an external
some other interrupt to be delivered event: I/O op completed,
event at a later time clock tick, power fail, etc.

Every entry to the kernel is the result of a trap, fault, or interrupt. The core sets its
mode to protected kernel mode and transfers control to the corresponding handler.
 Kernel Stacks and Trap/Fault Handling
Processes
System calls
execute user
and faults run
code on a user data
in kernel
stack in the
mode on the
user virtual
stack stack process
memory in the
kernel stack.
process virtual
address space.
Kernel code
running in P’s
Each process has a
syscall process context
second kernel
dispatch (i.e., on its
stack in kernel stack
stack kstack) has
space (VM table
access to P’s
accessible only to
virtual memory.
the kernel).

The syscall handler makes an indirect call through the system call
dispatch table to the handler registered for the specific system call.
 The kernel
A trap or fault handler may suspend (sleep) the current thread, leaving its
state (call frames) on its kernel stack and a saved context in its TCB.

syscall traps faults

sleep queue ready queue

interrupts

The TCB for a blocked thread is left on a sleep queue for some
synchronization object. A later event/action may wakeup the thread.
Thread states and transitions

running
yield

Scheduler governs
sleep these transitions.

wakeup
blocked ready

wait, STOP, read, write, Sleep and wakeup are internal

listen, receive, etc. primitives. Wakeup adds a thread to
the scheduler’s ready pool: a set of
STOP threads in the ready state.
wait
Contention on ready thread queues
• A large-scale system may have significant contention on
the spinlock for the ready thread queue.
– Each core removes a thread from the ready queue with
GetNextThreadToRun() on each context switch.
– Every wakeup adds a thread to the ready queue.
• On average, the frequency of these events is linear with
the number of cores.
– What is the average wait time for the spinlock?
• To reduce contention, an OS may have a separate run
queue for each machine partition.
– Each queue serves a partition of N cores.
Per-CPU ready queues

• lock per runqueue

• preempt on queue insertion Let’s talk about
• recalculate priority on expiration priority…
CPU dispatch and ready queues

In a typical OS, each thread has a priority, which may change over
time. When a core is idle, pick the (a) thread with the highest priority. If
a higher-priority thread becomes ready, then preempt the thread
currently running on the core and switch to the new thread. If the
quantum expires (timer), then preempt, select a new thread, and switch
Priority
Most modern OS schedulers use priority scheduling.
– Each thread in the ready pool has a priority value.
– The scheduler favors higher-priority threads.
– Threads inherit a base priority from the associated
application/process.
– User-settable relative importance within application
– Internal priority adjustments as an implementation
technique within the scheduler.
– How to set the priority of a thread?
How many priority levels? 32 (Windows) to 128 (OS X)
Per-CPU ready queues

On most architectures, a find-first-bit-set instruction is

used to find the highest priority bit set in one of five 32-
bit words (for the 140 priorities).