MIC Scheduler & Intel Thread DirectorStarting in MidnightBSD 4.2-CURRENT, the amd64 GENERIC kernel uses the sched_mic(4) (Michaelangelo) scheduler by default and includes the hfi(4) driver for Intel Hardware Feedback Interface (HFI) and Intel Thread Director (ITD).
These components work together through the cpucap(9) CPU capability layer:
cpucap(9) collects static topology classes (at boot via CPUID) and dynamic per-CPU scores, publishing updates to the scheduler outside the hot path.hfi(4) reads hardware performance and energy-efficiency tables from Intel hybrid processors and, when Thread Director is supported, enables per-thread instruction classification.sched_mic(4) uses precomputed per-CPU cost and rank tables to steer threads to appropriate cores and swap running CPU-bound threads when their instruction mix favors a different core class.On homogeneous processors (AMD non-hybrid, non-hybrid Intel, or virtual machines without exposed HFI), sched_mic(4) behaves identically to sched_ule(4) and hfi(4) does not attach. The only visible difference is that kern.sched.name reports MIC.
On heterogeneous (hybrid) x86 processors, sched_mic(4) recognizes four priority tiers when placing threads:
Core preference is a soft, tunable bias: real CPU load always decides first, and core class orders CPUs whose load is otherwise acceptable. When an idle CPU looks for queued work to steal, a CPU of the same or better class steals immediately once kern.sched.steal_thresh allows, whereas a worse-class CPU (or an idle SMT sibling of a busy core) waits until the thread has been queued for kern.sched.steal_delay ticks (about 4 ms by default). This prevents brief interrupt preemptions from bouncing threads off performance cores while allowing real backlogs to spill onto efficiency cores.
By default, kern.sched.prefer_compute is 1 (preferring Intel P-cores or AMD frequency/compute cores), except on AMD X3D parts where it defaults to 0 so threads prefer the large-cache 3D V-Cache CCD. Setting kern.sched.prefer_compute=0 on a hybrid Intel system instructs the scheduler to prefer efficiency cores using their efficiency scores.
hfi(4))Intel hybrid processors starting with the 12th generation Core (Alder Lake) provide the Hardware Feedback Interface. The hfi(4) driver allocates one table per physical package, and the processor writes per-core performance and energy-efficiency scores (0–255, where 0 indicates the processor recommends scheduling nothing on that core) into the table and raises a local APIC thermal interrupt when thermal or power limits change.
When kern.sched.use_scores is 1 (the default), sched_mic(4) derives placement costs directly from those scores using cost = (m - s) * 384 / m (where s is the CPU's score and m is the best score across all CPUs). CPUs within 1/16 of the best score (cost at most 24) rank alongside performance cores for work stealing and class swapping, so slightly lower-scoring P-cores remain peers while throttled P-cores rank with efficiency cores.
On processors that also support Intel Thread Director (CPUID.06H:EAX[23]), hfi(4) enables per-thread hardware classification (4 classes on current processors) by default:
sched_mic(4) reads the running thread's hardware class once per statistics clock tick and places it using that class's cost table. A thread whose instruction mix runs nearly as well on an E-core as on a P-core is steered less strongly toward P-cores than an AVX-heavy thread.HRESET, sched_mic(4) executes HRESET at every context switch so one thread's execution history is not attributed to the next thread.kern.sched.itd_swap=1 (the default), each pass of the periodic load balancer checks running threads and asks a pair to swap CPUs when doing so lowers the sum of their class costs by more than kern.sched.itd_swap_margin (default 32 load points). Stability (kern.sched.itd_swap_stable, default 8 ticks), cooldown (kern.sched.itd_swap_cooldown, default 2 seconds in hz ticks), and per-pass limits (kern.sched.itd_swap_max, default 2 pairs) prevent wavering threads from bouncing between cores.Note that once hfi(4) hands table memory to the processor, those pages must remain allocated for the rest of the boot. As a result, hfi(4) cannot be detached or unloaded once attached (returning EBUSY).
Use sysctl(8) to verify that the MIC scheduler, hfi(4), and Thread Director are active and inspect live scores and classification counters.
Check the active scheduler and score mode:
sysctl kern.sched.name kern.sched.use_scores kern.sched.scores_active
Check hardware Thread Director support and hfi(4) attachment:
sysctl hw.itd.supported hw.itd.hreset hw.itd.classes
sysctl dev.hfi.0.enabled dev.hfi.0.itd dev.hfi.0.classes
View the per-CPU HFI table summary (each entry shows cpuN:row:perf/eff, followed by additional ,perf/eff pairs for each Thread Director class):
sysctl dev.hfi.0.summary
Inspect cpucap(9) providers, detected core classes (1=P, 2=E, 4=LP-E), and computed scheduler costs and ranks:
sysctl kern.cpucap.providers kern.cpucap.class
sysctl kern.sched.cpu_cost kern.sched.cpu_rank kern.sched.class_cost
Monitor Thread Director classification and swap activity:
sysctl kern.sched.itd_active kern.sched.itd_classes
sysctl kern.sched.itd_reads kern.sched.itd_invalid kern.sched.itd_changes kern.sched.itd_hist
sysctl kern.sched.itd_swaps kern.sched.itd_swap_follows kern.sched.itd_swap_skipped
On some processors (such as 12th-gen Alder Lake mobile parts), the hardware writes the HFI table once at boot and may report invalid class reads while a thread is on an E-core; in that case sched_mic(4) preserves the thread's last valid class. Newer parts (such as Core Ultra processors) update the table dynamically at run time, which can be observed via rising dev.hfi.0.pkg.N.interrupts and dev.hfi.0.pkg.N.updates counters.
The table below summarizes the primary loader tunables (set in /boot/loader.conf) and runtime sysctl variables (set with sysctl(8) or /etc/sysctl.conf):
| Variable | Type | Default | Description |
|---|---|---|---|
hw.hfi.itd |
Loader tunable | 1 |
Enable Intel Thread Director at hfi(4) attach when supported by hardware. |
hint.hfi.0.disabled |
Loader tunable | 0 |
Set to 1 to prevent hfi(4) from attaching at boot. |
dev.hfi.0.delay_ms |
Tunable / sysctl | 100 |
Debounce delay in milliseconds between an HFI table update and publishing to cpucap(9). |
kern.sched.use_scores |
Tunable / sysctl | 1 |
Derive CPU placement costs from dynamic HFI scores instead of fixed class weights. |
kern.sched.itd |
Sysctl | 1 |
Use per-thread Thread Director class tables when hw.hfi.itd and use_scores are active. |
kern.sched.itd_swap |
Sysctl | 1 |
Allow the periodic balancer to swap running threads between core classes. |
kern.sched.prefer_compute |
Sysctl | 1 (0 on AMD X3D) |
Set to 1 to prefer performance/compute cores, or 0 to prefer efficiency/3D V-Cache cores. |
kern.sched.steal_delay |
Sysctl | ~4 ms (in ticks) | Ticks a thread must wait in a queue before an idle worse-class CPU may steal it. |
kern.sched.smt_busy_penalty |
Sysctl | 192 |
Penalty for an idle SMT sibling of a busy core. Set to 0 to restore sched_ule(4) SMT behavior. |
sched_ule(4)If you want to disable Thread Director, score-based placement, or hfi(4) without rebuilding the kernel, add the appropriate option to /boot/loader.conf and reboot:
# Disable Thread Director only (keep HFI base scores)
hw.hfi.itd="0"
# Or disable score-based placement (fall back to static class weights)
kern.sched.use_scores="0"
# Or prevent hfi(4) from attaching entirely
hint.hfi.0.disabled="1"
At runtime, you can also toggle Thread Director placement or swapping without rebooting:
doas sysctl kern.sched.itd=0
doas sysctl kern.sched.itd_swap=0
To build a custom kernel using the traditional sched_ule(4) scheduler instead of sched_mic(4) and hfi(4), create a kernel configuration file that includes GENERIC and removes both:
include GENERIC
ident MYKERNEL
nooptions SCHED_MIC
nodevice hfi
Developers and testers can exercise hybrid placement and score tables on homogeneous systems or virtual machines using cpucap(9) and sched_mic(4) overrides:
kern.cpucap.force_classes (boot-time tunable): accepts a comma-separated list of CPU classes (1=P, 2=E, 4=LP-E), such as kern.cpucap.force_classes="1,1,1,1,2,2,2,2,4,4".kern.cpucap.force_scores (sysctl): overrides per-CPU scores with comma-separated perf/eff pairs (e.g. 60/92,60/92,34/100,34/100). Set to an empty string to clear.kern.sched.mic.itd_force_class (sysctl): forces a Thread Director class (-1 off, 0..3 for a specific class, or 8 to spread threads across classes by thread ID).sched_mic(4), hfi(4), cpucap(9), sched_ule(4), sysctl(8), loader.conf(5)