Current trends in Linux Kernel Power Management

Two different concepts are distinguished in power management: suspending and runtime power management. The former means techniques like standby (ACPI^[5] S1), suspend to memory (ACPI S3) or disk (ACPI S4) where the computer is put into a non-working “sleeping” state. In contrast, runtime power management attempts to provide uninterrupted usability of the system, and so tries to stay hidden from the user point of view.

As this paper focuses on runtime power management, here follows only a short description on these techniques: The standby mode stops the CPU, but with all relevant data continuing to be stored in memory, normal operation can be resumed within short time. When suspending to memory, the CPU and several other devices are powered down and need to be re-initialized with information stored in system memory. This means the waking process takes a bit longer. Suspend to disk saves the userspace state into an image on a hard disk and then powers off the computer completely.^[6] When the computer is powered on the next time, the image is restored and terminals, editors and Internet browsers will show the same content and will behave exactly the same way as before the suspending command having been issued.

Even though the ACPI standard^[7] attempts to provide an operating-system independent way to put a computer into such a sleeping state and to awake it again, the actual implementations need to be aware of many hardware-specific issues. With several manufacturers withholding the information necessary to write appropriate drivers, support for suspending devices is still highly experimental in Linux.^[8]

When there is no work to do, certain parts of the CPU – the actual code execution units, for example – can be shut down and re-activated once they are required for operation again.

On modern processors there exist multiple different such “idle states”^[13], which allows to have states available where the CPU can be reactivated most quickly, but the power savings are limited (on the x86 architecture, this is the “hlt” command), and other states where on the one hand more power is saved, but on the other hand the wakeup latency – the time from an interrupt event to the CPU being ready to execute code again – increases.

Table 1 shows the energy consumption of some common CPUs of the x86 architecture depending on the idle state. With reductions of up to 60 to 98% compared to “continuous operation” and common CPU usage rates of 5 to 50%, this means a reduction in CPU energy consumption somewhere between 30% and 90%.

With more different idle states being implemented in hardware which allow for a more fine-grained compromise between energy consumption rates and responsiveness, the operating system needs to be “smart” enough to activate them. On the x86 hardware, only the ACPI subsystem can utilize multiple idle states, and even it was designed for only three different idle states at first. The ACPI specification revision 2.0 added a new method called “_CST” for communication between the BIOS and the operating system to allow for even more idle states. Starting with kernel 2.6.11; support for this method was added to the Linux kernel. You can check which ACPI-based idle states are available on your system by cat'ting /proc/acpi/processor/*/power.

As one physical CPU can only be put into an idle state as a whole, processors implementing synchronous multi-threading (SMT, for example HyperThreading) or multiple sub-CPUs on one CPU die (multi-core CPUs) need special care when they are to be put into an idle state. Also, on platforms using multiple physical CPU packages, special care needs to be taken that the CPU caches maintain valid content.^[14] Due to these obstacles, idle states higher than C1 are not present in many BIOSes and platforms, and the Linux kernel even lacked support for them as of release 2.6.12. A small and simple patch already included into the ACPI development tree adds support for real, effective idle states on multiprocessor systems. As it is a hardly invasive patch, it is very likely to be merged into kernel 2.6.13 and might already be available in kernels used at the time of LinuxTag 2005.

While the much discussed change from 100 HZ to 1000 HZ in kernel 2.6. – meaning the kernel is activated 1000 times a second (“ticks”) to determine which process is to run next and to fulfil some “housekeeping” tasks – provides better interactivity, it also means the time between entering and leaving idle states decreased dramatically. Taking into consideration that the putting into sleep and waking from sleep also consumes energy^[15] it becomes clear that the power usage increased because of this change. Therefore, certain distribution kernels are specially modified for notebooks and continue to run with 100 HZ on mobile devices.

The technically superior approach is to not wake up every millisecond, but only when there actually is a new task to run (“dynamic ticks” or “tickless systems”) or interrupt activity (e.g. the user hitting a key on the keyboard) demands reaction by the kernel and userspace programs. With the Linux kernel relying heavily on the concept of “jiffies” (which is a counter incremented each “tick”) for timers, fair scheduling, and the interrupt generating hardware needing special care to be deactivated or to run at modifying rates, making the kernel “tickless” is a demanding and tough task. With the timing core of the kernel likely to undergo a major overhaul^[16] making it aware of tickless systems, one obstacle seemed to disappear soon. Surprisingly, a group of developers did not even wait for this feature and proved with a medium-sized patch^[17] that tickless systems are not as far away as feared. Nonetheless, this approach still needs to be considered highly experimental and the modified code should not be used on so-called production systems yet.

What makes this such an interesting feature? If the CPU clock frequency is lowered, the energy consumption is reduced linearly. However, the voltage driving the CPU can be lowered as well, resulting in a highly increased “instruction per energy consumption” ratio^[18]. In plain speak: if you accept to wait longer for the result, you can compute much more data with the same amount of energy. Taking into consideration that usually there is not much work to do for the CPU – even watching a DVD is not very CPU intensive, and most often there are other “bottlenecks” like internet connections – and frequency and voltage scaling also has positive side-effects on CPU Power States^[19], this is most definitely something a user should take use of when running a notebook on battery power.

Support for CPU frequency (and voltage) scaling is highly hardware-dependant. Therefore, only a minority of all platforms have CPU frequency scaling implemented; however with the emergence of “AMD Cool&Quiet” and “Intel Enhanced SpeedStep” this technology has propagated to the server market.

In the Linux kernel, support for CPU frequency and voltage scaling is provided by the cpufreq subsystem in all 2.6. kernels^[20], and more and more hardware drivers are added. However, especially the userspace side of cpufreq is still in heavy development, with different daemons and tools springing out of the ground at a high rate. A common midlayer, cpufrequtils, is emerging, and using its cpufreq-info tool you can determine whether cpufreq is activated on a system.

While frequency switching only occurring on changes of the power source – an AC adapter being attached or removed – already provides important reductions in energy consumption and thus was used in first-generation CPU frequency scaling implementations, on-demand switching between multiple frequency levels leads to continuous heat reduction and allows to utilize the CPU fully even on battery power, with energy consumption suffering only a minor increase.

One physical CPU package must run at one frequency, meaning there needs to be some sort of coordination between multiple logical or physical CPUs. This was achieved by changing cpufreq to be “processor package”-centric instead of “logical CPU”-centric. Another problematic issue is that the primarily used timing source on the x86 architecture, the Time Stamp Counter (TSC), is located on the CPU die and therefore is affected by a changing CPU frequency. Especially on SMP systems you should use different timing sources (ACPI Power management Timer, HPET) to achieve sufficient precise results, even if they are slower to access.

In contrast to CPU frequency scaling, where the operating frequency is constantly modulated, throttling means the CPU is forced to a halt for short periods of time. If throttled, the CPU into a physical and electrical state comparable to the idling states mentioned above, so it can be described as an “enforced idling” of the CPU.

As certain CPU power state typically utilize similar hardware implementations,^[31]and as throttling does not have a positive effect on the energy consumption during CPU power states, throttling the CPU by a given rate is only useful if the CPU is less idle than the throttling rate.

Therefore, throttling makes only sense if the CPU temperature has become too hot because the CPU was active excessively. As throttling “forces” some “idling”, which then lowers the energy consumption and heat generation, it is a good tool for “passive cooling”. Passive cooling – in contrast to active cooling which utilizes fans – is done by lowering the CPU load and works best if using CPU frequency scaling for it, but is also possible using throttling.

Throttling is implemented in a few cpufreq processor drivers, most notably p4-clockmod,^[32] and on ACPI-based platforms it is user-controllable using the file „/proc/acpi/processor/*/throttling“.

While the CPU is in the throttling state, on typical Intel or AMD x86 and x86_64 CPUs it consumes as much power as in the Stop Grant (Cache Snoopable) state. Depending on how high the throttling rate is, and how idle the system is, the CPU energy consumption will vary.

Assuming an idleness rate of zero, the power usage rates of some CPUs of the x86 architecture can be seen in table 2. Assuming one specific computing task needs 1s to finish at 100% CPU power available, the CPU energy usage for this task actually increases if throttling is used (table 3).

This shows that throttling itself does not save battery power at all – it even increases battery usage of any specific computing task. However, if a fan doesn't need to be started because of passive cooling, or if certain tasks do not need to be run (e.g. the screen is normally refreshed each 10ms, but the CPU is throttled so much it misses the deadline every second occasion, so the screen is only refreshed each 20ms), energy consumption may be reduced slightly overall.

It is important not to shut down a device, or to put it to sleep, if it is needed by other devices still in use. The integrated device and driver model developed for kernel 2.6. attempts to show these dependencies.^[33] However, appropriately describing all logical, technical and electrical dependencies continues to be a major hurdle.

However, during normal operation the hard disk is accessed quite often: whenever a file^[38] or information related to a file^[39] is changed, the kernel waits at most five seconds until the information is written out to disk. Using the “laptop mode” tuning available in 2.6. kernels, this delay is increased. Also, the kernel tries to batch disk access, so that whenever the hard disk is woken up, all possibly pending requests are handled so that the hard disk can sleep for a longer time again, hopefully.

The scripts to run “laptop mode” are both included in the kernel sources^[40] and in distribution packages. They are highly configurable, and the possible settings are described in an excellent manner both in the kernel^[41] and in the configuration file itself^[42].

Also, manipulating userspace may reduce the disk accesses – for example, the CUPS daemon and any logging utilities commonly write data out at a continuous rate. Therefore, you can consider whether the loss of this functionality – or, at least, its reduction by storing the log file on a tmpfs partition – outweigh the increase in battery usability, and – as an added bonus – noise reduction. The CPU usage of these tools can usually be ignored, though.

The Linux device model^[43] offers a unified interface to put devices into a freeze or other runtime^[44] power management states, as this feature is needed for proper suspend to disk support. While an interface to userspace exists for each device in the file power/state inside the sysfs representation of a physical device, this interface is almost completely unusable at the moment.^[45] As it is intended to be fixed soon, here follows only a short description as of kernel 2.6.12-rc4: echoing “3” into power/state asks to put the device into a freeze^[46], writing “0” to this file puts the device back into full power.

However, PCI devices which do not have a driver attached are not put into a low power or even off state at the moment. Also, devices are not disabled or put into an off state which might offer even greater savings.