In-Kernel Power Management For ATI KMS

[lucky_]

No worries goodnight.

[bridgman]

By looking at the radeon_pm_set_state function, I see that you skipped the memory and voltage setting, isn't it a problem ? Are the frequencies of the components paired ? Or can you access the memory at every frequency from an engine at any given frequency ?

Memory clocks are more complicated than engine clocks, in the sense that memory timing depends on both engine activity and display activity (number of screens, resolution/refresh rate of each display, image depth, image tiling etc..).

Prior to having KMS it wasn't really practical to play with memory speeds since there was no place in the driver stack which had access to 2D, 3D, video and display activity information (unless a new protocol were defined to pass display info down to the kernel).

You also need to block all display and drawing activity when changing memory clocks to avoid corruption and again that could only be done in a KMS DRM.

Adjusting memory clocks on a GDDR5 system is more complicated because the memory subsystem needs to go through a training cycle when clocks are changed, and that can sometimes be too long to fit into a display blanking period (along with the obvious problem of what to do with two displays running at different refresh rates).

[Louise]

Adjusting memory clocks on a GDDR5 system is more complicated because the memory subsystem needs to go through a training cycle when clocks are changed, and that can sometimes be too long to fit into a display blanking period (along with the obvious problem of what to do with two displays running at different refresh rates).

I hope this isn't a too stupid question. But why is power management done in software on the cpu?

Couldn't the GPU "power manage" itself in hardware?

[bridgman]

Actually a very good question.

In principle, yes the hardware could be completely self sufficient for power management.

In practice, however, making the right power management decisions usually requires knowledge outside what the GPU or CPU has available. As an off-the-top-of-my-head example, you want to power down the 3D engine very aggressively when there is no drawing activity, but if your game stalls while swapping in a big texture you probably would *not* want to power down the 3D engine in that case. That decision heuristic would require that power management for the GPU be cognizant of disk activity.

The preferred approach for managing power is also changing over time, with a slight trend from "run the hardware as slowly as possible while maintaining acceptable performance" to "run at full speed, get the work done fast and then switch everything off as quickly as possible until more work comes along". As you can imagine, it helps a lot of the CPU and GPU power management follow the same approach

When setting memory clocks you need to make some pretty complex bandwidth and latency calculations in order to make sure that the display fifos don't get starved for data when the 3D engine is drawing into the same memory. One implication of this is that the minimum memory clock setting is a function of engine clock, drawing activity, number of active displays, depth / resolution / refresh rate.

Since changing clocks takes a non-trivial amount of time (you need to let the PLLs in the clock generators settle at the new frequency after changing dividers) you need to include recent behaviour trends in the decision-making logic, not just what is happening at this instant, in order to avoid too-frequent changes with the associated waste of power and time. All of this can be done in hardware but you can see how quickly the complexity can grow.

Anyways, bottom line is getting the very best power efficiency always seems to require making decisions using more information than what is available to each individual device, ie shifting from focusing on device power management to focusing on platform power management.

[Zajec]

Thanks, Zajec!

Do you know by chance something about reading out temperatures? I am very interested in that.

Don't know. I just checked AtomBIOS commands but don't see anything related to temperature there. Just
GPIOPinControl command makes me wondering. Can sb explain what for is GPIO used on ATI cards?
So we probably have to use some magic not-yet-known register to get temperature.



lucky_: OK, I see bridgman replied to you.


I still don't know how we should calculate load. Someone posted yesterday idea of checking command buffer (it's full → GPU too slow). Don't know much about that, have to check how CS works.

Also I don't know what modes we can use for whole clocking. Anything between minimum and maximum? Or only states found in AtomBIOS?

[DoDoENT]

As soon as these patches hit drm-next, I'm going to continue my work and prepare something really useful. Unfortunately the most complicated part is still ahead of us: determining which power state should be used in current moment. It needs measuring current load and some smart (and lucky) choosing mode. Remember that every mode has engine clock, memory clock and voltage. We have to somehow calculate all of that.

I'm not the expert, but tell my why is important for driver to choose the best power state to be used in current moment? Isn't just possible to make some file, such as /sys/*/radeon/power_state which will control the current power state? I mean: if I write into terminal:
cat /sys/*/radeon/power_state
it should print something like the following:
Current power state: 2 (x MHz Core Clock, y MHz Memory Clock)
And I could change the power state by entering e.g.: echo 1 > /sys/*/radeon/power_state
And available power states for current graphics card would be available in another file, e.g. /sys/*/radeon/power_states, which would be readonly.

In that case, determining which power state should be used in current moment could be left to the higher-level programs, e.g. PowerDevil, laptop-mode, or some custom user scripts.

Is it possible to enable this way of switching power states in radeon driver? In this way works the iwl3945 drivers's power management (of course iwl3945 is the WiFi driver, and I'm just curious if it's possible to create the similar power management architecture for graphics driver as is for wifi driver?)

Thank you in advance for your answers.

[Zajec]

I'm not the expert, but tell my why is important for driver to choose the best power state to be used in current moment? Isn't just possible to make some file, such as /sys/*/radeon/power_state which will control the current power state? I mean: if I write into terminal:
cat /sys/*/radeon/power_state
it should print something like the following:
Current power state: 2 (x MHz Core Clock, y MHz Memory Clock)
And I could change the power state by entering e.g.: echo 1 > /sys/*/radeon/power_state
And available power states for current graphics card would be available in another file, e.g. /sys/*/radeon/power_states, which would be readonly.

In that case, determining which power state should be used in current moment could be left to the higher-level programs, e.g. PowerDevil, laptop-mode, or some custom user scripts.

Is it possible to enable this way of switching power states in radeon driver? In this way works the iwl3945 drivers's power management (of course iwl3945 is the WiFi driver, and I'm just curious if it's possible to create the similar power management architecture for graphics driver as is for wifi driver?)

Thank you in advance for your answers.

What you talk about is userspace power management. This can be eventually introduced, but can not be the only solution.

Userspace just can not know about everything that happens in GPU. It has only some basic idea about current load (amount of apps, maybe Xv and 3D detecting). That's not enought. Only kernel side can have full view on load.

However as I said, we will want to introduce something you proposed anyway. For example in case of using battery power we don't know if user still prefers performance, or long battery life.

For some additional details you may also check discussion on dri-devel: http://sourceforge.net/mailarchive/m...mail.gmail.com

[bridgman]

What Zajec said

Doing simple power management in userspace is no problem, but doing the kind of aggressive real-time power management that you need for best battery life on a laptop either requires the decision making code to be in the kernel or a protocol to be defined that passes all the information up to userspace so that information can be accumulated and decisions made there.

It doesn't really matter where the other than (a) it's easier for userspace code to have access to user inputs etc,, (b) all the information comes from the kernel and decisions end up in the kernel so doing the first implementation *in* the kernel is probably less work unless all of the necessary user/kernel protocols already exist (which they might, of course).

[Louise]

Actually a very good question.

In principle, yes the hardware could be completely self sufficient for power management.

In practice, however, making the right power management decisions usually requires knowledge outside what the GPU or CPU has available. As an off-the-top-of-my-head example, you want to power down the 3D engine very aggressively when there is no drawing activity, but if your game stalls while swapping in a big texture you probably would *not* want to power down the 3D engine in that case. That decision heuristic would require that power management for the GPU be cognizant of disk activity.

The preferred approach for managing power is also changing over time, with a slight trend from "run the hardware as slowly as possible while maintaining acceptable performance" to "run at full speed, get the work done fast and then switch everything off as quickly as possible until more work comes along". As you can imagine, it helps a lot of the CPU and GPU power management follow the same approach

When setting memory clocks you need to make some pretty complex bandwidth and latency calculations in order to make sure that the display fifos don't get starved for data when the 3D engine is drawing into the same memory. One implication of this is that the minimum memory clock setting is a function of engine clock, drawing activity, number of active displays, depth / resolution / refresh rate.

Since changing clocks takes a non-trivial amount of time (you need to let the PLLs in the clock generators settle at the new frequency after changing dividers) you need to include recent behaviour trends in the decision-making logic, not just what is happening at this instant, in order to avoid too-frequent changes with the associated waste of power and time. All of this can be done in hardware but you can see how quickly the complexity can grow.

Anyways, bottom line is getting the very best power efficiency always seems to require making decisions using more information than what is available to each individual device, ie shifting from focusing on device power management to focusing on platform power management.

Okay that's just crazy the things that is going one behind the scene

I assume what you just described is the most advanced and probably the latest to get implemented?

So KMS is power management's hook into the 3D engine/OpenGL*?

I assume 3D engine and OpenGL is the same...?

[DoDoENT]

What Zajec said

Doing simple power management in userspace is no problem, but doing the kind of aggressive real-time power management that you need for best battery life on a laptop either requires the decision making code to be in the kernel or a protocol to be defined that passes all the information up to userspace so that information can be accumulated and decisions made there.

It doesn't really matter where the other than (a) it's easier for userspace code to have access to user inputs etc,, (b) all the information comes from the kernel and decisions end up in the kernel so doing the first implementation *in* the kernel is probably less work unless all of the necessary user/kernel protocols already exist (which they might, of course).

I understand.

But how about enabling just userspace power state switching by the time Fedora 12 ships? It would mean a lot to advanced users which would like to have at least some power management (better that than none). I know this wouldn't offer the best battery life for laptops, but it will increase it... at least as it was while I was using fglrx (the difference is approx. 35-40 minutes).