TurboSched Is A New Linux Scheduler Focused On Maximizing Turbo Frequency Usage

[phoronix]

Phoronix: TurboSched Is A New Linux Scheduler Focused On Maximizing Turbo Frequency Usage

TurboSched is a new Linux kernel scheduler that's been in development by IBM for maximizing use of turbo frequencies for the longest possible periods of time. Rather than this scheduler trying to balance the load across all available CPU cores, it tries to keep the priority tasks on a select group of cores while aiming to keep the other cores idle in order to allow for the power allowance to be used by those few turbo-capable cores with the high priority work...

http://www.phoronix.com/scan.php?page=news_item&px=Linux-TurboSched-V4

[GreenReaper]

I could see this having a real impact, however it could as easily be negative as it is positive. Still... the situation they envisage is what I see on an application server. There is a tendency for the system to spread work out among the cores and bring down the ability to reach the highest boost bins.

Existing benchmarks may not be well-suited to test this if they seek to be deterministic - either with one core, or a specific number of cores, doing exactly the same thing all the time. Real life isn't like that.

The article seems to link to one sub-patch, the abstract of the patch thread is here:

https://lkml.org/lkml/2019/7/25/296

[carewolf]

I could see this having a real impact, however it could as easily be negative as it is positive. Still... the situation they envisage is what I see on an application server. There is a tendency for the system to spread work out among the cores and bring down the ability to reach the highest boost bins.

Existing benchmarks may not be well-suited to test this if they seek to be deterministic - either with one core, or a specific number of cores, doing exactly the same thing all the time. Real life isn't like that.

The article seems to link to one sub-patch, the abstract of the patch thread is here:
https://lkml.org/lkml/2019/7/25/296

Games benchmarks might get a boost, especially games that currently favors single-threaded performance (Intel).

[AsuMagic]

Games benchmarks might get a boost, especially games that currently favors single-threaded performance (Intel).

Hm. To my eyes, this seemed rather to be destinated at maximizing throughput of tasks with no concern for interactivity, which matters for games.

[schmidtbag]

I'm a little surprised nobody has bothered to try this sooner. Seems a little too obvious.

I could see this having a real impact, however it could as easily be negative as it is positive.

What negatives do you have in mind (on a modern CPU)? The only negative side effects I'd see is maybe on newer Intel CPUs with a limited duration PL2. It really depends on how the CPU keeps track of how long it has been boosted. For example, if the CPU resets (or at least reduces) its boost counter whenever it switches cores then this scheduler could negatively impact how long it'll remain boosted.

[GreenReaper]

Well, I have in mind a need for processing that causes latency. Say, a query comes in that parallelizes and demands eight cores. But you put those cores into a deep C-state to save power. Suddenly, they're not there when needed. Of course, this could be the case for any situation, but the way it works it may make it more likely than something which makes it easy to grab a hot (if not as fast) CPU.

[xorbe]

This is a tough one because it's a few hundred MHz vs L1/L2 cache thrashing. I think I would personally prefer consistent speed across multiple cores as a desktop user. This is the sort of thing that one keeps in the tuning tools hand bag for specific applications.

[schmidtbag]

Well, I have in mind a need for processing that causes latency. Say, a query comes in that parallelizes and demands eight cores. But you put those cores into a deep C-state to save power. Suddenly, they're not there when needed. Of course, this could be the case for any situation, but the way it works it may make it more likely than something which makes it easy to grab a hot (if not as fast) CPU.

Most CPUs are still in the hundreds of MHz when idle. So, let's say your CPU idles at 500MHz. That's a 2 nanosecond delay between cycles. That's pretty much the longest you'll have to wait on idle cores, since the cores should otherwise ramp up their frequency once needed. If that delay is too long for you, I think your power profile is going to be a greater concern than the scheduler.

[pegasus]

I'm a little surprised nobody has bothered to try this sooner. Seems a little too obvious.

Sadly at workloads where performance really matters (HPC and such) this would have negligible impact on runtime but major impact on power usage. Most HPC workloads are limited by available memory bandwidth. I can demonstrate a pathological case where running cpu at 3.6GHz finishes only 13% faster than running it at 1.2GHz. Now imagine energy-to-result metric and you suddenly realize that you're better off with running your clusters at lower frequencies. That's the main reason why server grade cpus all run on lower frequencies than desktop grade ones. And this will hold true until server grade cpus come standard with some HBM on the cpu die. Intel Xeon phi aside, this should happen in 2021 or 2022.

[starshipeleven]

What negatives do you have in mind (on a modern CPU)?

Afaik Intel CPUs are actually disabling (putting in low C-states or even shutting down completely) unused cores when in turbo boost mode. Thawing these cores isn't fast.

AMD CPUs don't care, they also have no artificial limit on boost clock duration. Only limit is thermal (= physical), if your heatsink is able to cool the system, the CPU can stay in "boost" indefinitely.

I strongly suspect that also POWER9 processors don't have stupid limitations on duration nor go and disable cores.