Mesa Lands New Single File Cache To Help Steam's Pre-Compiled Shaders, Space Savings

[phoronix]

Phoronix: Mesa Lands New Single File Cache To Help Steam's Pre-Compiled Shaders, Space Savings

Mesa's on-disk shader cache, which is used for speeding up game load times by avoiding the redundant recompiling of shaders on successive loads and also helping performance for software that compiles shaders on-the-fly, is seeing a big improvement with Mesa 21.1. Mesa 21.1-devel merged this weekend the new single file cache implementation...

http://www.phoronix.com/scan.php?page=news_item&px=Mesa-Single-File-Cache

[mazumoto]

Hm, I never really understood that shader preloading in steam. Aren't shaders driver specific in the sense that they must match exactly the mesa version I'm running on? And since I'm often on mesa from git, it'd recompile everything anyway?

But probably I'm mistaken here ...

[MrCooper]

Hm, I never really understood that shader preloading in steam. Aren't shaders driver specific in the sense that they must match exactly the mesa version I'm running on? And since I'm often on mesa from git, it'd recompile everything anyway?

But probably I'm mistaken here ...

You’re spot on. It can only make a difference if you run the exact same build of Mesa as others before you. I.e. not for people which build Mesa themselves (which is probably a small minority).

[Chewi]

Hm, I never really understood that shader preloading in steam. Aren't shaders driver specific in the sense that they must match exactly the mesa version I'm running on? And since I'm often on mesa from git, it'd recompile everything anyway?

But probably I'm mistaken here ...

I've been wondering the same thing too. Am I just wasting bandwidth downloading these? Why do they get updated so often? It's not because I've updated Mesa because it happens more frequently than that.

[schmidtbag]

This sounds like a welcome change to me. But like others pointed out, too bad you have to keep re-downloading the caches.

[pal666]

But probably I'm mistaken here ...

your only mistake is to assume that valve handles you rather than majority of their users

[pal666]

But like others pointed out, too bad you have to keep re-downloading the caches.

it's unclear what part of it is bad. alternative to downloading precompiled ones is local compilation, which is much longer

[BNieuwenhuizen]

You’re spot on. It can only make a difference if you run the exact same build of Mesa as others before you. I.e. not for people which build Mesa themselves (which is probably a small minority).

Actually Steam has 2 caches:

1. That caches shader binaries and indeed has the problems you pointed out.
2. A cache that stores the pipelines for Vulkan replay. These can be compiled before launching the game to avoid stutter.

[Pentarctagon]

Please post a link to Valve's global statistics about shader caches in Steam Linux clients. Without data, any opinion is just as good as your opinion, irrespective of whether those opinions are contradictory.

Even with a lack of data, reasonable guesses can be made. It's reasonable to guess that that Valve doesn't create shader caches for every mesa commit for every game they create the caches for only for the sake of being able to provide them for the comparatively small number of people who compile mesa from git themselves.

[randomsalad]

I've been wondering the same thing too. Am I just wasting bandwidth downloading these? Why do they get updated so often? It's not because I've updated Mesa because it happens more frequently than that.

There are many shaders for many games. They're probably adding shader binaries from more games with each update for a variety of different mesa point releases and flavors so that everyone will have a smooth experience even playing the less popular games. I don't know if Steam downloads shaders for all games though, or if it focuses specifically on the game you're starting.