However, I have some kind of mental barrier against playing games in windowed mode instead of fullscreen. It's just not the same. I know, I know, I should seek help :P
Choose your max resolution in the launcher, then right click the window border (shortcut: Alt+F3, for when the border is hidden), Advanced, Check "No border" and "Keep Above Others".
That should work around it while still providing a fullscreen solution