Unsurprising. Drivers are better than they used to be, but some of them (Nvidia) have a long way to go in terms of optimization.
More importantly, however, is the complete lack of info the article provides about their testing methodology.
They said they tested on SteamOS—ok, but it’s not officially available on non-handheld devices. How did they install it? Did they use HoloISO? Did they install the version meant for handhelds on a desktop PC?
How did they run the games? Directly through an embedded gamescope session like the Steam Deck, or through KDE Plasma, which has a compositor that can’t be disabled on Wayland. Or, did they take the double hit and run gamescope as a window within Plasma?
Speaking of Wayland, did they use Wayland or X? They have different performance characteristics, and it’s not negligible.
How many runs did they do? One-and-done, then record what the game said the average FPS was? Average of 5 runs? Were runs with outliers excluded and retested?
Did they pre-run the scenes to ensure the assets were cached from the disk and the shader caches were available? Did they restart the system between games? Did they restart the system between runs?
And the way they present the results are also bad:
They graph the FPS achieved by each platform, but they have absolutely no detail about the 1% or 0.1% lows—and at a sufficiently-high average FPS, these are what make the games feel slow and stuttery.
What about frametime graphs and frame pacing information? If Linux can achieve more consistent pacing at 85% of the average FPS, it would still be a better experience than having the same frame being presented repeatedly because the game missed the vblank window.
They didn’t try multiple resolutions to identify where the bottlenecks are occuring in each game. If a game is CPU bottlenecked by their hardware choices, it’s not a good comparison of GPU performance. Likewise, if it’s GPU bottlenecked, it’s not a good comparison for CPU performance.
Unsurprising. Drivers are better than they used to be, but some of them (Nvidia) have a long way to go in terms of optimization.
More importantly, however, is the complete lack of info the article provides about their testing methodology.
They said they tested on SteamOS—ok, but it’s not officially available on non-handheld devices. How did they install it? Did they use HoloISO? Did they install the version meant for handhelds on a desktop PC?
How did they run the games? Directly through an embedded gamescope session like the Steam Deck, or through KDE Plasma, which has a compositor that can’t be disabled on Wayland. Or, did they take the double hit and run gamescope as a window within Plasma?
Speaking of Wayland, did they use Wayland or X? They have different performance characteristics, and it’s not negligible.
How many runs did they do? One-and-done, then record what the game said the average FPS was? Average of 5 runs? Were runs with outliers excluded and retested?
Did they pre-run the scenes to ensure the assets were cached from the disk and the shader caches were available? Did they restart the system between games? Did they restart the system between runs?
And the way they present the results are also bad:
They graph the FPS achieved by each platform, but they have absolutely no detail about the 1% or 0.1% lows—and at a sufficiently-high average FPS, these are what make the games feel slow and stuttery.
What about frametime graphs and frame pacing information? If Linux can achieve more consistent pacing at 85% of the average FPS, it would still be a better experience than having the same frame being presented repeatedly because the game missed the vblank window.
They didn’t try multiple resolutions to identify where the bottlenecks are occuring in each game. If a game is CPU bottlenecked by their hardware choices, it’s not a good comparison of GPU performance. Likewise, if it’s GPU bottlenecked, it’s not a good comparison for CPU performance.