PTS: Savage 2 / HoN?

[allquixotic]

The K2 engine games (Savage 2 and Heroes of Newerth) might be interesting to add to PTS to get a different perspective on testing OpenGL implementations on different platforms:

1. The 3d engine is written from scratch, so it will display distinct performance characteristics from the usual Unreal-based / idtech based engines.

2. It isn't as brutally demanding as Unigine, so it can run on pretty old hardware, as well as the latest Gallium3d drivers (at least for nouveau and radeon). It uses OpenGL 2.1 with shader language 1.20, which is just barely what Gallium3d supports right now.

3. They have a 64-bit version for Linux, which means you can compare the performance vs. the 32-bit version

4. It uses the best-supported API available on the platform: on Windows, it uses Direct3d 9.0c, and on Mac/Linux it uses OpenGL 2.1.

Because it runs on Windows, Mac and Linux, you can probably use rEFIt and/or Boot Camp to try it on Windows, Mac and Linux on a new Mac Mini (with the GT200 based IGP) and compare the FPS on OS X, Linux and Windows using the same hardware.

Of course, Apple's OpenGL performance has been steadily regressing for months, which is ironic considering that a very major player in Valve has just brought the Source engine to Mac. It very well may be that with a Mac Mini, you can get better performance using Linux and the Nvidia binary driver (or maybe even Nouveau!) than you can on good ol' XNU.

Anyway, what do you guys think about adding Savage 2 or HoN (or both) to PTS?

[Zhick]

I think Michael would have added HoN to the PTS a long time ago if it was that trivial. I guess the problem is that HoN doesn't have any "benchmarking mode" like most FPS-games have, where it reports the frames/sec.

[L33F3R]

I think Michael would have added HoN to the PTS a long time ago if it was that trivial. I guess the problem is that HoN doesn't have any "benchmarking mode" like most FPS-games have, where it reports the frames/sec.

nail on the head.

i would also like to see this incorporated however i doubt michael has the time to write a script up for HoN.

[Michael]

Right, I would love to support the game/engine, but as far as I know there is no reliable way to automatically run tests with it.

[Naib]

http://forums.heroesofnewerth.com/sh...&postcount=110

his thread is full of a lot of misinformation. Let my try to clear a whole lot of things up. Before I reply to specific posts, let me just say that the word "engine" gets thrown around all over the place, with very little understanding of what it actually entails. As far as it means anything to the average player at this point, the "engine" is really more of a brand name than anything else.

The program that is Heroes of Newerth comes from about 500,000 lines of text, 350,000 of which are actual instructions, the rest being comments, blank lines and other such formating niceties.

This text source code becomes hon.exe, k2.dll, vid_d3d9.dll, game.dll, game_shared.dll and cgame.dll. Those files contain the actual instructions that make everything happen. As a general rule, the stuff that is in k2 is the "engine", the stuff in the various game files are the "game code". Conceptually, engine code is stuff that could apply to any game, game code applies specifically to Heroes of Newerth.

Most of the time.

The thing is, most of the stuff that K2 actually does is stuff you don't even think about. Reading files, receiving input from the keyboard, transmitting network data. The engine is boring stuff, if you aren't a programmer.

The game code is where everything having to do with a hero or ability happens. K2 has no idea what a hero is.

The graphics capabilities are not even a part of the K2 source. The part we refer to as "the renderer" lives in its own little project. A completely new renderer could be written from scratch without changing a line of code anywhere else.

Anyway, with that little lesson out of the way, the firs thing I want to address is the idea that HoN is using an "old engine". Yes, we started with the same code base we had when we finished Savage 2 and yes, we are still calling it "K2", but it isn't the exact same thing by any means. New features were added, improvements were made, things changed. The over all structure is the same, but improvements have been (and continue to be) made. At this point, it is unlikely that we will ever throw away K2 and start from scratch, because there is no reason to. It is very modular and well structured in a way that it can continue to grow for a long time to come. There is no feature that we are flat out unable to add without starting over.

tl;dr
The term engine is thrown around far to liberally and in the context of HoN/Savage the K2 engine is actually a scripting engine NOT a graphics engine

[allquixotic]

tl;dr
The term engine is thrown around far to liberally and in the context of HoN/Savage the K2 engine is actually a scripting engine NOT a graphics engine

Actually, I think that's a misinterpretation of what S2Jason said.

What he said is that the engine is modular, and that they could rip out the graphics engine and write a new one without rewriting most of "K2".

There are two senses of "K2": the K2 Infrastructure, such as you might find in another engine; and the actual implementation of any particular module in a particular game.

For an example, consider this configuration data from Deus Ex, an Unreal engine game:

GameRenderDevice=D3D10Drv.D3D10RenderDevice
AudioDevice=Galaxy.GalaxyAudioSubsystem
NetworkDevice=IpDrv.TcpNetDriver
DemoRecordingDevice=Engine.DemoRecDriver
Console=Engine.Console
GameEngine=DeusEx.DeusExGameEngine
EditorEngine=Editor.EditorEngine
WindowedRenderDevice=SoftDrv.SoftwareRenderDevice
RenderDevice=GlideDrv.GlideRenderDevice
DefaultGame=DeusEx.DeusExGameInfo
DefaultServerGame=DeusEx.DeathMatchGame
ViewportManager=WinDrv.WindowsClient
Render=Render.Render
Input=Extension.InputExt
Canvas=Engine.Canvas

On the left-hand side of these assignments are internal Unreal engine properties that determine what class implements each of the core features: audio, networking, graphics, frame capture, etc.

On the right-hand side of these assignments are Deus Ex's particular implementations of these, and even an example or two of community modding where people have plugged in a completely new renderer based on DirectX 10 (D3D10Drv.D3D10RenderDevice).

What I'm saying is that S2Games has written their own equivalent of this type of pluggable engine, and have plugged into it their own home-grown OpenGL 2.1 and DirectX 9.0c renderers. I don't know how they're called internally, but if (hypothetically) they'd been using the Unreal engine, they might create a line something like this :-

GameRenderDevice=K2_D3D9Drv.D3D9RenderDevice

My point is that they actually programmed a working, production-ready, 3d rendering application, without purchasing a license from a third party, and without using any copyleft software (or they'd have to release the source to it). It's possible they took BSD-licensed code and used it, but I've never heard any claims or mention of that in my 3 years tenure in the S2Games community, despite talking at length about their technology with S2Mercenary.

I think it is to Phoronix's advantage to catalog the performance of as many unique 3d rendering applications -- an application that calls 3d rendering APIs, and which is not inherently derived from another application that uses the same 3d rendering core -- because this will let them factor out engines that have particular problems that cause weird performance graphs (for instance, an engine that is always CPU-bound doesn't properly benchmark GPU performance).

Just to elaborate on my definition of "unique": unfortunately, we can't always tell what's unique and what's not without full source code access to all of the engines we test. Without that, we are left to distinguish between unique products by looking at information provided by developers, analyzing file names (and sizes), and testing performance to see if it's similar.

In a few cases, it's extremely easy to know that a product does not have a unique graphics engine. For one, the default renderers shipped with Deus Ex are one and the same ones that shipped with Unreal. So it would probably not be meaningful to catalog the FPS of Deus Ex and Unreal; picking one of them would be sufficient.

In the case of HoN and Savage 2, there is currently a very significant difference in the performance and feature set of the graphics renderers in both of these games. To fully capture the flavor of these, it would best benefit Phoronix to measure both games. However, a patch is planned for the near future that will effectively port the graphics engine of HoN over to Savage 2, which would take away the uniqueness of Savage 2's engine.

There is a simple console command you can run to show the FPS playing Savage 2, and you can also script some activity like malphs running around and builders making buildings. I notice that the current Savage 2 engine lags badly on some hardware when certain types of particles are emitted, such as when a building is destroyed.

Maybe an enterprising modder could create a sandbox benchmarking and FPS counting modification for Savage 2, enabling automated testing? I'm sure it could be done; S2 is extremely moddable without modifying the binaries (Jason's post illustrates why).

Anyway just a thought.

[Naib]

Actually, I think that's a misinterpretation of what S2Jason said.

What he said is that the engine is modular, and that they could rip out the graphics engine and write a new one without rewriting most of "K2".

There are two senses of "K2": the K2 Infrastructure, such as you might find in another engine; and the actual implementation of any particular module in a particular game.

Actually it is exactly what S2Jason said

The graphics capabilities are not even a part of the K2 source. The part we refer to as "the renderer" lives in its own little project. A completely new renderer could be written from scratch without changing a line of code anywhere else.