Doom 3 engine release and game code

[revelator]

your SDL library might not have glMapBufferRange in which case you probably need to get the import from opengl itself.

More stuff i probably forgot to mention had a veritable shitstorm on my head these last few days trying to run down some illusive bug ill post back if i find anything.

[revelator]

Fixed dhewm3 source



included needed libraries they are in the external dir (yes it uses SDL-1.2 i just moved the missing pieces to the top of qgl.h)

and yes it works.

included my own build so you can check it out.

[motorsep]

You probably forgot to include a header. Tbh, changes like that should not be simply thrown into dhewm3 because dhewm3 is somewhat different from vanilla codebase.

[revelator]

tbh its not that different only the build method and the fact that it uses SDL is
and as for SDL it only maps the opengl calls SDL itself does not provide any opengl features but works a bit like GLEW when OpenGL is used so if its missing those calls you cannot link and theres really no more to it

The error on my part was lack of sleep so i forgot some changes i made in other places.

[revelator]

Sigh well one shouldnt take all crap a static analyzer throws out for litteral i just had a revelation after fixing some of idlibs math functions by doing array calls as a reference well it compiles fine and the game
loads but rofl all models are now 2D sprites i have skeletons flying all over the place i got attacked by a barrel... and to top it of when looking in a mirror in the game i had the head of an imp

So big warning to devs double check if the fix has the desired result and make a backup of the working version before you go tom cruise on it hehe.

[tobis87]

[revelator]

The alpha bit commit does not work on windows (all screens show static or black) thats why i reverted it nice catch though

As for what it does Mh might be better explaining that but as far as i understand from the code is that it limits the insane ammount of VBO calls doom3 normally has by checking for allready allocated blocks.
Im not quite into what the ARB_map_buffer_range function does for speed but Mh might have an explanaition.

I get a few fps more from this but nothing earthshattering either but as you also noticed the game feels more responsive so theres definatly something to it
doom3's biggest problem today seems to be that it actually runs worse on modern cards (go figure) i get more fps on my old geforce 6600 gfx card than i get with my 2x 560 gtx TI.
I guess optimizing for recent hardware will be a lenghty process and this is not the only thing that could be done to speed it up. Got some code from jmarshall that should speed up things considerable unfortunatly he forgot a few bits and pieces like me and hees been mia for some time but if he resurfaces ill see if we can get it fixed and then ill post a patch.

edit: btw timedemo is absolute garbage in doom3 im afraid on my rig it shows constant 60 fps but when i load a map my fps drops to 35 - 40 so its not really a good indicator

[frag.machine]

[qbism]

[revelator]

[mh]

What the glMapBufferRange stuff does is allow you to take advantage of a VBO streaming pattern that D3D has enjoyed since at least version 7 - in D3D terms it's known as the discard/no-overwrite pattern.

A VBO is a GPU resource, and normally, if you try to update a GPU resource that is currently in use for drawing with (entirely possible because of the asynchronous nature of CPU/GPU operation), everything must stall and wait for drawing to complete before the update can happen. The stock Doom 3 code actually double-buffers it's streaming VBOs to try avoid this (in a slightly obfuscated way) but glMapBufferRange is a more robust way.

So, I mentioned discard/no-overwrite above. Here's what they do.

The buffer is filled in a linear manner. You've got 2mb (or whatever) of space, vertexes are added beginning at position 0, as new vertexes are added they get appended until the buffer fills, then magic happens.

This standard update is no-overwrite; your code makes a promise to GL that it's not going to overwrite any region of the buffer that may be currently in use for drawing, and in return GL will let you update the buffer without blocking. In order to be able to keep this promise your code must maintain a counter indicating how much space in the buffer it has previously used, and add new verts to the buffer at this counter position.

When the buffer becomes full you "discard". This doesn't throw away anything previously added, instead GL will keep the previous block of buffer memory around for as long as is needed to satisfy any pending draw calls, but will give you a new, fresh block for any further updates. That's the "magic" I mentioned above, and it's what lets you use a streaming VBO without any blocking.

This pattern will also let you get rid of Doom 3's double buffering, thus saving you some GPU memory (I haven't yet done this in my code). Because there's no more blocking it will run faster in cases where there is a lot of dynamic buffer usage, but because Doom 3 locks at 60fps it may not be as directly measurable as if the engine was unlocked. Hence the "it feels more responsive but I can't quite put my finger on it" result.

There's another chunk of code in the standard Alloc call which deals with updates of non-streaming VBOs and which is implemented in quite an evil manner by the stock Doom 3 code. When updating such a VBO you can get a faster update if the glBufferData params are the same as was previously used for that VBO (the driver can just reuse the previous block of buffer memory instead of needing to fully reallocate). Doom 3 doesn't do that, so it doesn't get these faster updates, but by searching the free static headers list for a VBO that matches and using that instead of just taking the first one from it, it can. Obviously it sucks that you need to search the list in this way, and a better implementation would just store the VBO with the object that uses it, and reuse the same VBO each time. Since this mainly happens with model animations an ever better implementation would use transform feedback to animate the model instead of animating it on the CPU and needing to re-upload verts each frame, but I haven't even looked at that yet.

So all in all the stock VBO implementation is an unholy mess that needs serious work to get it functioning right, much the same way as Quake 1 lightmap updates were a mess. That code just represents the start of a process, but I personally don't think it's worth continuing with. I say - wait for the BFG edition, wait and see if that's going to get a source release (Carmack seems keen), and use that as a base for further work instead - chances are that all of this stuff will be fixed in that.

[revelator]

Ok that atleast made me understand a few things

Carmack planned on opensourcing the BFG edition code ??? well id like to see that but i guess its the proper way unless the Old doom3 code cannot run it it makes no sense keeping it closed source.
One thing id like to see also is the Quake4 source (mostly because it has a working glsl backend) Prey would also be nice

Edit Btw. if someone has a look at my Doom3 source and spots something out of the order (preferably an ATI user) im very interrested in hearing cause the ATI bug i seem to have introduced is really starting to annoy the heck out of me

[revelator]

Read up some more on the BFG edition and a few things to notice.

It is not compatible with the old Doom3 engine (uses parts of idtech5) so mods etc made for Doom3 will not work.
Carmack might not reveal all of the new code but theres a probability he will reveal some of it.

Hmm a glimpse at idtech5 code sounds rather interresting.

[revelator]

Btw Mh i fixed up the source you send me a while back (Sortof like a cleansource for Doom3).

I converted allmost all the remaining calls to use vertexattribs instead of the old clientstate method (only missing ac->normals atm).
Added some missing source files to the project (AAS).
renamed DOOM3.exe to MHDoom3.exe
Added your VBO optimizations.
Removed a few unnessesary things.

BUG:

Shadows work fine for casted shadows but model shadows look weird as hell (split in half) not something i introduced the unmodified version also does it.



exe size is somewhat smaller now but the above bug is rather annoying to look at

Ill upload it to my realm site if you want it

[revelator]

agh just noticed replacing the texcoord calls with vertex atrribs is not quite working (breaks doom sky) could possibly be a bug in Doom3's code cause thats the only place it breaks.