Approaching Beta 4
<i>This post was originally made by <b>skyjake</b> on the dengDevs blog. It was posted under the categories: Beta 4, Engine, Games, jDoom, jHeretic, jHexen, Mac OS X, Releases, Unix/Linux, Version 1.9, Windows.</i>
Let's list here the things that still must be fixed before Beta4 can be released.
<h3>All Platforms</h3>
<ul>
<li>Update release notes/change log.</li>
<li>FIXED (rev 3167) jDoom: The map name in the automap is still not visible sometimes. Its placement should be fixed.</li>
<li>FIXED (rev 3168) jDoom: Floorclipped mobjs
The floorclipping ported from jHeretic/jHexen doesn't work correctly yet.</li>
<li>FIXED (rev 3163): Savegame issues? Is jDoom working correctly in this regard? jHeretic and jHexen now appear to.</li>
<li>Map loading:
<ul>
<li>Currently if gl node buidling is disabled Doomsday segfaults during the load process.</li>
<li>DoomII MAP30/Slige maps bug: Currently Doomsday will segfault when warping to MAP30 when it has been loaded once already. I think somehow there is a mixup between whether the map lumps are loaded via the primary or auxilary lump cache. The first time the map is loaded the lumps are in the primary while subsequent attempts look to the auxliary cache.</li>
</ul>
</li>
<li>XG: Still some noticeable problems here due to the existing DMU implementation being incomplete. Noteably dummy object handling - Doomsday isn't checking if an object is a dummy before calling P_Callback(p).</li>
<li>FIXED (rev 3082): "MISSING" texture binding not working correctly.</li>
<li>Current light range is reset when when changing sectors.</li>
<li>R_UpdateSector - Need to do something about the glow property update?</li>
<li>jDoom: Line attack weapons are hitting planes with the skyflat and spawning PUFFs. Update; I've looked in to this and this bug only happens within a very particular set of circumstances and is VERY rare. I've not yet found the cause.</li>
</ul>
<h3>Windows</h3>
I have to check that the release scripts are up to date. Shouldn't be too difficult.
<ul>
<li>Any serious Windows-only bugs?</li>
</ul>
<h3>Mac</h3>
The build scripts have been updated so that I can create a working disk image of the release build containing the "Doomsday Engine" application and a readme file.
<h3>Linux/Unix</h3>
<ul>
<li>Anything?</li>
</ul>
I think we should try to get Beta4 released as soon as possible so we can move on to working on the next set of larger pre-1.9.0 changes (the engine/game savegame split being among the most important of those).
<h3>Postpone to Beta5</h3>
<ul>
<li>jDoom: Investigate Doom2 map16 rendering bug (as reported by KuriKai below).</li>
<li>rend-bias is so incomplete that fixing it completely would take too long. I suggest we postpone further work on the bias lighting system until the next release, in which we can try to finish it. BIAS lighting glitches:
<ul>
<li>lighting is not consistent between onesided seg middles and twosided seg uppers+lowers.</li>
<li>surface colors are not intergrated with BIAS, twosided seg middles are currently hidden to remind us of this</li>
<li>Lightning in jHexen causes massive slowdown due to having to change the light levels of all outdoor sectors (lightgrid update). It might be possible to implement this using a value shift in the lightRangeModMatrix depending on if the sector has a sky flat instead of actually changing the sector's light_level (would that be OK or do we need to do it properly?)</li>
<li>bias light sources don't affects mobjs yet. When drawing a mobj, the bias light level at the location should be evaluated to get a basic ambient light level.</li>
<li>automatic conversion of huge dynamic lights to bias sources needs to be revised. Currently any sources acquired by mobjs are not released back into the pool when the mobj is destroyed while the current sprite frame has a light.</li>
</ul>
</li>
</ul>
Let's list here the things that still must be fixed before Beta4 can be released.
<h3>All Platforms</h3>
<ul>
<li>Update release notes/change log.</li>
<li>FIXED (rev 3167) jDoom: The map name in the automap is still not visible sometimes. Its placement should be fixed.</li>
<li>FIXED (rev 3168) jDoom: Floorclipped mobjs
The floorclipping ported from jHeretic/jHexen doesn't work correctly yet.</li>
<li>FIXED (rev 3163): Savegame issues? Is jDoom working correctly in this regard? jHeretic and jHexen now appear to.</li>
<li>Map loading:
<ul>
<li>Currently if gl node buidling is disabled Doomsday segfaults during the load process.</li>
<li>DoomII MAP30/Slige maps bug: Currently Doomsday will segfault when warping to MAP30 when it has been loaded once already. I think somehow there is a mixup between whether the map lumps are loaded via the primary or auxilary lump cache. The first time the map is loaded the lumps are in the primary while subsequent attempts look to the auxliary cache.</li>
</ul>
</li>
<li>XG: Still some noticeable problems here due to the existing DMU implementation being incomplete. Noteably dummy object handling - Doomsday isn't checking if an object is a dummy before calling P_Callback(p).</li>
<li>FIXED (rev 3082): "MISSING" texture binding not working correctly.</li>
<li>Current light range is reset when when changing sectors.</li>
<li>R_UpdateSector - Need to do something about the glow property update?</li>
<li>jDoom: Line attack weapons are hitting planes with the skyflat and spawning PUFFs. Update; I've looked in to this and this bug only happens within a very particular set of circumstances and is VERY rare. I've not yet found the cause.</li>
</ul>
<h3>Windows</h3>
I have to check that the release scripts are up to date. Shouldn't be too difficult.
<ul>
<li>Any serious Windows-only bugs?</li>
</ul>
<h3>Mac</h3>
The build scripts have been updated so that I can create a working disk image of the release build containing the "Doomsday Engine" application and a readme file.
<h3>Linux/Unix</h3>
<ul>
<li>Anything?</li>
</ul>
I think we should try to get Beta4 released as soon as possible so we can move on to working on the next set of larger pre-1.9.0 changes (the engine/game savegame split being among the most important of those).
<h3>Postpone to Beta5</h3>
<ul>
<li>jDoom: Investigate Doom2 map16 rendering bug (as reported by KuriKai below).</li>
<li>rend-bias is so incomplete that fixing it completely would take too long. I suggest we postpone further work on the bias lighting system until the next release, in which we can try to finish it. BIAS lighting glitches:
<ul>
<li>lighting is not consistent between onesided seg middles and twosided seg uppers+lowers.</li>
<li>surface colors are not intergrated with BIAS, twosided seg middles are currently hidden to remind us of this</li>
<li>Lightning in jHexen causes massive slowdown due to having to change the light levels of all outdoor sectors (lightgrid update). It might be possible to implement this using a value shift in the lightRangeModMatrix depending on if the sector has a sky flat instead of actually changing the sector's light_level (would that be OK or do we need to do it properly?)</li>
<li>bias light sources don't affects mobjs yet. When drawing a mobj, the bias light level at the location should be evaluated to get a basic ambient light level.</li>
<li>automatic conversion of huge dynamic lights to bias sources needs to be revised. Currently any sources acquired by mobjs are not released back into the pool when the mobj is destroyed while the current sprite frame has a light.</li>
</ul>
</li>
</ul>
Comments
FIXED: All known issues/bugs relating to loading 1.8.6 jDoom and jHeretic save games with Beta4.
The current system is always going to be pretty inefficient when there are lots of sector light changes simultaneously. We could prepare a secondary matrix of "sky lights", that can be multiplied by a factor and added to the actual bias light grid when evaluating the light level.
This requires modifying the Hexen lightning a bit so that instead of adjusting sector light levels directly, the game tells the engine how strong a sky light should be applied. The sky light grid only needs to be updated when a plane's texture is set to skyflatnum, or a plane's texture is changed away from skyflatnum.
Also, the current method of updating the lightgrid is very inefficent when updating the lighting of very large sectors. Currently each block for the sector being updated is set upto 25 times each due to spreading.
It seems to me to that we can severly reduce the amount of work that needs to be carried out by collecting blocks into two sets per sector. The first set would be a collection of "clusters". A cluster would contain blocks that are entirely within the sector and do not contribute light to blocks "owned" by other sectors (their contribution factor will always be 1). The second set of blocks will be updated as per the current implementation.
Example of how the blocks would be collected: <a href="http://www.daniel.ddsdesign.co.uk/Stuff/gridblocks.jpg">gridblocks.jpg</a>
<blockquote>We could prepare a secondary matrix of "sky lights", that can be multiplied by a factor and added to the actual bias light grid when evaluating the light level.</blockquote>
That sounds like a good idea. It would work correctly without change to the lightrangemodmatrix too.
<a href="http://jfiles.org/scult/doom2.90.jpg" rel="nofollow"></a>
<code>http://jfiles.org/scult/doom2.90.jpg</code>
I do note however that gcc 4.0 & 4.1 are giving a lot more warnings on the code then older compilers, so after the beta4 release, it will be a good idea to do a code clean up, considering they are becomming the default in many distros.
I'm not happy with the configure scripts, they will work for the release, but I've yet to get arround to making them automatically generate the headers from makedmt.py and mapdata.hs - I plan to give them an overhaul after the archive re-arangement. When time alows I will try to set it up to allow deng to be build under MinGW for Windows so it will be possible to build with either VisualC or MinGW.
As far as I can tell it works fine on mine and my testers nvidia boxes, I'm waiting on some spare parts from ebay to test on my ati (gpl drivers) system. Muliplayer hasn't been tested either due to waiting for spare parts. Constant reports that hexen is slow from users, but it becomes playable after I advise setting the model to sprite distance to 750.
Thanks. It is a known issue.
The emulation of some of the DOOM.EXE rendering hacks have been intentionally removed for now as they were causing other issues in various PWADs. Some of them were even preventing some maps from loading at all and lead to Doomsday crashing.
I have some good ideas on how to support 99% of the DOOM renderer hacks but the code hasn't been implemented yet. So for now, anything that was causing other problems has either been disabled or removed.
There is but regardless of which setting its at this should not happen. I'm missing something simple like a single "!" character somewhere... I'll go through it and recompare the existing logic with that of 1.8.6
I've found a serious bug with the light grid with regard to DOOM.EXE hacks. Here is a map for debug (DOOMII MAP01) - <a rel="nofollow" href="http://www.daniel.ddsdesign.co.uk/Stuff/ictest.wad">ictest.wad</a>
Walk towards the red pillar and you'll hit an invisible wall (which SHOULD have a mid texture on it btw) and press use to open the midgrate door.
The crash appears to be happening in LG_SectorChanged(), some debug code reveals:
<blockquote>info->blockcount = 65535
info->changedblockcount = 0</blockquote>
That sounds wrong to me. Something is going wrong during the lightgrid init. Now the sector in question is unclosed so that might be throwing things completely off...
<blockquote>There is but regardless of which setting its at this should not happen. I'm missing something simple like a single "!" character somewhere... I'll go through it and recompare the existing logic with that of 1.8.6</blockquote>
Fixed in SVN 3168.
It looks like the changes I made to Rend_RenderWallSeg() in SVN 3082 fixed this problem as I can no longer reproduce it. The problem was due to the way I was manipulating the surface color ptrs for the wall section. I just never got around to testing the fix.
In SVN 3173:
FIXED: Mobjs flagged with MF2_FLOORCLIP were clipping into the floor if the floor texture was the "MISSING" texture (flatlumpnum = -1). Always treat "MISSING" surfaces as FLOOR_SOLID.
The first job is to (re)initialize the properties of the dummies at the start of each level. Then we need to make sure that calls to eg R_UpdateSector() are never made with reference to a dummy object.
We also need to be careful of not dereferencing object->object ptrs on dummies (for example DMU_LINE_OF_SECTOR on a dummy sector will nodoubt lead to a seg violation somewhere). Not all of cases would be invalid but most will be. However this might prove tricky...
Currently it is perfectly possible via DMU to request a dummy sector and then P_SetPtrp(realline, DMU_FRONT_SECTOR, dummysector). We should prevent assigning ptrs to dummies to "real" map object properties completely by catching the attempt with P_IsDummy and present a proper debug message to the game developer.
Currently, shadowlinks are allocated linearly in R_LinkShadow() which is more than likely the cause of the incredibly long load time I'm experiencing on large levels (on the map I'm using for profiling there are 19877 shadowpolys each requiring shadowlink(s), so there is at least that number of individual allocations of a small number of bytes, taking upwards of 160 seconds(!) by the time all have been allocated).
I doubt there is a direct correlation between the number of subsectors/shadowpolys and the number of shadowlinks required(?). So I guess we can't allocate all we need in one go so it would be best to allocate in batches of a reasonable number using a pool.
Given that as of Beta4 - Doomsday is capable of playing maps up to twice the physical size it was limited to in previous releases, it would be nice to implement a revised shadowlink allocation mechanism for this release.
It shouldn't take long to implement, we just need to pick an appropriate method.
Oh, I wasn't suggesting you do it all yourself
The shadow link allocation I could take care of. Do you have a preference for a particular method to use for this? It would be good to have a generic block allocation mechanism that can be used for all similar tasks. The subsector block map initialization and the block map creation would benefit from this too.
This would go a long way to reducing fragmentation when not using memory allocated from the zone.
Can you mail me your large test map, btw? That would be helpful.
Can you mail me your large test map, btw? That would be helpful.</blockquote>
Email sent.
It's not the largest of test maps but at 11,000+ linedefs and 1,500+ sectors it's plenty big enough for profiling.
EDIT: I've sent it to your doomsdayhq.com address as it was too large for your sourceforge address.
EDIT2: Hmm, that address bounced back too. I'll try your mac.com address... If that fails I send it to your dengine.net address since I know that one is working.
Beat DooMII and while its going through the charachters of the game press escape and use the arrow keys to change options in the menu. You should notice that pressing a key will kill the monster (even though the menu(not doomsdays control panel) is up)
For example when calling P_Callbackp with a dummy sector and eg DMU_SECTOR, R_UpdateSector is called on the dummy. Now because it is a dummy it doesn't have a valid subsector or line list so that is where the segmentation violation occurs.
Another problem area is that dummy sectors don't have the plane back ptrs set, so any attempt to get the sector of a plane will cause seg vio (my bad there I should have plugged that when I added planes as objects via DMU).
Basically its any instance where there is object->object reference in the DMU code.
Line Type {
ID = 10001;
Comment = "Destroy ball when it crosses the line";
Flags = any_cross | activator_type
Flags2 = when_act | any
Class = damage
Thing type = "PAIN" # Replace with ID of the ball
Type = timed_off
Count = -1
Time = 1
Act sound = "swtchn" # Replace with the sound to play when a goal is scored
p0 = 10000
}
Line Type {
ID = 10000;
Comment = "Spawn a new ball when there is none";
Flags = mobj_gone | ticker
Flags2 = when_act | any
Class = command
Type = timed_off
Count = -1
Time = 1
Thing type = "PAIN"; # Replace with ID of the ball
Ticker tics = 5;
Texmove angle = 90;
Sp0 = "spawnmobj PAIN 0 0 0" # Replace with ID of the ball and the spawn location
}
using the "-def" command doomsday crashes with
"Fatal signal: Segmentation Fault (SDL Parachute Deployed)"
"Fatal signal: Segmentation Fault (SDL Parachute Deployed)"</blockquote>
The problem it would appear is that somewhere along the line you've converted that definition to unicode.
The problem here is the quotation characters " & " are the unicode varients and not ASCII which is ". DED definitions need to be in ASCII. See here <a rel="nofollow" href="http://qcpages.qc.cuny.edu/~nixon/links/asciiUnicode.html">encoding differences</a>.
Also note you have a typo "p0 = 10000" should be "Sp0 = 10000".
The problem here is the quotation characters `"` & `"` are the unicode varients and not ASCII. DED definitions need to be in ASCII</blockquote>Dani, it might be time to support unicode then. All modern *NIX system have converted over to unicode by default (some editors no longer have an option for ascii), as it seems quite likely KuriKai has edited that .ded on an *NIX system.
The whole DED reader is due a rewrite anyway. Perhaps sometime in the 1.9.x series we can look at replacing it completely with a "proper" parser with syntax mnemonics and checking, token lists, unicode support and other fanciful things like in-line equations and token declarations.
Perhaps it would be a good idea to use something like Bison for parsing DED files?