Save game format changes
<i>This post was originally made by <b>danij</b> on the dengDevs blog. It was posted under the categories: Engine, Games, Version 1.9.</i>
Now that map data is managed mostly by the engine we should be begining to think about moving the save game code (at least partially) over to the engine. By doing so we'll be able to save the state of various types of data that previously we've been unable to do.
Given the way the game object data is split between the engine and the game we need to think carefully about how such a system should work.
One potential problem is that there is no easy split in the existing map object data properties for each element between engine and game side properties (eg there is no cutoff between engine side sector_t properties and game side xsector_t properties). So it may be that we'll have to have some game-side code to handle reading old saves.
In preperation for this in jDoom and jHeretic; I have removed the dependencies on compile time issues regarding struct member packing to provide byte offsets and also streamlined the way thinkers are handled. Not to mention numerous other changes.
Due to the amount of changes I would appreciate feedback on any issues with the save game functionality in jDoom and jHeretic.
Now that map data is managed mostly by the engine we should be begining to think about moving the save game code (at least partially) over to the engine. By doing so we'll be able to save the state of various types of data that previously we've been unable to do.
Given the way the game object data is split between the engine and the game we need to think carefully about how such a system should work.
One potential problem is that there is no easy split in the existing map object data properties for each element between engine and game side properties (eg there is no cutoff between engine side sector_t properties and game side xsector_t properties). So it may be that we'll have to have some game-side code to handle reading old saves.
In preperation for this in jDoom and jHeretic; I have removed the dependencies on compile time issues regarding struct member packing to provide byte offsets and also streamlined the way thinkers are handled. Not to mention numerous other changes.
Due to the amount of changes I would appreciate feedback on any issues with the save game functionality in jDoom and jHeretic.
Comments
It would appear there is nothing wrong with the read/write routines as additional debug code reveals the read data is exactly the same as the written. So something else is going wrong with them. Now where does the spawning happen...
Edit: Update 2:
The issue seems to be limited to thinkers spawned via EV_BuildStairs and EV_DoDonut only. The search continues...
Edit: Update 3:
Fixed in SVN rev 3158. The problem was due to the speed properties of thinkers being expanded/contracted with FRACBITS during read/write respectively. This resulted in small speeds being reduced to 0 which meant that although the thinkers were being created, T_MovePlane never moved the plane(s).
After some initial investigation it appears there is something is causing the read process to go out of sync long before it gets to the sectors.
I added some debug code to dump the texarchive'd names to console and I'm getting a real mess. It seems to me that something is aliasing during the read process either something being read when it shouldn't or vice-versa (due to not being present in an earlier version).
Here is a small sample of the output of the dumped names:
<blockquote>0 "5_5"
1 "5_1"
2 "1"
3 "5_2"
4 "5_4"
5 "E14"
6 "K09"
7 "1_2"
8 "S1"
9 "Y1"
10 "3"
11 "OP2"
12 "3"
13 "R7_1NUKA"
14 "GE3"
15 "E15"
16 "GE2"
17 "23"
18 "ORBLU"
19 "SHITTYBI"
20 "GDOOR2ME"
21 "TAL2"
22 "AY4"
23 "EP5"
24 "PPORT2DO"</blockquote>
SV_LoadGame: Bad savegame (consistency test failed!)
Will test with 3158 shortly.
SV_LoadGame: Bad savegame (consistency test failed!)
Will test with 3158 shortly.</blockquote>
<blockquote>No change on 3158.</blockquote>
That will not be fixed.
I have no desire to make sure save games work between every single SVN revision and to a large extent, even previous beta versions. I only intend compatibility between previous offical full version releases (and 1.9.0-beta3 due to it being so widely used and having been available for so long).
However if you mean there is a problem loading a jHeretic save made in 3158 with 3158 then obviously it is a bug and I'll look into it.
<blockquote>Actually, thinking ahead now to a 64bit deng, it would be nice to ensure savegames are compatable between big/little endian and 32/64 bit machines.</blockquote>
Yes that would be nice. Given the complete lack of endianness handling stuff in Common/p_saveg.c I'm assuming that the LZSS lib takes care of that side of things though I don't know whether the save files are little or big endian.
I'll assume that's what you meant as I've just tested the above case and had the consistency failure report.
I've found the cause. I'll upload the fix to SVN in a minute.
Considering how long beta3 has been out, we should have called it 1.9.0
FIXED: Consistency report issue in __JHERETIC__
CHANGED: Removed MAX_ARCHIVED_THINGS fixed limit (speaks for itself).
The page up and page down buttons in the console take it to the top and bottom of the console (so you can see the text that has dissapeard higher than the screen)
Not a bug. jDoom now has flying implemented as a cheat "give f" (as in give me the power of flight) accessible via the console. I don't know why I didn't just use "fly" instead, I'll change this.
<blockquote>The page up and page down buttons in the console take it to the top and bottom of the console (so you can see the text that has dissapeard higher than the screen)</blockquote>
Not a bug. I've recently added a bit more functionality to the console and some of the shortcuts had to be changed.
Here are the currently available controls in the console:
<ul>
<li>PgUp/PgDn:
Scroll history window to the start/end of history buffer respectively.</li>
<li>PgUp/PgDn + Shift:
Scroll history window up/down two lines in the buffer respectively.</li>
<li>Insert:
Toggle "insert mode" on the cmd line.</li>
<li>Backspace:
Delete the character behind the cursor.</li>
<li>Delete:
Delete the character under the cursor.</li>
<li>Home/End:
Move console window up/down one line respectively.</li>
<li>Home/End + Shift:
Move console window up/down three lines respectively.</li>
<li>Tab:
Use the default autocomplete mode.</li>
<li>Tab + Shift:
Use the console autocomplete mode NOT set as the default completion mode.</li>
<li>Left/Right Arrow:
Move the cursor on the input line left/right one character respectively.</li>
<li>Left/Right Arrow + Shift:
Move the cursor on the input line to the start/end of the input line respectively.</li>
<li>Right Arrow (when cursor is at the end of the input line AND there is previous commands in the history buffer AND there are fewer characters on the input line than in that of the command in the current history point):
Copy one character from the command in the current history position to the end of the input line. Works like a DOS terminal.</li>
<li>Enter:
Execute the command(s) on the current input line.</li>
<li>F5
Clear the console history buffer.</li>
<li>Alt + "C":
Clear the current input line.</li>
<li>Tilde:
Open/Close the console.</li>
<li>Tilde + Shift:
Switch between half and full screen mode.</li>
</ul>
<p>OK. It is possible to reporduce KuriKai's bug under the following circumstances. get into a fight with a cyberdemon (level32 on doom2 is good for this). Launch a bfg shot at the cybedemon after it had lanched a few rockets at you. Now save the game before any of the projectiles impact. Load the saved game, and watch deng crash.</p>
</blockquote>
<p>Thanks. I'll investigate this.</p>
<p>I would imagine that this bug has been around a while though, not just in 1.9.0</p>
Maybe we should swap all controls for moving the console window with the history window?
It's taken me a while but I can now reproduce this crash reliably. I'll fix the problem.
Thanks for the bug report KuriKai.
Update:
OK, I've found the problem. I didn't update the way the thing_archive was accessed when making it dynamically allocated so each request for a new thing archive number was returning zero. When spawning the BFG ball after loading a save game this resulted in a null mobj->target pointer which leads to the crash when dereferenced in P_AimLineAttack.
While debugging this I think I've found another error to do with the thing archive ids so I'm trying to fix that one too.
I should have a fix commited to SVN shortly.
FIXED: Bugs introduced during removal of the hardcoded MAX_ARCHIVED_THINGS fixed limit. KuriKai's BFG spray crash.
Ok, found the problem. Should have a suitable fix implemented soon. It's due to NUMPOWERS being one larger in SVN 315? than it was in earlier releases.
This highlights an issue with the way various stuff like this is saved. I'll implement an improved mechanism so that new powers, keys etc don't interfere with the save format.
Update:
In SVN rev 3162:
Fixed: Loading of 1.8.6 save games was completely broken due to global counter changes the save game format is dependant upon. Removed these dependencies for players by implementing a playerheader which writes the size of the dependant counters.
TODO: Its still not possible to load 1.8.6 save games in SVN rev 3162. Currently I'm getting a segfault at somepoint in the read. This will be fixed shortly.
FIXED: All known issues/bugs relating to loading 1.8.6 jDoom and jHeretic save games with Beta4.
What was the problem btw? Those routines arn't accessed outside of p_saveg.c so I made them static.
From the build log
<blockquote>../../../Src/jDoom/../Common/p_saveg.c:1150: error: static declaration of 'P_ArchivePlayers' follows non-static declaration
../../../Include/Common/p_saveg.h:56: error: previous declaration of 'P_ArchivePlayers' was here
../../../Src/jDoom/../Common/p_saveg.c:1163: error: static declaration of 'P_UnArchivePlayers' follows non-static declaration
../../../Include/Common/p_saveg.h:57: error: previous declaration of 'P_UnArchivePlayers' was here
../../../Src/jDoom/../Common/p_saveg.c:1217: error: static declaration of 'P_ArchiveWorld' follows non-static declaration
../../../Include/Common/p_saveg.h:58: error: previous declaration of 'P_ArchiveWorld' was here
../../../Src/jDoom/../Common/p_saveg.c:1231: error: static declaration of 'P_UnArchiveWorld' follows non-static declaration
../../../Include/Common/p_saveg.h:59: error: previous declaration of 'P_UnArchiveWorld' was here
../../../Src/jDoom/../Common/p_saveg.c:1847: error: static declaration of 'P_ArchiveThinkers' follows non-static declaration
../../../Include/Common/p_saveg.h:60: error: previous declaration of 'P_ArchiveThinkers' was here
../../../Src/jDoom/../Common/p_saveg.c:1941: error: static declaration of 'P_UnArchiveThinkers' follows non-static declaration
../../../Include/Common/p_saveg.h:61: error: previous declaration of 'P_UnArchiveThinkers' was here
make[3]: *** [p_saveg.lo] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2</blockquote>
Doomsday.out is basically full of
<blockquote>SV_GetArchiveThing: Invalid NUM 0??
SV_GetArchiveThing: Invalid NUM 0??
SV_GetArchiveThing: Invalid NUM 0??
SV_GetArchiveThing: Invalid NUM 0??
SV_GetArchiveThing: Invalid NUM 0??
SV_GetArchiveThing: Invalid NUM 0??
SV_GetArchiveThing: Invalid NUM 0??
Fatal signal: Segmentation Fault (SDL Parachute Deployed)</blockquote> That was surprisngly easy to repoduce.
He also has reported slow-downs in heretic, with very noticible frame rate drops in heretic, the longer it is played. It should be noticable in eg e1m4 after about 10 minutes. I'm having a bit of trouble reproducing this as I've been testing on a higher spec system. Can anyone else repoduce this ?
I could not reproduce that slowdown one.
Could not get jHeretic to crash, though, or show any significant slowdown. Perhaps those would've occured if I had played longer.