The Road to Beta7
<i>This post was originally made by <b>skyjake</b> on the dengDevs blog. It was posted under the categories: Beta 7, Engine, Games.</i>
Now that I can run the client and server in separate processes locally, I have drafted a set of actions to fix the multiplayer once and for all. In short, the current engine-side networking code will need to be thrown in the trash and replaced with something sensible. As a general rule, I will be reusing the original singleplayer game logic as much as possible for the client's local world.
<ul>
<li>Discard the current libdeng <tt>cl_*.c</tt> and <tt>sv_*.c</tt>. Not much there to write home about. Particular offenders are <tt>cl_player.c</tt> and <tt>cl_mobj.c</tt>, which never really worked that great and duplicate gameside logic (insufficiently).
<li>Also discard the current low-level network communications code that sends and receives packets over TCP/UDP.
<li>Start from the basics. Using new libdeng2 classes, set up the data link between the server and client. I'm planning on using TCP streams only, initiatiated upon the client opening a TCP connection via the port the server is listening on.
<li>Redo the communications protocol and the way network packets are created and (de)serialized.
<li>The principle of how things work is that the game world exists on the server, and the clients mirror copies of it. The clients are using local game logic for moving the players with zero latency. Impulses such as attacking and opening doors are signaled to the server, where they get performed in the real game world. This causes changes to occur in the map data, all of which can be recorded as a set of DMU operations. These operations are prioritized based on distance to clients and sent over the data links to them. Mobjs need to be tracked separately (unless we do a DMU-like interface for them as well -- which might not be a bad idea). On clientside, the DMU operations are applied to the client's copy of the world. On clientside, mobjs are there only for appearances and basic collision testing. On clientside, the only mobj controlled by the game logic is the player. On serverside, mobjs are controlled by the game logic as usual. Other events such as sounds, chat messages, state changes (e.g., inventory updates) are handled with gameside messages.
<ul> <li>That's a whole bunch of stuff to do, but I'll begin with the player movement, move to DMU map updates and then proceed to handling the mobjs. </ul>
<li>Savegames are done on the server, as it has all the "official" information about the game. I'm considering that the savegame data could be copied to clientside as well, so that the game could be resumed if any of the involved players start hosting a game. (Since the savegame system is identical for local singleplayer games and networked games, this should do nicely.) At least the client's 3D view at the time of the save will need to be included in the clientside copy of the savegame. As to the format, it will be a ZIP directory containing a .jpg for the screen capture, and a separate file for each binary array of data, e.g., a file called "sectors" for the sector data, etc.
<li>Finally, demo recording will work as before by simulating the incoming data stream from a server, accompanied with a record of the local clientside camera movements. The demo file will be a ZIP directory with some metadata files included in addition to the serialized packet stream. For instance, a .jpg could be included in the demo file to act as an icon in the GUI.
</ul>
It's a sizeable amount of work, but what else would I do on my vacation?
And it should take care of our biggest issues while removing the walls between singleplayer and multiplayer games.
When all of those things are done, and the client is adjusted to launch local games on the local dengsv, we have ourselves a 1.9.0-beta7 release.
Now that I can run the client and server in separate processes locally, I have drafted a set of actions to fix the multiplayer once and for all. In short, the current engine-side networking code will need to be thrown in the trash and replaced with something sensible. As a general rule, I will be reusing the original singleplayer game logic as much as possible for the client's local world.
<ul>
<li>Discard the current libdeng <tt>cl_*.c</tt> and <tt>sv_*.c</tt>. Not much there to write home about. Particular offenders are <tt>cl_player.c</tt> and <tt>cl_mobj.c</tt>, which never really worked that great and duplicate gameside logic (insufficiently).
<li>Also discard the current low-level network communications code that sends and receives packets over TCP/UDP.
<li>Start from the basics. Using new libdeng2 classes, set up the data link between the server and client. I'm planning on using TCP streams only, initiatiated upon the client opening a TCP connection via the port the server is listening on.
<li>Redo the communications protocol and the way network packets are created and (de)serialized.
<li>The principle of how things work is that the game world exists on the server, and the clients mirror copies of it. The clients are using local game logic for moving the players with zero latency. Impulses such as attacking and opening doors are signaled to the server, where they get performed in the real game world. This causes changes to occur in the map data, all of which can be recorded as a set of DMU operations. These operations are prioritized based on distance to clients and sent over the data links to them. Mobjs need to be tracked separately (unless we do a DMU-like interface for them as well -- which might not be a bad idea). On clientside, the DMU operations are applied to the client's copy of the world. On clientside, mobjs are there only for appearances and basic collision testing. On clientside, the only mobj controlled by the game logic is the player. On serverside, mobjs are controlled by the game logic as usual. Other events such as sounds, chat messages, state changes (e.g., inventory updates) are handled with gameside messages.
<ul> <li>That's a whole bunch of stuff to do, but I'll begin with the player movement, move to DMU map updates and then proceed to handling the mobjs. </ul>
<li>Savegames are done on the server, as it has all the "official" information about the game. I'm considering that the savegame data could be copied to clientside as well, so that the game could be resumed if any of the involved players start hosting a game. (Since the savegame system is identical for local singleplayer games and networked games, this should do nicely.) At least the client's 3D view at the time of the save will need to be included in the clientside copy of the savegame. As to the format, it will be a ZIP directory containing a .jpg for the screen capture, and a separate file for each binary array of data, e.g., a file called "sectors" for the sector data, etc.
<li>Finally, demo recording will work as before by simulating the incoming data stream from a server, accompanied with a record of the local clientside camera movements. The demo file will be a ZIP directory with some metadata files included in addition to the serialized packet stream. For instance, a .jpg could be included in the demo file to act as an icon in the GUI.
</ul>
It's a sizeable amount of work, but what else would I do on my vacation?
And it should take care of our biggest issues while removing the walls between singleplayer and multiplayer games.When all of those things are done, and the client is adjusted to launch local games on the local dengsv, we have ourselves a 1.9.0-beta7 release.
Comments
<ul>
<li>Make sure that dengcl is running on Windows and Linux with the new plugin system, within the "install" directory.
<li>Get rid of the old GetGameAPI mechanism. The functions exported from the game will be regular exported functions, named "deng_Something". Any numerical values in game_export_t can be queried with <code>deng_GetInteger()</code>. This will be consistent with the plugin interface, named <code>deng_InitializePlugin()</code> and <code>deng_ShutdownPlugin()</code>.
</ul>
When those are done, I'll merge the branch back to the trunk and start work on phase3, where I'll rip out the old network code and start with the new libdeng2-based stuff. I expect phase3 will end with the player being able to move in the map, but with no actual updates coming from the server yet -- the map will just remain static, no mobjs will be visible anywhere, and no world sounds will play.
<b>EDIT:</b> On second thought, I'll do the merge now and begin phase3 by removing GetGameAPI. Many of the functions will disappear anyway, so no point in taking time to refactor the existing netcode to use <code>de::Library</code> exports if it's just getting deleted soon.
Phase 4 is all about getting the world state transferred from the server to the clients. This has two parts: 1) the initial state, and 2) the updates. The initial state will most likely be a zipped copy of the internal map data. The updates will come via DMU calls.
The trouble with doing things The Right Way and not cutting corners is that the effects may be quite far-reaching withing the codebase. I expect that the internal map data (including objects) will be migrated in at least some form to libdeng2 during this phase. I will have to find a way not to break all the existing code in libdeng that accesses the internal map data.
At the end of phase 4 the world state, objects included, should be transferred to the clients and updated in real time.