On Week 3/2012
skyjake:
For me the 3rd week of the Candidate phase was busy with network and documentation related work.
I started out by having the server send its thing/state IDs to the client right after connection handshake, so that the client can verify that the indices sent in subsequent mobj deltas refer to the correct definitions. The client can also verify if it has the same definitions available as on server-side.
This change increased the size of the handshake considerably so I decided to spend some time improving the network protocol. For many years we have been using a protocol that adds a 2-byte header that contains the size of the network message, followed by the message contents encoded with Huffman codes. However, due to the nature of the coding, and because our code table is hard-coded, this is mostly useful for bit reduction of zeros in the message. Doomsday uses zlib for reading the contents of zip files so I thought, why not take advantage of it for better compression?
The network protocol now compresses long messages using zlib, while short messages (less than 128 bytes) are Huffman-coded as before (since zlib's deflate algorithm doesn't really compress those). The message header's size is also variable, so that when multiple short messages are sent in rapid succession, header overhead is reduced. The end result is improved efficiency for network messages.
This means that network compatibility is broken with earlier builds. Old clients will not be allowed to join servers running build 388 or later.
With these changes complete I should be able to start the public testing servers again. Improvements to the multiplayer code will continue over time; the ultimate goal is to get it running identically to single-player games so that we can start using the same client/server split for all games, not just multiplayer games (see Unified networking).
The other thing I was working on was documentation. I split the old readme Amethyst source into more manageable parts and added the Unix manual page and Windows readme.txt in the respective distribution packages. However, since I want to use RTF for the Mac readme, I still need to spend some time improving Amethyst's RTF output so that tables are laid out correctly.
My plan is to continue updating the readme documentation (some outdated sections remain) and work on the to-do issues.
danij:
Last week I was primarily focused on resolving the outstanding resource loading regressions.
The main change was the implementation of a percent-encoded representation for resource paths (lump names, texture and material paths and more).
Doom's Wad format allows characters in lump names not generally allowed for file and/or directory names on the local file system, including those typically used as path delimiters. As all resource paths throughout the engine are now represented as URIs in a virtual directory hierarchy, it was necessary to normalize these with a unified path representation.
To this end a percent-encoded representation was implemented which rewrites the virtual paths for all lumps loaded from Wad archives. As the lump names are interpreted into symbolic texture (and thus material) identifiers, those too are now percent-encoded.
This percent encoding is mostly hidden from a user's point of view and the engine will normally decode paths before they are presented to the user.
Hitherto, these reserved characters in lump names prevented high resolution replacement resources being used on lumps which use them. One prominent example being Doom's Archvile, which uses the backslash character for an entire frame of it's resurrection animation. The new percent-encoded representation means that such resources can now use high resolution replacements by applying this encoding to the resource file name. For example, to replace the sprite frame VILE\1 the high resolution resource should be named VILE%5C1 (.png/tga/pcx).
I also fixed a couple of bugs in the resource locator which resulted in the incorrect use of resources in certain game modes (for example, Doom's titlepic being used when playing Doom II).
In addition to the work on resource system, I also spent some time tweaking the renderer and updating a few default effects. Users updating to today's build will need to delete their game configs in order to see some of these changes (binding configs are unaffected).
My plan for this week is to polish up the games' user interfaces (primarily the game menu) and document the changes and new features now available since the merger of the ringzero branch.
For me the 3rd week of the Candidate phase was busy with network and documentation related work.
I started out by having the server send its thing/state IDs to the client right after connection handshake, so that the client can verify that the indices sent in subsequent mobj deltas refer to the correct definitions. The client can also verify if it has the same definitions available as on server-side.
This change increased the size of the handshake considerably so I decided to spend some time improving the network protocol. For many years we have been using a protocol that adds a 2-byte header that contains the size of the network message, followed by the message contents encoded with Huffman codes. However, due to the nature of the coding, and because our code table is hard-coded, this is mostly useful for bit reduction of zeros in the message. Doomsday uses zlib for reading the contents of zip files so I thought, why not take advantage of it for better compression?
The network protocol now compresses long messages using zlib, while short messages (less than 128 bytes) are Huffman-coded as before (since zlib's deflate algorithm doesn't really compress those). The message header's size is also variable, so that when multiple short messages are sent in rapid succession, header overhead is reduced. The end result is improved efficiency for network messages.
This means that network compatibility is broken with earlier builds. Old clients will not be allowed to join servers running build 388 or later.
With these changes complete I should be able to start the public testing servers again. Improvements to the multiplayer code will continue over time; the ultimate goal is to get it running identically to single-player games so that we can start using the same client/server split for all games, not just multiplayer games (see Unified networking).
The other thing I was working on was documentation. I split the old readme Amethyst source into more manageable parts and added the Unix manual page and Windows readme.txt in the respective distribution packages. However, since I want to use RTF for the Mac readme, I still need to spend some time improving Amethyst's RTF output so that tables are laid out correctly.
My plan is to continue updating the readme documentation (some outdated sections remain) and work on the to-do issues.
danij:
Last week I was primarily focused on resolving the outstanding resource loading regressions.
The main change was the implementation of a percent-encoded representation for resource paths (lump names, texture and material paths and more).
Doom's Wad format allows characters in lump names not generally allowed for file and/or directory names on the local file system, including those typically used as path delimiters. As all resource paths throughout the engine are now represented as URIs in a virtual directory hierarchy, it was necessary to normalize these with a unified path representation.
To this end a percent-encoded representation was implemented which rewrites the virtual paths for all lumps loaded from Wad archives. As the lump names are interpreted into symbolic texture (and thus material) identifiers, those too are now percent-encoded.
This percent encoding is mostly hidden from a user's point of view and the engine will normally decode paths before they are presented to the user.
Hitherto, these reserved characters in lump names prevented high resolution replacement resources being used on lumps which use them. One prominent example being Doom's Archvile, which uses the backslash character for an entire frame of it's resurrection animation. The new percent-encoded representation means that such resources can now use high resolution replacements by applying this encoding to the resource file name. For example, to replace the sprite frame VILE\1 the high resolution resource should be named VILE%5C1 (.png/tga/pcx).
I also fixed a couple of bugs in the resource locator which resulted in the incorrect use of resources in certain game modes (for example, Doom's titlepic being used when playing Doom II).
In addition to the work on resource system, I also spent some time tweaking the renderer and updating a few default effects. Users updating to today's build will need to delete their game configs in order to see some of these changes (binding configs are unaffected).
My plan for this week is to polish up the games' user interfaces (primarily the game menu) and document the changes and new features now available since the merger of the ringzero branch.
Comments
http://pastebin.com/xwzjM3kK