Week 46/2012: Column unstuck, reorganized internals
skyjake's notes:
Last week was relatively busy for me with the vanilla-blockmap branch, bug fixing, reorganizing engine internals, and working with DaniJ on the file system refactoring.
vanilla-blockmap
The E1M7 stuck column has been a known issue for a long time. We thought this would be solved by using the original, vanilla BLOCKMAP data and applying it just as the original game did, but it turned out this is not the correct solution for E1M7. While using the vanilla blockmap will help solve subtle gameplay bugs and regressions vs. vanilla behavior, the stuck column was caused by our use of accurate, floating-point math for map and object coordinates. In the original game, all math was fixed-point, with means that the fractional part of coordinates has only 16-bit accuracy. With this accuracy, the bounding boxes of the guys standing on the raised E1M7 column were not considered to be inside the surrounding walls.
The solution we applied to fix this problem was to use fixed-point math in the specific case of testing whether bounding boxes are colliding with lines. However, since we do want to take advantage of the higher precision of our coordinates, we will still improve this solution later by only applying it in the appropriate vanilla compatibility mode, when relying on the original BLOCKMAP data and iteration routine.
Work on vanilla-blockmap will continue with a look at the gameplay bugs that can be solved by applying the original blockmap data.
Internal reorganization
A big portion of my time was spent moving engine source files into appropriate domain-specific subfolders, such as "render", "resource", "ui", "audio", "client", "server", etc. This brings a lot of needed clarity into the internal structure of the engine, and allows us to better manage the next steps of the transition towards the 2.0 architecture. When we split the client and the server into separate executables, we have to know which parts are shared functionality (belongs in libdeng1/2) and which parts are client or server specific. The reorganization is also important so that we (the developers) get a clearer mental model of how the engine is divided into different domains of functionality, and how those domains should interoperate.
In addition to the renderer and resource management subsystems, there has been a vaguely defined subsystem called "refresh". Its job has roughly been to bind the renderer to the UI. To bring more clarity into the engine's internal structure, the "refresh" subsystem was broken down and the contents were moved to where they actually belong (render/resource/ui).
I continued moving shared/generic code into libdeng1 (timer, blockset, StringArray, Animator, Point, Size, and Rect were moved). There are still several generic modules/classes left in the engine that should be moved.
File system
I was working together with DaniJ on the FS refactoring. In our discussions it become clear that the Uri class is a crucial element in the overall architecture of the engine: it will be used for identifying resources of any kind, including files. Therefore I took some time to clean its implementation to better separate the C++ interface from the C legacy wrapper (facilitating its move to libdeng2 further down the road) and refactoring the implementation for simplicity/consistency and to remove redundancies.
I hope to be able to contribute more to the file system refactoring in the coming weeks so we can get it in a good shape before the 1.9.10 candidate phase begins in December.
Minor things
The API documentation needed a bit of sprucing up. I upgraded Doxygen and the CSS files, and went through some of the errors reported by Doxygen to improve the quality of the documentation.
Finally, I added a new command line option called "-loglevel" that allows one to set the level of log messages (e.g., "-loglevel verbose" or "-loglevel debug"). This is part of the work where we're shifting responsibility of certain facilities to libdeng2.
- - - - - - - - - - - - - - -
danij's notes:
skyjake has amply covered most of what we've been doing over the past week. So, I'll instead focus on what has been achieved thus far in the file system refactoring work and what this will ultimately mean for Doomsday.
Currently Doomsday implements two independent file systems in parallel. We have the original file system (referred to as "FS1"), which manages the majority of the files. There is also the new file system, originally developed for Hawthorn (dubbed "FS2"). FS2 was introduced to the architecture by skyjake alongside the new Doomsday Script scripting core because of how deeply scripting has been interwoven into the fabric of the 2.0 architecture.
Over the years since FS1 was originally designed it has evolved several interesting features and design characteristics that I felt would be beneficial in the 2.0 architecture. Not least of which being it's optimized search algorithms and relatively small memory footprint. My focus for the upcoming 1.9.10 release has been to develop the ideas and concepts of the FS1 architecture into such a form that they can be assimilated into the design of the new FS2 filesystem and thereby hopefully achieving the best of both.
As one might expect from a system which has evolved incrementally, FS1's implementation lacked coherence (not helped by the fact it was limited by the relative constraints of the C programming language). Consequently cleaning this up has been rather a long process. Last week that work reached conclusion and we are now at the point where these components can be integrated (e.g., PathTree and Uri) into FS2.
My plan for this week is to begin work on that integration.
Last week was relatively busy for me with the vanilla-blockmap branch, bug fixing, reorganizing engine internals, and working with DaniJ on the file system refactoring.
vanilla-blockmap
The E1M7 stuck column has been a known issue for a long time. We thought this would be solved by using the original, vanilla BLOCKMAP data and applying it just as the original game did, but it turned out this is not the correct solution for E1M7. While using the vanilla blockmap will help solve subtle gameplay bugs and regressions vs. vanilla behavior, the stuck column was caused by our use of accurate, floating-point math for map and object coordinates. In the original game, all math was fixed-point, with means that the fractional part of coordinates has only 16-bit accuracy. With this accuracy, the bounding boxes of the guys standing on the raised E1M7 column were not considered to be inside the surrounding walls.
The solution we applied to fix this problem was to use fixed-point math in the specific case of testing whether bounding boxes are colliding with lines. However, since we do want to take advantage of the higher precision of our coordinates, we will still improve this solution later by only applying it in the appropriate vanilla compatibility mode, when relying on the original BLOCKMAP data and iteration routine.
Work on vanilla-blockmap will continue with a look at the gameplay bugs that can be solved by applying the original blockmap data.
Internal reorganization
A big portion of my time was spent moving engine source files into appropriate domain-specific subfolders, such as "render", "resource", "ui", "audio", "client", "server", etc. This brings a lot of needed clarity into the internal structure of the engine, and allows us to better manage the next steps of the transition towards the 2.0 architecture. When we split the client and the server into separate executables, we have to know which parts are shared functionality (belongs in libdeng1/2) and which parts are client or server specific. The reorganization is also important so that we (the developers) get a clearer mental model of how the engine is divided into different domains of functionality, and how those domains should interoperate.
In addition to the renderer and resource management subsystems, there has been a vaguely defined subsystem called "refresh". Its job has roughly been to bind the renderer to the UI. To bring more clarity into the engine's internal structure, the "refresh" subsystem was broken down and the contents were moved to where they actually belong (render/resource/ui).
I continued moving shared/generic code into libdeng1 (timer, blockset, StringArray, Animator, Point, Size, and Rect were moved). There are still several generic modules/classes left in the engine that should be moved.
File system
I was working together with DaniJ on the FS refactoring. In our discussions it become clear that the Uri class is a crucial element in the overall architecture of the engine: it will be used for identifying resources of any kind, including files. Therefore I took some time to clean its implementation to better separate the C++ interface from the C legacy wrapper (facilitating its move to libdeng2 further down the road) and refactoring the implementation for simplicity/consistency and to remove redundancies.
I hope to be able to contribute more to the file system refactoring in the coming weeks so we can get it in a good shape before the 1.9.10 candidate phase begins in December.
Minor things
The API documentation needed a bit of sprucing up. I upgraded Doxygen and the CSS files, and went through some of the errors reported by Doxygen to improve the quality of the documentation.
Finally, I added a new command line option called "-loglevel" that allows one to set the level of log messages (e.g., "-loglevel verbose" or "-loglevel debug"). This is part of the work where we're shifting responsibility of certain facilities to libdeng2.
- - - - - - - - - - - - - - -
danij's notes:
skyjake has amply covered most of what we've been doing over the past week. So, I'll instead focus on what has been achieved thus far in the file system refactoring work and what this will ultimately mean for Doomsday.
Currently Doomsday implements two independent file systems in parallel. We have the original file system (referred to as "FS1"), which manages the majority of the files. There is also the new file system, originally developed for Hawthorn (dubbed "FS2"). FS2 was introduced to the architecture by skyjake alongside the new Doomsday Script scripting core because of how deeply scripting has been interwoven into the fabric of the 2.0 architecture.
Over the years since FS1 was originally designed it has evolved several interesting features and design characteristics that I felt would be beneficial in the 2.0 architecture. Not least of which being it's optimized search algorithms and relatively small memory footprint. My focus for the upcoming 1.9.10 release has been to develop the ideas and concepts of the FS1 architecture into such a form that they can be assimilated into the design of the new FS2 filesystem and thereby hopefully achieving the best of both.
As one might expect from a system which has evolved incrementally, FS1's implementation lacked coherence (not helped by the fact it was limited by the relative constraints of the C programming language). Consequently cleaning this up has been rather a long process. Last week that work reached conclusion and we are now at the point where these components can be integrated (e.g., PathTree and Uri) into FS2.
My plan for this week is to begin work on that integration.
Comments
However, Unfortunetly, there is still a case regarding mobj hitboxes overlapping inappropriate sectors, that wasn't fixed by the fix.
On one of HeXen DK's maps, Map35, there is an Ettin right next to a lift sector (sector 15). If one activates the lift without alerting the Ettin, he suddenly jumps into the ceiling up to the height of the floor of the lift sector and blocks the lift from coming down. As if the game is confused about what sector the unalerted Ettin is in; his hitbox does just overlap the lift sector.
If you alert the Ettin before activating the lift, he moves away from it without any issue.
If you look at this map, you'll also quickly notice the map also has this unrelated bug:
http://sourceforge.net/p/deng/bugs/1033/