Memory savings and metadata cache
<div id="bridgedd"> <div class="excerpt-title">This is a blog excerpt. <a href="http://dengine.net/blog/2017/02/memory-savings-and-metadata-cache/" target="_blank">Read Full Article</a> </div> <br /> <p><a href="http://files.dengine.net/builds/build2237">Build 2237</a> introduces a number of changes aimed at making Doomsday launch faster and use less memory overall. It also fixes a couple of threading issues that were found with the help of <a href="http://clang.llvm.org/docs/ThreadSanitizer.html">Thread Sanitizer</a>.</p>
<p>As part of preparing the stable 2.0 release, I've been looking at performance and memory optimizations.</p>
<h2>Trimming memory use</h2>
<p>I examined memory use in Instruments on macOS and identified a number of places for savings:</p>
<ul>
<li>There is now a <a href="http://source.dengine.net/apidoc/sdk/classde_1_1_pointer_set.html">purpose-built class</a> for keeping track of observer relationships between objects. There are hundreds of thousands of these being used in the engine, so using a class that is more optimal for the task saves both memory and improves performance.</li>
<li>Some objects had a lot of observers (20,000+) but worst of all, this was completely unnecessary because the observers were never being notified. These cases have now been removed.</li>
<li>Reduced memory used by Doomsday Script bindings for file system access by removing several unnecessary variables and replacing them with callable functions. This saves memory because functions can be stored in the class objects rather than in the objects themselves.</li>
<li>The file system caches uncompressed ZIP file contents in memory to avoid repeatedly decompressing them when the ZIP is being accessed. However, these cached contents were never released. Now ZIP caches are freed from memory whenever you return back to the Home screen.</li>
</ul>
<p>After all these changes, Doomsday's memory use on my system was roughly halved (from 830 MB to 460 MB). In practice, though, this heavily depends on how many resource packs and other data files you have available.</p>
<h2>Caching metadata</h2>
<p>When Doomsday is starting up it indexes and quickly analyzes all the resource packs, add-ons and other data files that you may have available. This may take a while depending on the number of these files and how fast your hard drive is. Starting with build 2237, Doomsday is now able to cache this information for future use, so after the analysis has been done it doesn't have to repeated until the files change or new ones are added. </div>
<p>As part of preparing the stable 2.0 release, I've been looking at performance and memory optimizations.</p>
<h2>Trimming memory use</h2>
<p>I examined memory use in Instruments on macOS and identified a number of places for savings:</p>
<ul>
<li>There is now a <a href="http://source.dengine.net/apidoc/sdk/classde_1_1_pointer_set.html">purpose-built class</a> for keeping track of observer relationships between objects. There are hundreds of thousands of these being used in the engine, so using a class that is more optimal for the task saves both memory and improves performance.</li>
<li>Some objects had a lot of observers (20,000+) but worst of all, this was completely unnecessary because the observers were never being notified. These cases have now been removed.</li>
<li>Reduced memory used by Doomsday Script bindings for file system access by removing several unnecessary variables and replacing them with callable functions. This saves memory because functions can be stored in the class objects rather than in the objects themselves.</li>
<li>The file system caches uncompressed ZIP file contents in memory to avoid repeatedly decompressing them when the ZIP is being accessed. However, these cached contents were never released. Now ZIP caches are freed from memory whenever you return back to the Home screen.</li>
</ul>
<p>After all these changes, Doomsday's memory use on my system was roughly halved (from 830 MB to 460 MB). In practice, though, this heavily depends on how many resource packs and other data files you have available.</p>
<h2>Caching metadata</h2>
<p>When Doomsday is starting up it indexes and quickly analyzes all the resource packs, add-ons and other data files that you may have available. This may take a while depending on the number of these files and how fast your hard drive is. Starting with build 2237, Doomsday is now able to cache this information for future use, so after the analysis has been done it doesn't have to repeated until the files change or new ones are added. </div>