New Neural Upscale Project
Hi everyone! I don't think I've really been active here since the old forum, but I've been lurking periodically and benefiting from projects like the DHTP and DHMP. That's why I want to share something with the DE community that I've been working on for a few weeks. Even though it's a long way from finished, and not yet ready for a wider release, I think it's far enough along that it's "useable."
https://drive.google.com/open?id=1XJcOykz2xhnyVZVwl9gF1TVT9QONDXq-
I know there's already a neural up-scaling project, but it's limited to only 2x the original resolution, and the tools available have improved since it was made. Using a variety of pre-trained models, I currently have every patch and flat in The Ultimate Doom (including monster sprites) scaled up by 4x. Wall textures are scaled by 8x.
In terms of detail and realism, these are inferior to the flats and textures in the DHTP, but my goal is to complement and supplement other resource packs, not to compete with them. I've tried loading the DHTP and DHMP on top of my project, and I think it works pretty well as a way to fill the gaps. By itself, I think it meshes better with PWADs that introduce new sprites and textures alongside the originals.
At this point, this is mostly raw output from several pre-trained neural networking models (which I will catalog once I've made more final decisions on what to use) that I've hand-selected. Very little editing has been done so far, and most of what has been was automated. I've retouched a handful of sprites where I felt it was desperately needed. Later on, I will be selectively compositing pieces from different results for the textures, so the finished project should be much better. I'm not sure the sprites will ever look great under close scrutiny, but I think they look pretty good in action during regular play.
Still to do:
I'll post some example textures and sprites below, but I think the best way to check it out is to try it. It all looks much better in game than it does on its own. Let me know what you think, but it is a long way from complete, so please be kind!
https://drive.google.com/open?id=1XJcOykz2xhnyVZVwl9gF1TVT9QONDXq-
I know there's already a neural up-scaling project, but it's limited to only 2x the original resolution, and the tools available have improved since it was made. Using a variety of pre-trained models, I currently have every patch and flat in The Ultimate Doom (including monster sprites) scaled up by 4x. Wall textures are scaled by 8x.
In terms of detail and realism, these are inferior to the flats and textures in the DHTP, but my goal is to complement and supplement other resource packs, not to compete with them. I've tried loading the DHTP and DHMP on top of my project, and I think it works pretty well as a way to fill the gaps. By itself, I think it meshes better with PWADs that introduce new sprites and textures alongside the originals.
At this point, this is mostly raw output from several pre-trained neural networking models (which I will catalog once I've made more final decisions on what to use) that I've hand-selected. Very little editing has been done so far, and most of what has been was automated. I've retouched a handful of sprites where I felt it was desperately needed. Later on, I will be selectively compositing pieces from different results for the textures, so the finished project should be much better. I'm not sure the sprites will ever look great under close scrutiny, but I think they look pretty good in action during regular play.
Still to do:
- Test more models
- Fix minor alpha channel errors
- Correct seams and tiling issues
- Cover assets from DOOM II, Final DOOM, and SIGIL
- Rebuild textures from the output of multiple models, according to which ones give the best results for different parts of a texture
- Harmonize textures and flats that share similar materials (wood textures should all use the same neural model, etc)
- Adjust colors and brightness levels to be more faithful to the original assets
I'll post some example textures and sprites below, but I think the best way to check it out is to try it. It all looks much better in game than it does on its own. Let me know what you think, but it is a long way from complete, so please be kind!
Comments
Imp
Nukage barrel
The screaming head of John Romero
Thanks for the feedback! Yeah, I agree about the sprites. The trouble is that compared to the textures, the original sprites are jammed with little one or two pixel details that are subtle and open to interpretation. The AI models think it's noise or dithering, or they make strange interpretations (you should see some of the versions I rejected). I picked the results that were best overall, but there are other results that capture some of those details a little better. Unfortunately, the more detail a model sees, the more weird interpretations it tends to make, so the best overall are a bit smooth. I hope to be able to selectively transplant some of that detail over, but I don't think they'll ever look great up close. At least not without an unreasonable amount of editing and redrawing.
On the plus side, I do think they look somewhat better at medium and slightly-close distances than blurred or unfiltered scaling, especially in front of hi-res textures. Every alternative besides DHMP models look pretty bad when they're up in your face, too.
I'm hoping that once these have been cleaned up a little and had some detail restored, I might be able to upscale them again to 8x or 16x to make them look a little better up close, but that remains to be seen.
I joined the forum just to congratulate you on your fine work and to also thank you for it. I have been looking for a neural texture for Doom that works on Doomsday engine for AGES!! I will be checking out your progress often. Thanks again!!
If anyone cares to help out, please compare this pack with the previous one and let me know if you feel there have been any regressions or specific areas where I should give extra attention. I'm also not too great at getting good, dramatic screenshots, so feel free to post any here if you can.
Thank you so much for the positive feedback! I hope you'll like it better with each update
Overall I think it looks much better. In particular, all weapons, the Imp and Cacodemon are absolutely perfect. The objects that I think need improvement/focus are the following:
-large medkit (the red cross looks rounded rather square on each arm)
- pinky (outer edges of character does not look right)
-zombieman and the shotgun guy faces look quite messy and their eyes look too big.
Keep up the great work and thank you so much!!
-small medkit with rounded cross
.
I've also tried quantizing and palettizing the textures, and found that it causes no obvious quality degradation for most textures and even slightly improves a some of them. Very few textures end up with noticeable dithering. I'll leave those few as truecolor and palletize the rest, meaning the package will be drastically smaller.
This last bit is for much later on, but I've made some initial attempts at enhancing things the AI simply can't. For example, I've made an effort to reconstruct the typeface of the UAC computer terminals by hand. I also have a proof-of-concept version of a single frame of the imp sprite with details added from the Doom II RPG. I won't be working on these types of improvements in earnest until the project is more mature, and I'll definitely make them optional.
@morklii thanks for the feedback! I'll keep these in mind when I get back to focusing on sprites.