Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

New Neural Upscale Project

Hi everyone! I don't think I've really been active here since the old forum, but I've been lurking periodically and benefiting from projects like the DHTP and DHMP. That's why I want to share something with the DE community that I've been working on for a few weeks. Even though it's a long way from finished, and not yet ready for a wider release, I think it's far enough along that it's "useable."

https://drive.google.com/open?id=1XJcOykz2xhnyVZVwl9gF1TVT9QONDXq-

I know there's already a neural up-scaling project, but it's limited to only 2x the original resolution, and the tools available have improved since it was made. Using a variety of pre-trained models, I currently have every patch and flat in The Ultimate Doom (including monster sprites) scaled up by 4x. Wall textures are scaled by 8x.

In terms of detail and realism, these are inferior to the flats and textures in the DHTP, but my goal is to complement and supplement other resource packs, not to compete with them. I've tried loading the DHTP and DHMP on top of my project, and I think it works pretty well as a way to fill the gaps. By itself, I think it meshes better with PWADs that introduce new sprites and textures alongside the originals.

At this point, this is mostly raw output from several pre-trained neural networking models (which I will catalog once I've made more final decisions on what to use) that I've hand-selected. Very little editing has been done so far, and most of what has been was automated. I've retouched a handful of sprites where I felt it was desperately needed. Later on, I will be selectively compositing pieces from different results for the textures, so the finished project should be much better. I'm not sure the sprites will ever look great under close scrutiny, but I think they look pretty good in action during regular play.

Still to do:
  • Test more models
  • Fix minor alpha channel errors
  • Correct seams and tiling issues
  • Cover assets from DOOM II, Final DOOM, and SIGIL
  • Rebuild textures from the output of multiple models, according to which ones give the best results for different parts of a texture
  • Harmonize textures and flats that share similar materials (wood textures should all use the same neural model, etc)
  • Adjust colors and brightness levels to be more faithful to the original assets

I'll post some example textures and sprites below, but I think the best way to check it out is to try it. It all looks much better in game than it does on its own. Let me know what you think, but it is a long way from complete, so please be kind! :D

Comments

  • Pinky
    lsh371dhpqf3.png
    lg2agr4d9nqj.png

    Imp
    52as9qcsylug.png

    Nukage barrel
    56jydtkf2csn.png

    The screaming head of John Romero
    8ojo4g7ea164.png
  • Hm, the enemy sprites seem rather uniform/smooth, in a way, sterile. The originals seem to have more definition to them. The algorithm seems to have designated a bunch of things that are actually supposed to be there as low-res artifacts and smoothed them away.
  • Played with them ingame; the textures are good, the hud weapons pretty good too; but the all the enemies, too much details has been smoothed out. At the distance they're okay, but if the monster get in your face, you notice, heavily.
  • edited Jun 3, 19:51
    3DMaster wrote: »
    Played with them ingame; the textures are good, the hud weapons pretty good too; but the all the enemies, too much details has been smoothed out. At the distance they're okay, but if the monster get in your face, you notice, heavily.

    Thanks for the feedback! Yeah, I agree about the sprites. The trouble is that compared to the textures, the original sprites are jammed with little one or two pixel details that are subtle and open to interpretation. The AI models think it's noise or dithering, or they make strange interpretations (you should see some of the versions I rejected). I picked the results that were best overall, but there are other results that capture some of those details a little better. Unfortunately, the more detail a model sees, the more weird interpretations it tends to make, so the best overall are a bit smooth. I hope to be able to selectively transplant some of that detail over, but I don't think they'll ever look great up close. At least not without an unreasonable amount of editing and redrawing.

    On the plus side, I do think they look somewhat better at medium and slightly-close distances than blurred or unfiltered scaling, especially in front of hi-res textures. Every alternative besides DHMP models look pretty bad when they're up in your face, too.

    I'm hoping that once these have been cleaned up a little and had some detail restored, I might be able to upscale them again to 8x or 16x to make them look a little better up close, but that remains to be seen.
  • I might also experiment with upscaling the 2x sprites from the other neural upscaling project. For now, I don't want to use anyone else's work, even though the author gave permission.
Sign In or Register to comment.