New Neural Upscale Project

Hi everyone! I don't think I've really been active here since the old forum, but I've been lurking periodically and benefiting from projects like the DHTP and DHMP. That's why I want to share something with the DE community that I've been working on for a few weeks. Even though it's a long way from finished, and not yet ready for a wider release, I think it's far enough along that it's "useable."

https://drive.google.com/open?id=1XJcOykz2xhnyVZVwl9gF1TVT9QONDXq-

I know there's already a neural up-scaling project, but it's limited to only 2x the original resolution, and the tools available have improved since it was made. Using a variety of pre-trained models, I currently have every patch and flat in The Ultimate Doom (including monster sprites) scaled up by 4x. Wall textures are scaled by 8x.

In terms of detail and realism, these are inferior to the flats and textures in the DHTP, but my goal is to complement and supplement other resource packs, not to compete with them. I've tried loading the DHTP and DHMP on top of my project, and I think it works pretty well as a way to fill the gaps. By itself, I think it meshes better with PWADs that introduce new sprites and textures alongside the originals.

At this point, this is mostly raw output from several pre-trained neural networking models (which I will catalog once I've made more final decisions on what to use) that I've hand-selected. Very little editing has been done so far, and most of what has been was automated. I've retouched a handful of sprites where I felt it was desperately needed. Later on, I will be selectively compositing pieces from different results for the textures, so the finished project should be much better. I'm not sure the sprites will ever look great under close scrutiny, but I think they look pretty good in action during regular play.

Still to do:
  • Test more models
  • Fix minor alpha channel errors
  • Correct seams and tiling issues
  • Cover assets from DOOM II, Final DOOM, and SIGIL
  • Rebuild textures from the output of multiple models, according to which ones give the best results for different parts of a texture
  • Harmonize textures and flats that share similar materials (wood textures should all use the same neural model, etc)
  • Adjust colors and brightness levels to be more faithful to the original assets

I'll post some example textures and sprites below, but I think the best way to check it out is to try it. It all looks much better in game than it does on its own. Let me know what you think, but it is a long way from complete, so please be kind! :D

Comments

  • Pinky
    lsh371dhpqf3.png
    lg2agr4d9nqj.png

    Imp
    52as9qcsylug.png

    Nukage barrel
    56jydtkf2csn.png

    The screaming head of John Romero
    8ojo4g7ea164.png
  • Hm, the enemy sprites seem rather uniform/smooth, in a way, sterile. The originals seem to have more definition to them. The algorithm seems to have designated a bunch of things that are actually supposed to be there as low-res artifacts and smoothed them away.
  • Played with them ingame; the textures are good, the hud weapons pretty good too; but the all the enemies, too much details has been smoothed out. At the distance they're okay, but if the monster get in your face, you notice, heavily.
  • edited 2020 Jun 3
    3DMaster wrote: »
    Played with them ingame; the textures are good, the hud weapons pretty good too; but the all the enemies, too much details has been smoothed out. At the distance they're okay, but if the monster get in your face, you notice, heavily.

    Thanks for the feedback! Yeah, I agree about the sprites. The trouble is that compared to the textures, the original sprites are jammed with little one or two pixel details that are subtle and open to interpretation. The AI models think it's noise or dithering, or they make strange interpretations (you should see some of the versions I rejected). I picked the results that were best overall, but there are other results that capture some of those details a little better. Unfortunately, the more detail a model sees, the more weird interpretations it tends to make, so the best overall are a bit smooth. I hope to be able to selectively transplant some of that detail over, but I don't think they'll ever look great up close. At least not without an unreasonable amount of editing and redrawing.

    On the plus side, I do think they look somewhat better at medium and slightly-close distances than blurred or unfiltered scaling, especially in front of hi-res textures. Every alternative besides DHMP models look pretty bad when they're up in your face, too.

    I'm hoping that once these have been cleaned up a little and had some detail restored, I might be able to upscale them again to 8x or 16x to make them look a little better up close, but that remains to be seen.
  • I might also experiment with upscaling the 2x sprites from the other neural upscaling project. For now, I don't want to use anyone else's work, even though the author gave permission.
  • Dear flamepanther,
    I joined the forum just to congratulate you on your fine work and to also thank you for it. I have been looking for a neural texture for Doom that works on Doomsday engine for AGES!! I will be checking out your progress often. Thanks again!!
  • Just played Doom for a while with flamepanther's pk3 file and it looks AMAZING!! What progress for a first try. Love. It. :smiley:
  • I've gone through all the suitable models at my disposal for the textures, so I uploaded an updated pk3 to my Google Drive. These are still mostly untouched "best overall" textures. It will be awhile before I have another update, as the next few changes I want to do will require a lot of manual work. My main focus is still on the wall textures.

    If anyone cares to help out, please compare this pack with the previous one and let me know if you feel there have been any regressions or specific areas where I should give extra attention. I'm also not too great at getting good, dramatic screenshots, so feel free to post any here if you can.
    morklii wrote: »
    Just played Doom for a while with flamepanther's pk3 file and it looks AMAZING!! What progress for a first try. Love. It. :smiley:
    Thank you so much for the positive feedback! I hope you'll like it better with each update :)
  • Hi flamepanther,

    Overall I think it looks much better. In particular, all weapons, the Imp and Cacodemon are absolutely perfect. The objects that I think need improvement/focus are the following:
    -large medkit (the red cross looks rounded rather square on each arm)
    - pinky (outer edges of character does not look right)
    -zombieman and the shotgun guy faces look quite messy and their eyes look too big.

    Keep up the great work and thank you so much!!
  • Here are some more things that need improvement which I attach pictures to show:
    -small medkit with rounded cross 4z2oohz7c4cq.jpg

  • It can be seen that the Zombie's nose bends up to the top of his face
    l69e0fkjaba8.jpg
  • Forearm has wrong colors (blue and yellow bits) and its outline looks wrong. Also the resting black outline of the resting fist looks wrong x28t8qbqnf1v.jpg
    .
  • A dramatic screen shot showing amazing work as requested :)
    umzhpkoila73.jpg
  • edited 2020 Jul 21
    Work is coming along well on building composite wall textures from upscales made with different AI models. So for example, on a wood and metal door, I can use the best result for the wood and the best result for the metal, instead of having to pick one or the other. I'll probably make another upload once I'm done, and then update the flats using the same methods I've used for the walls.

    I've also tried quantizing and palettizing the textures, and found that it causes no obvious quality degradation for most textures and even slightly improves a some of them. Very few textures end up with noticeable dithering. I'll leave those few as truecolor and palletize the rest, meaning the package will be drastically smaller.

    This last bit is for much later on, but I've made some initial attempts at enhancing things the AI simply can't. For example, I've made an effort to reconstruct the typeface of the UAC computer terminals by hand. I also have a proof-of-concept version of a single frame of the imp sprite with details added from the Doom II RPG. I won't be working on these types of improvements in earnest until the project is more mature, and I'll definitely make them optional.

    @morklii thanks for the feedback! I'll keep these in mind when I get back to focusing on sprites.
  • Here's a proof of concept for enhancing the imp by hand. It's not finished, and only the front and back angles are possible. But those are the angles you'd see them from most often anyway, especially up close. I don't think the inconsistencies will be too jarring while the enemies are moving around.

    bialcq603w7z.png
  • Your update sounds very exciting and look forward to seeing the completed imp rework! Thanks for all your hard work flamepanther! One small comment on the imp - the color on the legs and arms looks banded. Other than that, it looks great.
  • Flamepanther - I'm dying for some updates from you! ;)
  • Flamepanther - I'm still dying (getting a bit dark now, breaths are shallow, life flashing before my eyes) for some updates from you! ;);)
Sign In or Register to comment.