Add-on info and package metadata

This discussion was created from comments split from: Build 2889: weapon slot keybinding messed up.

Comments

  • edited 2018 Dec 2
    1. I'm struggling with the info file format. Current Doomsday seems not to honor every provided entry.

    Example:
    name: Hi-res Skies pack (deus-ex edition v1.1)
    category: textures
    provides: doom-hires-skies (environment images)
    component: game-jdoom
    category-resources: Resource Packs
    
    language english (
      version: 1.1
      summary: jDOOM Hi-res Skies pack (deus-ex edition)
      contact: https://talk.dengine.net
      author: KuriKai, lostless, PickleBro, et al., deus-ex
      copyright: Copyright (c) 2004-2018 deus-ex
      license: Free for non-commercial use
      readme: This module provides hi-res skies (environment images) for jDOOM.
    )
    

    And this is what I get to see in the launcher.
    jwswe2q5qf5s.jpg

    Besides being erroneously detected as Hexen addon, the entries 'name', 'category', 'provides', 'component', 'category-resources', 'summary' and 'copyright' seem to get ignored, i.e. when I rename the package itself, the new name appears in the description view, not the one defined in the info file. I also tried adding 'tags: doom doom2', but without success.

    2. Adding an icon.png/.jpg does show a small image next to the addon entry, but right-clicking the addon the icon image is not shown in detail view, as opposed to DD 2.0 addon packs using a info.dei file, like the DHMP.

    3. I tried porting the info to the DD 2.0 .pack info.dei format, but that fails as Doomsday expects a block section for the model, despite providing the .ded file in the .pack file.

    Please clarify whether the .pack format support supports old MD2/DMD models and .ded definitions.

    4. Please post working examples both of a old info file including the parameters that are actually honored by the current Doomsday builds, and also a example info.dei if it does support old MD2/DMD models with a .ded file.
  • edited 2018 Dec 3
    deus-ex wrote: »
    1. I'm struggling with the info file format. Current Doomsday seems not to honor every provided entry.
    You are correct, there are currently entries that are not used. The Info format was originally created for the Snowberry front-end app, so it contains things that do not have a counterpart in Doomsday 2.

    It's a good idea to document this in detail with examples, I'll do that in the Manual.
    deus-ex wrote: »

    2. Adding an icon.png/.jpg does show a small image next to the addon entry, but right-clicking the addon the icon image is not shown in detail view, as opposed to DD 2.0 addon packs using a info.dei file, like the DHMP.
    That sounds like a bug, I'll check it out.
  • edited 2018 Dec 5
    I could not yet find any example of a Snowberry addon definition explaining what parameters are actually honored by Doomsday 2.0. Are you going to post one soonish?

    Because I have the unresolved issue with Doomsday identifying my "Doom Hi-res Sky textures pack" as a Hexen addon, when I add the"info" file (see my previous post above) to it.

    I continued trying to figure out how to port Snowberry addons to the Doomsday 2.0 pack format, studying the available documentation. For testing I used the TechPillar model from the jDRP.
    • First of all the DMD format is not supported, those models have to be back ported to the MD2 format first.
    • Despite adding the parameter 'defsPath = "defs"' and placing the DED file in the subfolder 'defs', Doomsday appears to be unable to find the model file. I tried adjusting the path directive in the DED file, and also making available copies of the MD2 model in several locations, all to no avail:
      root folder\Techpillar.md2
      root folder\models\Techpillar.md2
      root folder\decor\Techpillar\Techpillar.md2
      
    • Trying to get the model to show up in Doomsday, without making use of the old DED file, also did not succeed:

      (I commented out several entries, hoping to get it to work, without success)
      title: Tech Pillar (deus-ex edition)
      version: 1.0
      license: Free for non-commercial use
      tags: doom doom2
      
      asset model.thing.techpillar {
        path = "techpillar.md2"
      #  front <0, 0, 1>
      #  up <0, 1, 0>
      #  autoscale = true
      #  alignment.yaw = random
        material {
            variant "default" {
                @0 {
                   diffuseMap  = "textures/techpillar.png"
      #            specularMap = "textures/twinspot.png"
                }
            }
         }
      }
      
    1. Doomsday does not complain if the meta data file is named 'info.dei' (as used in the DHMP) instead of just 'info', as according to your manual. Probably not an issue, I just mention it to clarify.
    2. It is unclear how the values of the parameters front and up are working, this needs more detailed explanation. Paramter up appears to be a substitute for the old Scale XYZ { 1 1 1 }?
    3. There seem to be no parameter available to control the model shadow?
    4. There seem to be no parameters available to control the model shiny strength and color, with regards to Snowberry models that do not have additional normal, specular and emissive textures?
    5. Quote from manual: "Tags cannot contain spaces."
      This is ambiguous. If I got it right, more than one tag can be defined, each separated by a space char, e.g. 'tags: doom doom2'.
  • deus-ex wrote: »
    I could not yet find any example of a Snowberry addon definition explaining what parameters are actually honored by Doomsday 2.0. Are you going to post one soonish?
    Sorry, I'm a bit busy so haven't yet posted and not sure exactly when. I'll try to do that during the next couple of days.

    In general, though, you should note that there are currently two entirely separate model renderers in the engine. The old MD2/DMD one is being kept around for backwards compatibility, but is not being developed any more. It requires PK3 packages (or regular unpackaged separate files). The new rendered that supports modern file formats, skeletal animation, PBR shaders, etc. requires Doomsday 2 packages. The Info for these two renderers is different: MD2/DMD metadata is the old DED formats described in the Manual's DED section, and the new renderer uses the 3D model assets system.

    I can elaborate and reply to your other questions after a while.
  • The main concept of the two package formats and why they exist was already clear to me. The idea trying to port the old addons to DD2.0 packs was fueled by the following manual entry:

    22wstc3qxrtw.png
    Source: https://manual.dengine.net/fs/packages

    I guess for non-model based addons containing just textures or patches, the addon format does not make any difference? Or would it be preferable to have those ported over? If so, a description of and/or a example DD2.0 pack would be helpful.
  • edited 2018 Dec 6
    For clarity, this comment will refer to two kinds of Info files:
    • Snowberry-style PK3/Box "Info" file (with "component: game-jdoom" etc.). Info found in PK3s is always assumed to be Snowberry-style.
    • Doomsday 2 package metadata in a .pack, typically named "info.dei", but "Info" is an accepted alternative name.
    deus-ex wrote:
    Besides being erroneously detected as Hexen addon
    That is strange... The "component" value is checked in Snowberry Infos and automatically causes the "doom" and "doom2" tags to be applied to the file. Hmm, maybe they got overridden due to something else later on.
    the entries 'name', 'category', 'provides', 'component', 'category-resources', 'summary' and 'copyright' seem to get ignored, i.e. when I rename the package itself, the new name appears in the description view, not the one defined in the info file.
    "name" is checked and it defines the title of the file. It does work for me in PK3s that I've tested. (And your screenshot has the correct title, too.)

    Have you tried manually clearing the resources cache after your renaming?

    "provides", "category-resources", and "copyright" are not read by Doomsday.

    The following values from the language block are read and applied: "version", "author", "license", "contact", "readme".
    info file. I also tried adding 'tags: doom doom2', but without success.
    At the moment, tags are not supported in Snowberry-style Info files. Snowberry never had a tagging feature. It did use a categorization, though, and those become Doomsday 2 tags when the file is indexed. For instance, you can see that your "category: textures" has become a "textures" tag.
    Please clarify whether the .pack format support supports old MD2/DMD models and .ded definitions.
    It does not. Doomsday 2 packages can contain DED files, but the old MD2/DMD model resource loader cannot read Doomsday 2 packages.

    There is a workaround where you can put PK3 files inside a Doomsday 2 package and auto-load them when the .pack is loaded. That way you can control the Doomsday 2 package metadata exactly without relying on the automatic PK3 backwards compatible metadata.
    Please post working examples
    That will take some time to prepare, but it should be an attachment in the Manual page discussing the metadata support. It's in the todo list now.
    unresolved issue with Doomsday identifying my "Doom Hi-res Sky textures pack" as a Hexen addon
    It could be a bug in the metadata parser. If you can share the files with me I can try to debug it.
    First of all the DMD format is not supported, those models have to be back ported to the MD2 format first.
    Models in the Doomsday 2 package format, using the "asset" definitions, always use the new model renderer. The new renderer does not support DMD and MD2 models (becase these rely on frame-based animation instead of skeletal animation).
    placing the DED file in the subfolder 'defs', Doomsday appears to be unable to find the model file.
    As noted above, the old model renderer is unable to read files from Doomsday 2 packages. This will not be changing in the future (according to current plans).
    It is unclear how the values of the parameters front and up are working, this needs more detailed explanation.
    Yeah I can see the documentation about those is a bit terse. :) If one knows which way the coordinate axes are pointing in the model, one can use these parameters to tell that information to Doomsday, so it can map the coordinate axes to the Doomsday world directions. The length of the vectors affects the sizing of the model, because the vectors get normalized to length=1.0 during the process.
    There seem to be no parameter available to control the model shadow?
    Object shadows are a feature in the old/classic renderer, and I doubt a similar feature will be implemented for the new renderer. (Shadows will be handled via ambient occlusion and actual shadow mapping.)
    no parameters available to control the model shiny strength and color
    These are features of the old model renderer, and therefore not supported in the new format.
    "Tags cannot contain spaces."
    This is ambiguous. If I got it right, more than one tag can be defined, each separated by a space char, e.g. 'tags: doom doom2'.
    Ah yes, the "tags" field can definitely contain spaces, but each individual tag cannot.
    I guess for non-model based addons containing just textures or patches, the addon format does not make any difference? Or would it be preferable to have those ported over?
    In general, I would advise keeping old addons in the old formats. As the new renderer gets implemented and has wider support for different resource types, the Doomsday 2 asset system will be expanded to support these. Until then, porting old resources won't make much sense.
  • skyjake wrote: »
    Doomsday 2 package metadata in a .pack, typically named "info.dei", but "Info" is an accepted alternative name.
    The manual refers to the file 'info', the name 'info.dei' is not mentioned anywhere.

    sq3szvr8cfof.png
    Source: https://manual.dengine.net/fs/packages

    skyjake wrote: »
    Have you tried manually clearing the resources cache after your renaming?
    No, I haven't. I just updated to build 2895, the pack naming appears to be working as intended.

    skyjake wrote: »
    The new renderer does not support DMD and MD2 models (becase these rely on frame-based animation instead of skeletal animation).
    The manual provides following information regarding the Doomsday 2 package format:

    8er0yjp73tut.png
    Source: https://manual.dengine.net/fs/packages

    skyjake wrote: »
    That is strange... The "component" value is checked in Snowberry Infos and automatically causes the "doom" and "doom2" tags to be applied to the file. Hmm, maybe they got overridden due to something else later on.
    I tried several different component entries for the info file of my 'Hi-res Skies' addon, which gives the following results:
    Parameter                  Detection
    ____________________________________________
    component: game-jhexen   = Hexen addon
    component: game-jheretic = Heretic-Ext addon
    component: game-jdoom    = Hexen addon
    component: <n/a>         = Hexen addon
    

    It could be a bug in the metadata parser. If you can share the files with me I can try to debug it.
    Yes, I just found the nasty little bastard causing me so much trouble:

    I tested my 'Hi-res Skies' addon with different iterations of the info file, starting with barely no content and each time adding back more info after a successful detection as a DOOM component. Doomsday would detect my addon correctly as a DOOM addon every time, but once you put any word into the readme: section that contains the character sequence "mage", like in "image", the addon gets identified as a Hexen addon. That's quite a funny bug.

    Example:
    language english (
      readme: mage
    )
    
    Placing only this much content into the info file brakes the component detection. Shortening the word to "age" or "mag" reenables correct detection.

    Since I found the bug, will it be named after me? You know, like stars get named after their discoverer. :D
  • deus-ex wrote: »
    The manual refers to the file 'info', the name 'info.dei' is not mentioned anywhere.
    Manual updated.
    The manual provides following information regarding the Doomsday 2 package format:
    You can put MD2 files inside a .pack as shown in the example, but the old MD2/DMD renderer cannot load them...

    (The test application, whose package that is in the example, loads the MD2 for testing purposes but it's simply a non-animated unlit model with one texture.)
    Yes, I just found the nasty little bastard causing me so much trouble:
    Quite a nice find! :+1:

    I pushed a fix, let's see if it helps. :)
Sign In or Register to comment.