Trim unused resources from Haks?
WilliamDraco
Member Posts: 175
A server I work with has recently transitioned to NWSync, which is working to great effect and we are very happy with.
Our understanding of the nwsync data is that resources are now stored on a per-resource basis and - for example should we use CEP and another NWsync server also use CEP - that data will not be downloaded again (assuming identical copies of the specific resource). In the past, our Hak files included the full CEP haks, even though we did not utilise the entire package, in order to save the need to re-download this content for players who already had the CEP.
Given out understanding of the anti-duplication above, the idea has therefore arise to strip the manifest of all such shared-data and only include the resources which the server actually uses. This will reduce the hak files, but additionally minimise the size (mb) and length (#of items) for the Manifest, which must be downloaded and parsed each time the server manifest updates.
I ask therefore whether a tool currently exists which can:
1: Identify all resources referenced within a Module
2: Compare this to a list of all resources in the associated hak files
3: and remove all resources not referenced from the hak?
or alternatively do any part of the above three.
I'm familiar with the work Niv has done on neverwinter_utils library, which seems like the starting point for items 1 and 2 perhaps? I have an amateur interest in learning nim myself, but it's a long way from being able to fully comprehend and adapt the tools if needed.
Thanks for any assistance that can be provided!
Our understanding of the nwsync data is that resources are now stored on a per-resource basis and - for example should we use CEP and another NWsync server also use CEP - that data will not be downloaded again (assuming identical copies of the specific resource). In the past, our Hak files included the full CEP haks, even though we did not utilise the entire package, in order to save the need to re-download this content for players who already had the CEP.
Given out understanding of the anti-duplication above, the idea has therefore arise to strip the manifest of all such shared-data and only include the resources which the server actually uses. This will reduce the hak files, but additionally minimise the size (mb) and length (#of items) for the Manifest, which must be downloaded and parsed each time the server manifest updates.
I ask therefore whether a tool currently exists which can:
1: Identify all resources referenced within a Module
2: Compare this to a list of all resources in the associated hak files
3: and remove all resources not referenced from the hak?
or alternatively do any part of the above three.
I'm familiar with the work Niv has done on neverwinter_utils library, which seems like the starting point for items 1 and 2 perhaps? I have an amateur interest in learning nim myself, but it's a long way from being able to fully comprehend and adapt the tools if needed.
Thanks for any assistance that can be provided!
3
Comments
For a lot of resources - e.g. item/creature/placeable appearances - it is not enough to scan the module for their uses, because scripts might dynamically modify them. SetAppearance(oNPC, Random(1000)); means that any of the first 1000 creature appearances could be used. There's also the DM SetAppearance console command, or creatures spawned via DM creator.
If you were to first remove the appearances from the 2DA, then a tool can find all the models/textures that are not used.
Other resource types, like unused tilesets, would be much easier to find and prune. But there is no tool to do it currently.
As a relatively once-off (if very large) process, we're content to manage the manafest as it is and simply manually curate on the occasion we spot stuff for now.
Thanks for you response!