1) haks at the server cannot all be distributed - due to script security in custom hak with scripts - distributed version is empty.
I'm glad you brought this up. We have a "server" hak full of resources players don't need, which looks mostly empty to them, but which builders can rev at our convenience server side. We put some stuff in there like server-only 2das, but also stable stuff like the crafting system scripts, so module recompiles are quicker. I wouldn't care too much if players were given updates of this stuff, but they' don't need it.
For security type script files, I guess those could land in your server override, no? We have enough clutter in our server HAK I wouldn't bother dumping it override if my only other option was to have players auto-update; that said, you riase some other good points about conflicts of versions of files between servers. Good times.
I think ability to edit and compile scripts inside haks would help a lot.
The ability to easily edit any file in a hak from the toolset would be amazing!
Yes Yes Yes - I said this either here or on another discussion - the need to remove resources to haks is necessary due to the module container resource limitation. Being able to work with areas, scripts, creatures, and well anything, in a hak, would be a BIG improvement.
Just a tiny request to add... But if we end up with an auto-downloader, could we have the server details text visible next to the download progress? (with the character limit removed.) It was/is pretty neat in NWN2 to be able to read the server info, rules etc whilst waiting for files to download.
@Azimn A community modder fixed it for Obsidian Entertainment a long time after the NWN2 launch.
exact @FreshLemonBun in NWN2 has partially solved this problem, every time you access a specific module or PW in NWN2 automatically before accessing the server if that module or PW has Hack or modd the game automatically downloads them inside the client of the connecting player, but the problem of that system and that the download was too slow and it took a while for the server to download the files, the solution could be this BD could implement it in NWN: EE.
what I meant, and the best solution, BG could further improve it so that for players and modders it is not always a drama to download the game files of a given PW.
Yes for the player it is not much of an issue but the server host must do some work so it might not be so accessible and convenient for new hosts in NWN2.
Yes for the player it is not much of an issue but the server host must do some work so it might not be so accessible and convenient for new hosts in NWN2.
Beamdog making that easier as well would clearly be a positive step.
Something I have not seen in this discussion: Bittorrent. Starcraft 2 uses the Bittorrent protocol for content and patch updates. This reduces the load on the central content server. Since Bittorrent is a standard protocol and has content integrity checks, I think it is a viable solution.
Something I have not seen in this discussion: Bittorrent. Starcraft 2 uses the Bittorrent protocol for content and patch updates. This reduces the load on the central content server. Since Bittorrent is a standard protocol and has content integrity checks, I think it is a viable solution.
A curious idea, though such would want to give players with terrible upload the ability to adjust the rate to ensure they don't choke their connection anytime a new player without haks tries to join. Not to mention the size of an uncompressed hak file can be abit large and not everyone wants to keep a compressed copy alongside an uncompressed copy on their PC.
Personally I think the ability for server host to link to an external ftp server with the haks and auto download and unpack them would be the best thing. Either an FTP server the host setup themselves or to a place like NWNvault or some such that already exists for them to upload their content to for download.
What if Steam Workshop integration were used to facilitate this? Workshop compares, downloads and auto-updates subscribed items.
When the client connects to a new world without the required haks/tlk, it could give them an option to open the Steam Workshop page/collection for the module. From there they can subscribe to and download all of the required haks and tlk.
The benefit of allowing Steam to auto-update subscribed players with the custom/unique hakpacks of their game worlds would massively open up the ability to modify, create, share, and utilize custom content.
What if Steam Workshop integration were used to facilitate this? Workshop compares, downloads and auto-updates subscribed items.
When the client connects to a new world without the required haks/tlk, it could give them an option to open the Steam Workshop page/collection for the module. From there they can subscribe to and download all of the required haks and tlk.
The benefit of allowing Steam to auto-update subscribed players with the custom/unique hakpacks of their game worlds would massively open up the ability to modify, create, share, and utilize custom content.
Personally I don't want to see the modding community be forced to hand over it's keys to the Valve Corporation.
I've had a bit of experience in creating my own downloader for my own server resources.
The way I see it - we would need to
1. Get a list of files from the server that need to be present on the client. 2. The checksum/hash of the file needs to match the one specified by the server. 3. If it does not match - re-download it 4. If it does match - skip to next file
One technology I have started getting familiar with is the IPFS (Interplanetary File System) - which is based on the DHT technology that bittorrent is built on. (Someone above mentioned bittorrent)
Pretty much anyone can choose to run a IPFS Daemon, and donate say 1GB of space to the network. Files can be added to the IPFS network via that daemon, and they only get distributed across the network, when they are requested by others on the network. Files are uniquely identified on the network by their contents (hash) - which prevents duplication of any file on the network (makes it very efficient). And content is shared P2P which means that no one person is having their bandwidth drained by sharing 3GB of haks with the wider world.
Now putting something simple into practice for nwnserver and nwmain.
nwserver would need to send the hak / tlk list - it already does this, but perhaps in addition to the filenames, we send the hashes of the files too or maybe the filenames are the hashes?
If the filenames were hashes of the file content, this would alleviate the issue of other servers having the same duplicate content etc. It would mean that the filename is based on the content, and it would only exist or download once - and once the client has it, it never needs to download again.
In anycase- this download issue is easy to solve outside of the game itself, perhaps an addition to the nwn launcher would be an easier solution? You add your PWServer to the launcher, it checks on startup with the server the hashes of server content vs local content and downloads the appropriate files. The question then becomes - does this downloader / verification function go into the server executable itself, or into a separate executable.
In my mind - it would be easier to add it into a separate executable, and I don't think the community would mind as long as the functionality is somehow provided.
On this thread in general, bear in mind that the topic is PW. No reason why PW should not have its own solutions, on which I wouldn't presume to comment.
However, no one should assume that ideas discussed here would be acceptable to SP authors and players. That would require a new thread, with an open invitation to the whole community.
Arguably, the discussion ought to be more open anyway, since a solution adopted for PW might create player expectations, pressurizing SP authors to follow suit.
SP authors are likely to have views on
Version control at the game object level (not merely at hak level)
Its interesting you mention the 'game object' level - I was just thinking to myself that it might be interesting if 'any' system that were created were to operate at the are,itm,gic,git etc file level (gff objects I think?)
If we are looking for a solution that is avoiding commercial sites, or any sites that are centralized in general - then a technology such as IPFS could be perfect for that.
Lets imagine someone has decided what their 'content' list looks like, they have then got it arranged in a JSON format - or some other serialized format.
That file list can then be added to the IPFS network, which generates a unique hash - which essentially becomes the 'address' of the file on the network.
Sharing the file list becomes a simple case of sharing the unique identifying hash. A hash looks like this on IPFS: QmPjAZoG5fmCtkt7aA8q5wc13nRf37VUwkKJHEaSyYQpNi
This is an example fileList - you might be able to tell from this filelist that I have added the html files from the 1.69 Lexicon.chm file to the IPFS network as a POC.
Each of those hashes in the file entries can be used to access / pull that resource from the IPFS network. It takes a bit of effort to get it working for a website, but it works perfectly for general file storage.
I digress... In any case- that file list would then contain the hashes of other files on the IPFS network. (Your haks, your itm files, your tlk files - anything...) The SP or MP player would then just need to feed that FileList into their client or downloader application to retrieve those files from the IPFS network.
Certain disclaimers need to be understood though - anything added to the IPFS is immutable. Its basically uncensorable - once something is on the network, its there for as long as anyone wants it.
Content does get cleared from the network eventually (theoretically) - if the content is never requested by anyone, then the content never spreads across the network. (Its self cleaning essentially). Everyone who runs a 'node/daemon' for IPFS, allocates a fixed amount of space to the storage of said files. More popular/requested files will take priority over files that have not been requested in a while. So that file that no one wants will eventually slip into obscurity and be cleaned from the network. (Requesting the files regularly or hosting a node of your own is a good way to ensure your content always remains online)
It also prevents duplication of content - because each file only exists on the network once. (The system will not allow two duplicate files to be added, even if they have different names - its done purely based on file content)
An auto-downloader is about convenience. A prospective player does not need to know or care about a server, they click to join, a screen with some information and a progress bar pops up and they leave it on for a few minutes. What it is done they can click on create new character or choose existing character, they are now in the game and using the proper files from the character creation screen onward. No restarts, tabbing out, using steam, using a server's downloader, unpacking a zip or any other method is necessary.
The feature is about pure convenience and lowering the bar to entry for players to use servers that use hak content. Even the best of us can be lazy sometimes and we just want to try something out without even having to type a name or keyword into a search bar, we want to click and enter the world.
Syncing haks with steam might be nice for some folks that want to use that platform to manage their content and for creators that want to share their haks there. It should be an optional and separate issue from providing the most convenient and access friendly method possible.
Providing your own installer is something some already do, it's fine too, some provide installers, megazips, and individual zips all as options. Even in NWN2 where there is a functional auto-downloader some servers provide their haks as zips or from dropbox so there is always another option should someone have problems.
I think the main difference with NWN and NWN2 pw communities from a technical standpoint right now is that on NWN2 I'm pretty sure no server exists without their own haks.
An auto-downloader is about convenience. A prospective player does not need to know or care about a server, they click to join, a screen with some information and a progress bar pops up and they leave it on for a few minutes. What it is done they can click on create new character or choose existing character, they are now in the game and using the proper files from the character creation screen onward. No restarts, tabbing out, using steam, using a server's downloader, unpacking a zip or any other method is necessary.
The feature is about pure convenience and lowering the bar to entry for players to use servers that use hak content. Even the best of us can be lazy sometimes and we just want to try something out without even having to type a name or keyword into a search bar, we want to click and enter the world.
Syncing haks with steam might be nice for some folks that want to use that platform to manage their content and for creators that want to share their haks there. It should be an optional and separate issue from providing the most convenient and access friendly method possible.
Providing your own installer is something some already do, it's fine too, some provide installers, megazips, and individual zips all as options. Even in NWN2 where there is a functional auto-downloader some servers provide their haks as zips or from dropbox so there is always another option should someone have problems.
I think the main difference with NWN and NWN2 pw communities from a technical standpoint right now is that on NWN2 I'm pretty sure no server exists without their own haks.
I agree that the average user wants convenience. However - like Proleric said - there are two aspects of this. What about Single Player situations, where they want to download Haks / Resources for the single player module.
While a seamless 'Click on server, get files downloaded' approach does solve the Multiplayer aspect, it does not solve the single player situation.
I think 'content' needs to be easily install-able, regardless of single or multiplayer scenarios.
The issue I foresee is that if the solution is designed for a Multiplayer scenario, it results in a Single Player solution having to be built. However, if you make a system that is totally agnostic of Single or Multiplayer, then it is less dev time for the developers to implement / build into the base game. Eg: They don't have to cater to Single or Multi players - they just cater to 'users'.
Well, where are they getting the singleplayer module from? Usually, you find these modules on the vault or some other site. You click download, then unpack it in the modules folder.
Unless you're proposing a change to the above sequence (like an in-game vault browser?), it is safe to assume that the average player of custom SP mods is capable and comfortable performing those steps. If a SP module uses custom haks, the additional step is.. well, none. The packed module would have mod/, hak/, tlk/, mus/ and similar folders in the archive. Player unpacks it into their NWN userdir and they're all set.
If the SP module uses shared haks (like CEP, Q, etc), it's a bit more involved. They are linked to the master hak page, where they repeat the process. They'd need to manually check (or remember) if they already have this package, but that's hardly a huge hassle.
It seems to me the biggest pain point when it comes to single-player modules is their discoverability. Once a player has found a module they want to play, the process is pretty straightforward. But finding the modules requires leaving the game, going to a third party site and learning how to use it. This is a perfectly valid concern that could be addressed, but I think it is barely related to the feature requested here.
FWIW, there are installer-makers available that package all the custom content into a standalone .exe, and can check if the shared content is present, downloading it only if not. Module makers could then use these to ship their modules, and players would only need to run the installer. This has been the case for the last 10+ years, but hardly any modules used this (there are exceptions, like DoD). Not sure what conclusion is to be drawn from that fact..
Downloading and managing personal haks and overrides is a totally separate issue to this one.
Having a cleaner and more friendly way of downloading single player content might be beneficial but it does nothing to address the server issue which is what this topic specifically addresses. Beamdog has already stated their lack of interest in hosting and distributing files like the vault, and ppl can geek out over bit torrent or steam if they want but you still need to go somewhere to find the haks and overrides you want. Presumably you find the haks and the modules in the same place for example on the neverwintervault site, which is atrocious to browse, use google on it instead.
For a pw the demands are entirely different. You don't get a copy of the module, the server list is just a matching service and the module exists on a private server. The server builders have already selected and packaged the content over many collective years and in order to play you need an exact match or you risk problems for the player. The content is already non negotiable and the auto-downloader simply removes the excessive steps needed to start playing.
Just quoting you because it's the last reply, but this is a general reply to many of the posts above.
I still believe Steam allows many of these problems to be solved for the majority of non-PW cases. Using the Steam workshop requires NWN:EE have Steam integration anyway, so it's not an external utility or even in a separate window from the game, and virtually everybody already has experience using Steam.
Steam downloading is agnostic of singleplayer or multiplayer modules. When you don't have the haks to connect or open the module it can send you to the appropriate page, it doesn't matter if it's a persistent world or not.
From what I recall (somebody at) BD already wants some degree of Steam integration anyway, which is why I suggest this. Steam Workshop integration is far from my preferred solution, especially as a persistent world builder myself, believe me; I'd much prefer a more robust and extensible approach that doesn't rely on Valve or any kind of mandated central storage servers. But for what I imagine is the vast majority of singleplayer, LAN, and persistent world use cases the Steam Workshop should be more than sufficient; and where a PW has different needs or concerns incompatible with using the Steam Workshop, they can still use external loaders/launchers and whatnot.
Using Steam doesn't so much as solve any problems rather than give steam users another alternative way to download if and only if both the user and the creator have it setup for that. Using custom content in NWN doesn't require any special amount of work, just put the files in the right folder and it's done, in that respect every download method is agnostic of multi and single player.
Steam and other external services are a separate issue to the multiplayer auto-downloader.
While I have a Steam account I do not play this game there. I tend to avoid DRM as do others. I think a good number of people will not be playing on Steam. Any solution which mention steams provides a solution for only Steam users. What of hosts that do not own a Steam copy? Must they create a steam user to their team in order to upload things to the proposed Steam: EE workshop?
In that case why not make something which works with the nexus (I know some have issue with them) where files are automatically downloaded from there?
The community is already using neverwintervault.org, which we control ourselves. Why on earth would we want to use the Nexus, and be controlled by others? From past experience, it's an authoritarian site, which is actively hostile to the open content sharing and cooperation that has made NWN great. They have had an NWN section for years, but very few of us use it. The Vault is the go-to site for new work, and all the content made over the years.
Where the content gets hosted is arguably a different issue. I would imagine that BeamDog - if they provide the means for Automated / Seamless downloading of haks / tlk at server entrance - would give the server admin the ability to specify the path to the files in question. Be they on Vault or on Nexus, or on the server itself.
not sure what the solution for a Single Player scenario would be though.
Eg: try to spin up a module, it doesnt work because you need the haks and tlks. Perhaps better reporting in such a situation.
I think solving the MP issue would be a huge accomplishment - At the moment communities on PW's are contained in silos - it is hard to play on some servers because their haks and tlks are vastly different from others. If an auto download system was in place, that kept the resources automatically segregated per PW - that would help. That being said - if the filenames of the haks and tlks were hashes - then no segregation would be needed. Im not sure if the filename limit has increased or decreased beyond its previous size limit.
1) haks at the server cannot all be distributed - due to script security in custom hak with scripts - distributed version is empty.
I'm glad you brought this up. We have a "server" hak full of resources players don't need, which looks mostly empty to them, but which builders can rev at our convenience server side. We put some stuff in there like server-only 2das, but also stable stuff like the crafting system scripts, so module recompiles are quicker. I wouldn't care too much if players were given updates of this stuff, but they' don't need it.
For security type script files, I guess those could land in your server override, no? We have enough clutter in our server HAK I wouldn't bother dumping it override if my only other option was to have players auto-update; that said, you riase some other good points about conflicts of versions of files between servers. Good times.
-Dave
I think my statement - due to script security - is misunderstood. We have scripting systems we do not want to share with the rest of the world - things that make Realms of Annakolia unique and special. If these scripts were to leak out, other world could use them, and we would loose some of that which make us unique. SO - script security means scripts we do not want to share with the rest of the NWN community.
1) haks at the server cannot all be distributed - due to script security in custom hak with scripts - distributed version is empty.
I'm glad you brought this up. We have a "server" hak full of resources players don't need, which looks mostly empty to them, but which builders can rev at our convenience server side. We put some stuff in there like server-only 2das, but also stable stuff like the crafting system scripts, so module recompiles are quicker. I wouldn't care too much if players were given updates of this stuff, but they' don't need it.
For security type script files, I guess those could land in your server override, no? We have enough clutter in our server HAK I wouldn't bother dumping it override if my only other option was to have players auto-update; that said, you riase some other good points about conflicts of versions of files between servers. Good times.
-Dave
I think my statement - due to script security - is misunderstood. We have scripting systems we do not want to share with the rest of the world - things that make Realms of Annakolia unique and special. If these scripts were to leak out, other world could use them, and we would loose some of that which make us unique. SO - script security means scripts we do not want to share with the rest of the NWN community.
Isn't that what the entire open source movement is based upon? That a more diverse code base creates inovation for all users. Allowing projects to better flourish by providing the efforts of all in return for their own willing contribution?
Look how diverse and unique the Linux ecosystem is. How fantastic are OpenOffice, Wikipedia, MySQL, Firefox, WordPress, Apache... Android.
Being Open Source could benefit your world significantly more by providing you a brand new wealth of code. To learn tricks and workarounds you had never thought of and contribute in kind to that vast pool of knowledge. How much information. How much history have we lost over the past 15 years because our walled kingdoms collapsed and their is no recovering them?
It helps I am not a fan of boutique modding but if the worst thing that happens is we are all forced to go open source, everyone makes better worlds, and NWN:EE finds a larger long term fanbase when relaunched then I am kinda ok with you knowing how better my fishing script is than all of yours.
It's just nwn codes it's not going to be that special. I'm thinking more along the lines of "original content do no steal" memes rather than philosophical debate of proprietary vs open source vs free software. A lot tend to be tailored for a specific server rather than general systems useful to others.
All servers should not need to put scripts in haks either way whether it's an amazing script or a one line thing from a script generator.
I'm thinking more along the lines of "original content do no steal" memes rather than philosophical debate of proprietary vs open source vs free software.
If this were an Open Source kind of discussion we could have all three on separate but connected branches.
Downloading and managing personal haks and overrides is a totally separate issue to this one.
Having a cleaner and more friendly way of downloading single player content might be beneficial but it does nothing to address the server issue which is what this topic specifically addresses. Beamdog has already stated their lack of interest in hosting and distributing files like the vault, and ppl can geek out over bit torrent or steam if they want but you still need to go somewhere to find the haks and overrides you want. Presumably you find the haks and the modules in the same place for example on the neverwintervault site, which is atrocious to browse, use google on it instead.
For a pw the demands are entirely different. You don't get a copy of the module, the server list is just a matching service and the module exists on a private server. The server builders have already selected and packaged the content over many collective years and in order to play you need an exact match or you risk problems for the player. The content is already non negotiable and the auto-downloader simply removes the excessive steps needed to start playing.
"Presumably you find the haks and the modules in the same place for example on the neverwintervault site, which is atrocious to browse, use google on it instead. "
Exactly why I have never choosen to add my Realms of Annakolia haks to the vault - and have added the CEP 2.4 download to my private site for download. The issue there is, sending people to get CEP 2.4 to the NWvault cause confusion - what do I download and where is it and - even with a direct link - continuing issues.
"The server builders have already selected and packaged the content over many collective years and in order to play you need an exact match or you risk problems for the player. The content is already non negotiable"
Exactly - the EXACT content is not negotiable - Realms of Annakolia has been built since first release (15 years now isn't it?) - some of our content is custom made for us and exists nowhere else.
Content MUST be exactly matching
This is why we are using CEP 2.4 and NOT supposed updated by TAD - they are NOT backward compatible and DO cause trouble - something the CEP is NOT supposed to do.
I can foresee much trouble with a lot of schemes proposed here - exact constant must be delivered and packaged as the module requires or the module is broken.
Comments
For security type script files, I guess those could land in your server override, no? We have enough clutter in our server HAK I wouldn't bother dumping it override if my only other option was to have players auto-update; that said, you riase some other good points about conflicts of versions of files between servers. Good times.
-Dave
Actually here is the official tutorial for it.
https://neverwintervault.org/rolovault/projects/nwn2/nwn2tutorials/118/NWN2_Autodownloader_Toolset_User_Guide.pdf
Personally I think the ability for server host to link to an external ftp server with the haks and auto download and unpack them would be the best thing. Either an FTP server the host setup themselves or to a place like NWNvault or some such that already exists for them to upload their content to for download.
When the client connects to a new world without the required haks/tlk, it could give them an option to open the Steam Workshop page/collection for the module. From there they can subscribe to and download all of the required haks and tlk.
The benefit of allowing Steam to auto-update subscribed players with the custom/unique hakpacks of their game worlds would massively open up the ability to modify, create, share, and utilize custom content.
- Jamie
The way I see it - we would need to
1. Get a list of files from the server that need to be present on the client.
2. The checksum/hash of the file needs to match the one specified by the server.
3. If it does not match - re-download it
4. If it does match - skip to next file
One technology I have started getting familiar with is the IPFS (Interplanetary File System) - which is based on the DHT technology that bittorrent is built on.
(Someone above mentioned bittorrent)
Pretty much anyone can choose to run a IPFS Daemon, and donate say 1GB of space to the network.
Files can be added to the IPFS network via that daemon, and they only get distributed across the network, when they are requested by others on the network.
Files are uniquely identified on the network by their contents (hash) - which prevents duplication of any file on the network (makes it very efficient).
And content is shared P2P which means that no one person is having their bandwidth drained by sharing 3GB of haks with the wider world.
Now putting something simple into practice for nwnserver and nwmain.
nwserver would need to send the hak / tlk list - it already does this, but perhaps in addition to the filenames, we send the hashes of the files too or maybe the filenames are the hashes?
If the filenames were hashes of the file content, this would alleviate the issue of other servers having the same duplicate content etc.
It would mean that the filename is based on the content, and it would only exist or download once - and once the client has it, it never needs to download again.
In anycase- this download issue is easy to solve outside of the game itself, perhaps an addition to the nwn launcher would be an easier solution?
You add your PWServer to the launcher, it checks on startup with the server the hashes of server content vs local content and downloads the appropriate files.
The question then becomes - does this downloader / verification function go into the server executable itself, or into a separate executable.
In my mind - it would be easier to add it into a separate executable, and I don't think the community would mind as long as the functionality is somehow provided.
However, no one should assume that ideas discussed here would be acceptable to SP authors and players. That would require a new thread, with an open invitation to the whole community.
Arguably, the discussion ought to be more open anyway, since a solution adopted for PW might create player expectations, pressurizing SP authors to follow suit.
SP authors are likely to have views on
- Version control at the game object level (not merely at hak level)
- Avoiding commercial download sites
- DRM/lurkware-free solutions
amongst other things.If we are looking for a solution that is avoiding commercial sites, or any sites that are centralized in general - then a technology such as IPFS could be perfect for that.
Lets imagine someone has decided what their 'content' list looks like, they have then got it arranged in a JSON format - or some other serialized format.
That file list can then be added to the IPFS network, which generates a unique hash - which essentially becomes the 'address' of the file on the network.
Sharing the file list becomes a simple case of sharing the unique identifying hash.
A hash looks like this on IPFS: QmPjAZoG5fmCtkt7aA8q5wc13nRf37VUwkKJHEaSyYQpNi
For example - you can see what this hash maps to by going to the ipfs.io website.
https://ipfs.io/ipfs/QmPjAZoG5fmCtkt7aA8q5wc13nRf37VUwkKJHEaSyYQpNi
This is an example fileList - you might be able to tell from this filelist that I have added the html files from the 1.69 Lexicon.chm file to the IPFS network as a POC.
Each of those hashes in the file entries can be used to access / pull that resource from the IPFS network.
It takes a bit of effort to get it working for a website, but it works perfectly for general file storage.
I digress...
In any case- that file list would then contain the hashes of other files on the IPFS network.
(Your haks, your itm files, your tlk files - anything...)
The SP or MP player would then just need to feed that FileList into their client or downloader application to retrieve those files from the IPFS network.
Certain disclaimers need to be understood though - anything added to the IPFS is immutable.
Its basically uncensorable - once something is on the network, its there for as long as anyone wants it.
Content does get cleared from the network eventually (theoretically) - if the content is never requested by anyone, then the content never spreads across the network. (Its self cleaning essentially).
Everyone who runs a 'node/daemon' for IPFS, allocates a fixed amount of space to the storage of said files.
More popular/requested files will take priority over files that have not been requested in a while.
So that file that no one wants will eventually slip into obscurity and be cleaned from the network.
(Requesting the files regularly or hosting a node of your own is a good way to ensure your content always remains online)
It also prevents duplication of content - because each file only exists on the network once. (The system will not allow two duplicate files to be added, even if they have different names - its done purely based on file content)
The feature is about pure convenience and lowering the bar to entry for players to use servers that use hak content. Even the best of us can be lazy sometimes and we just want to try something out without even having to type a name or keyword into a search bar, we want to click and enter the world.
Syncing haks with steam might be nice for some folks that want to use that platform to manage their content and for creators that want to share their haks there. It should be an optional and separate issue from providing the most convenient and access friendly method possible.
Providing your own installer is something some already do, it's fine too, some provide installers, megazips, and individual zips all as options. Even in NWN2 where there is a functional auto-downloader some servers provide their haks as zips or from dropbox so there is always another option should someone have problems.
I think the main difference with NWN and NWN2 pw communities from a technical standpoint right now is that on NWN2 I'm pretty sure no server exists without their own haks.
However - like Proleric said - there are two aspects of this.
What about Single Player situations, where they want to download Haks / Resources for the single player module.
While a seamless 'Click on server, get files downloaded' approach does solve the Multiplayer aspect, it does not solve the single player situation.
I think 'content' needs to be easily install-able, regardless of single or multiplayer scenarios.
The issue I foresee is that if the solution is designed for a Multiplayer scenario, it results in a Single Player solution having to be built.
However, if you make a system that is totally agnostic of Single or Multiplayer, then it is less dev time for the developers to implement / build into the base game.
Eg: They don't have to cater to Single or Multi players - they just cater to 'users'.
Unless you're proposing a change to the above sequence (like an in-game vault browser?), it is safe to assume that the average player of custom SP mods is capable and comfortable performing those steps.
If a SP module uses custom haks, the additional step is.. well, none. The packed module would have mod/, hak/, tlk/, mus/ and similar folders in the archive. Player unpacks it into their NWN userdir and they're all set.
If the SP module uses shared haks (like CEP, Q, etc), it's a bit more involved. They are linked to the master hak page, where they repeat the process. They'd need to manually check (or remember) if they already have this package, but that's hardly a huge hassle.
It seems to me the biggest pain point when it comes to single-player modules is their discoverability. Once a player has found a module they want to play, the process is pretty straightforward. But finding the modules requires leaving the game, going to a third party site and learning how to use it.
This is a perfectly valid concern that could be addressed, but I think it is barely related to the feature requested here.
FWIW, there are installer-makers available that package all the custom content into a standalone .exe, and can check if the shared content is present, downloading it only if not. Module makers could then use these to ship their modules, and players would only need to run the installer. This has been the case for the last 10+ years, but hardly any modules used this (there are exceptions, like DoD). Not sure what conclusion is to be drawn from that fact..
Having a cleaner and more friendly way of downloading single player content might be beneficial but it does nothing to address the server issue which is what this topic specifically addresses. Beamdog has already stated their lack of interest in hosting and distributing files like the vault, and ppl can geek out over bit torrent or steam if they want but you still need to go somewhere to find the haks and overrides you want. Presumably you find the haks and the modules in the same place for example on the neverwintervault site, which is atrocious to browse, use google on it instead.
For a pw the demands are entirely different. You don't get a copy of the module, the server list is just a matching service and the module exists on a private server. The server builders have already selected and packaged the content over many collective years and in order to play you need an exact match or you risk problems for the player. The content is already non negotiable and the auto-downloader simply removes the excessive steps needed to start playing.
I still believe Steam allows many of these problems to be solved for the majority of non-PW cases. Using the Steam workshop requires NWN:EE have Steam integration anyway, so it's not an external utility or even in a separate window from the game, and virtually everybody already has experience using Steam.
Steam downloading is agnostic of singleplayer or multiplayer modules. When you don't have the haks to connect or open the module it can send you to the appropriate page, it doesn't matter if it's a persistent world or not.
From what I recall (somebody at) BD already wants some degree of Steam integration anyway, which is why I suggest this. Steam Workshop integration is far from my preferred solution, especially as a persistent world builder myself, believe me; I'd much prefer a more robust and extensible approach that doesn't rely on Valve or any kind of mandated central storage servers. But for what I imagine is the vast majority of singleplayer, LAN, and persistent world use cases the Steam Workshop should be more than sufficient; and where a PW has different needs or concerns incompatible with using the Steam Workshop, they can still use external loaders/launchers and whatnot.
Steam and other external services are a separate issue to the multiplayer auto-downloader.
In that case why not make something which works with the nexus (I know some have issue with them) where files are automatically downloaded from there?
I would imagine that BeamDog - if they provide the means for Automated / Seamless downloading of haks / tlk at server entrance - would give the server admin the ability to specify the path to the files in question.
Be they on Vault or on Nexus, or on the server itself.
not sure what the solution for a Single Player scenario would be though.
Eg: try to spin up a module, it doesnt work because you need the haks and tlks.
Perhaps better reporting in such a situation.
I think solving the MP issue would be a huge accomplishment - At the moment communities on PW's are contained in silos - it is hard to play on some servers because their haks and tlks are vastly different from others.
If an auto download system was in place, that kept the resources automatically segregated per PW - that would help.
That being said - if the filenames of the haks and tlks were hashes - then no segregation would be needed.
Im not sure if the filename limit has increased or decreased beyond its previous size limit.
Look how diverse and unique the Linux ecosystem is. How fantastic are OpenOffice, Wikipedia, MySQL, Firefox, WordPress, Apache... Android.
Being Open Source could benefit your world significantly more by providing you a brand new wealth of code. To learn tricks and workarounds you had never thought of and contribute in kind to that vast pool of knowledge. How much information. How much history have we lost over the past 15 years because our walled kingdoms collapsed and their is no recovering them?
It helps I am not a fan of boutique modding but if the worst thing that happens is we are all forced to go open source, everyone makes better worlds, and NWN:EE finds a larger long term fanbase when relaunched then I am kinda ok with you knowing how better my fishing script is than all of yours.
Oh, and people can connect to our worlds easier.
All servers should not need to put scripts in haks either way whether it's an amazing script or a one line thing from a script generator.
...Just sayin'
Exactly why I have never choosen to add my Realms of Annakolia haks to the vault - and have added the CEP 2.4 download to my private site for download. The issue there is, sending people to get CEP 2.4 to the NWvault cause confusion - what do I download and where is it and - even with a direct link - continuing issues.
"The server builders have already selected and packaged the content over many collective years and in order to play you need an exact match or you risk problems for the player. The content is already non negotiable"
Exactly - the EXACT content is not negotiable - Realms of Annakolia has been built since first release (15 years now isn't it?) - some of our content is custom made for us and exists nowhere else.
Content MUST be exactly matching
This is why we are using CEP 2.4 and NOT supposed updated by TAD - they are NOT backward compatible and DO cause trouble - something the CEP is NOT supposed to do.
I can foresee much trouble with a lot of schemes proposed here - exact constant must be delivered and packaged as the module requires or the module is broken.