I'm thinking more along the lines of "original content do no steal" memes rather than philosophical debate of proprietary vs open source vs free software.
If this were an Open Source kind of discussion we could have all three on separate but connected branches.
...Just sayin'
I know I am not the only one who does not distribute scripts, and some other custom content. I have no desire to provide this material to open source. I am not negotiable on this point.
Other content I have in Haks I do not wish to distribute - Areas - yes areas are 3 container resources count for 1 area - keeping a module container lower in content make it faster and keeps from going over limit.
Conversations - yes i have some conversations in haks as well that I do not want to distribute.
There are a few other odds and ends that are normally in a module that i have removed to haks for the above reason - module max container limit.
So you want all files in 7z / rar compression file that read straight out of it and encrypted(only NWN has access too) so player cannot open it up and then when it uncheck(with custom UI in-game list or steam mod file) deleted?
So you want all files in 7z / rar compression file that read straight out of it and encrypted(only NWN has access too) so player cannot open it up and then when it uncheck(with custom UI in-game list or steam mod file) deleted?
Only after you sign an NDA and scan your pinkie for the legally binding pinkie promise.
There are a few other odds and ends that are normally in a module that i have removed to haks for the above reason - module max container limit.
So no container limit, no problem and everyone else can have an easy means of distributing content to the rest of the active community.
Windows forces a container limit - this is why there are many CEP haks instead of one or two - even if the container limit is expanded there is STILL a container limit enforced by Windows
What!? There's a 4GB file limit on FAT32 systems, which I don't think anyone uses? And there's a 4 billion files limit in the ERF, which you're not going to hit. What other limit is there @Barry_1066 ?
What!? There's a 4GB file limit on FAT32 systems, which I don't think anyone uses? And there's a 4 billion files limit in the ERF, which you're not going to hit. What other limit is there @Barry_1066 ?
When developing the CEP we ran into a container wall - I contacted Bioware for details and was told it was a windows limitation. This was talking about hak limitations - for clarity - and when we exceeded a certain point the haks no longer worked.
I had also experienced OS failure with the files for an overfull hak no longer being accessible in a windows folder (files removed from the hak and in the folder). True I am talking about 32 bit WinXP experience but unless the game is not going to work on 32 bit machines with older OS systems, haks will have to work under those perimeters unless Beamdog makes some changes not ONLY to the module container limit with the stand-alone server but with hak limits as well. Win7 to Win10 supported but 32 bit as well as 64.
This is hard lessons learned by years of CEP Team management - and we all (the CEP Team) struggled with these limitations and received guidance from Bioware as well.
Unless something has drastically changed, these limitations are still in place.
There's no practical limit on the combined sizes of all the files in a folder, though there may be limits on the number of files in a folder. More importantly, there are limits on individual file size that depend on what filesystem you're using on your hard disk. (The "filesystem" is nothing more than the specification of exactly how files are store on disk.)
Let's break this down by file system: FAT aka FAT16
FAT, for File Allocation Table, is the successor to the original FAT12 filesystem that shipped with MS-DOS many, many years ago.
Maximum disk size: 4 gigabytes Maximum file size: 4 gigabytes Maximum number of files on disk: 65,517 Maximum number of files in a single folder: 512 (if I recall correctly, the root folder "/" had a lower limit of 128).
FAT32
"There's no practical limit on the combined sizes of all the files in a folder, though there may be limits on the number of files in a folder."FAT32 was introduced to overcome some of the limitations of FAT16.
Maximum disk size: 2 terabytes Maximum file size: 4 gigabytes Maximum number of files on disk: 268,435,437 Maximum number of files in a single folder: 65,534
NTFS, or "New Technology File System" introduced with Windows NT, is a completely redesigned file system.
Maximum disk size: 256 terabytes Maximum file size: 256 terabytes Maximum number of files on disk: 4,294,967,295 Maximum number of files in a single folder: 4,294,967,295
Note that when I say "disk" above, I'm really talking about "logical" disks, not necessarily physical. No one makes a 256 terabyte disk drive, but using NTFS you can treat an array of disk drives as a single logical disk. Presumably if you have enough of them, you can build a huge logical drive.
Also note that the NTFS's 256 terabyte limitation may well simply be an implementation restriction - I've read that the NTFS format can support disks up to 16 exabytes (16 times 1,152,921,504,606,846,976 bytes).
Yes if from Windows7 on everyone is using NTFS - and NWN EE updated the container limitations so the server runs with extended room, then much change can be make - however to remain backward compatible to all older modules, the CEP would need to have dummy haks with the required names if all, for instance, core haks were combined into one core hak.
Under FAT32: Maximum number of files in a single folder: 65,534 - we met that limit working with CEP - I know win7 onward should be NTFS but some external drives (maybe just older ones but made for win7) use FAT32.
It is too early to know what is being done concerning this and I am not rushing to change things until I get word that these changes have been made - even then I will test before doing any such changes.
Under FAT32: Maximum number of files in a single folder: 65,534 - we met that limit working with CEP - I know win7 onward should be NTFS but some external drives (maybe just older ones but made for win7) use FAT32.
This is true, but the only thing it is limiting is your workflow. With FAT32, you would not be able to put all files in one folder and pack them into an ERF/HAK. But you could split it in two, and add one at a time. Once packed, the players would never unpack it, and thus wouldn't hit this even on FAT32.
This isn't t a limitation that should be fixed. If you're hitting it, consider changing your workflow.
Yes if from Windows7 on everyone is using NTFS - and NWN EE updated the container limitations so the server runs with extended room, then much change can be make - however to remain backward compatible to all older modules, the CEP would need to have dummy haks with the required names if all, for instance, core haks were combined into one core hak.
Under FAT32: Maximum number of files in a single folder: 65,534 - we met that limit working with CEP - I know win7 onward should be NTFS but some external drives (maybe just older ones but made for win7) use FAT32.
Something has been bothering me about this and I just figured out what that is...
The practical lifespan of a mechanical hard drive is 3-5 years before it begins to develop faults. 4-6 for Enterprise drives.
So this is an argument for a theoretical hard drive that could have failed 5 times over in the lifespan you are talking about. Possibly with a processor over the 12 year life span and a motherboard with it's capacitors replaced. Being argued in defense for the future direction of a game that requires a Windows 7 in it's minimum specs.
My experience with NWN:EE should not be lessened because someone else didn't find their data important enough to move to a newer file system (using one of a hundred ways to do so) and left their hardware to rot for over a decade.
If I am to be frank, sod that imaginary person in that made up scenario. If they want to update to the Enhanced Edition they should get a new hard drive that isn't full of faults and format it to a file system from this decade. It costs all of $40 and there are numerous free file managers and cloud solutions to aid in transferring that data.
Yes, even on Windows XP.
And maintaining 100% backwards compatibility isn't going to happen if the game is to move forward. With the removal of bink NWN:EE is already incompatible with far more single player content than your proposed fringe scenario of "Luddite with phobia of copy/paste".
We have to promote this card. The lack of this feature is preventing players to play multiplayer.
We know its a beta, but if the nwn golden gooze is the custom multiplayer content. We are not opening doors right now for newcomers to see it.
Old players know the trick. Search server forums. Get the downloads. Even knowing that. Is tedious steps than a simply connect to a server and wait for content being downloaded.
For those nwn ee pw up there. Just check how many players have tried to connect your server and how many finally achieved it. I've seen in my server nwplayer.ini many accounts trying to connect.
I think people who own EE want to play in EE. But there's no approach to ease the multiplayer experience.
We have to promote this card. The lack of this feature is preventing players to play multiplayer.
We know its a beta, but if the nwn golden gooze is the custom multiplayer content. We are not opening doors right now for newcomers to see it.
Old players know the trick. Search server forums. Get the downloads. Even knowing that. Is tedious steps than a simply connect to a server and wait for content being downloaded.
For those nwn ee pw up there. Just check how many players have tried to connect your server and how many finally achieved it. I've seen in my server nwplayer.ini many accounts trying to connect.
I think people who own EE want to play in EE. But there's no approach to ease the multiplayer experience.
The solution and in the NWN2 multiplayer content download system there is such a system and it works quite well, but this has been repeated many times with some slight alternative and citation to other systems and games, which exploit a similar system...
Yes if from Windows7 on everyone is using NTFS - and NWN EE updated the container limitations so the server runs with extended room, then much change can be make - however to remain backward compatible to all older modules, the CEP would need to have dummy haks with the required names if all, for instance, core haks were combined into one core hak.
Under FAT32: Maximum number of files in a single folder: 65,534 - we met that limit working with CEP - I know win7 onward should be NTFS but some external drives (maybe just older ones but made for win7) use FAT32.
Something has been bothering me about this and I just figured out what that is...
The practical lifespan of a mechanical hard drive is 3-5 years before it begins to develop faults. 4-6 for Enterprise drives.
So this is an argument for a theoretical hard drive that could have failed 5 times over in the lifespan you are talking about. Possibly with a processor over the 12 year life span and a motherboard with it's capacitors replaced. Being argued in defense for the future direction of a game that requires a Windows 7 in it's minimum specs.
My experience with NWN:EE should not be lessened because someone else didn't find their data important enough to move to a newer file system (using one of a hundred ways to do so) and left their hardware to rot for over a decade.
If I am to be frank, sod that imaginary person in that made up scenario. If they want to update to the Enhanced Edition they should get a new hard drive that isn't full of faults and format it to a file system from this decade. It costs all of $40 and there are numerous free file managers and cloud solutions to aid in transferring that data.
Yes, even on Windows XP.
And maintaining 100% backwards compatibility isn't going to happen if the game is to move forward. With the removal of bink NWN:EE is already incompatible with far more single player content than your proposed fringe scenario of "Luddite with phobia of copy/paste".
Not that this is an argument against what you have said - and I agree with you. I do want to point you that there is still FAT 32 use in modern day machines - The problem exists in external duel platform use between Mac and Windows machines. Mac external (for windows compatibility - the new format is not windows compatible) is still FAT32.
Since installing the game on an external drive is not the best option - yet it IS an option - the there is that ONE limitation is this modern age of Mac not keeping up with the rest of us.
Anyway - in principle I agree with you - I would also point out I have a 10 year old external HD that is rarely plugged in - it is backup - and uses FAT32 - so such a drive has a far longer life span than sited - but that is immaterial for our basic purposes - just saying, they exist and are just fine.
I'm not saying that fringe case can't exist, just that it is a very specific issue that will be encountered by so few people that it does not seem like a good reason to hold back the other 99.99% of the userbase.
I'm not saying that fringe case can't exist, just that it is a very specific issue that will be encountered by so few people that it does not seem like a good reason to hold back the other 99.99% of the userbase.
Given that we are invalidating previously valid operating systems (ala Windows XP), I don't see why invalidating old and defunct hardware configurations would be off the table either. If you've got a FAT32 drive, you can always move the files off to something else.
I'm not saying that fringe case can't exist, just that it is a very specific issue that will be encountered by so few people that it does not seem like a good reason to hold back the other 99.99% of the userbase.
Given that we are invalidating previously valid operating systems (ala Windows XP), I don't see why invalidating old and defunct hardware configurations would be off the table either. If you've got a FAT32 drive, you can always move the files off to something else.
Yes but Mac is a current NWN EE supported system - and, as i pointed out bi-platform external drives for Mac/Windows use, are FAT32 now, and, unless Apple makes some major change, they will continue to be FAT32.
This fact, until very recently with the release of a new (for Apple ONLY) external drive file format, restricts Mac users to the single internal drive. I don't have a problem with that but then I am not using a Mac. I have an iMac and I know an external drive would be needed if I was using the computer as most people do. Just pointing out that all situations (IN modern day, if purchased off the shelf NOW) must be looked at since Mac IS a supported platform for NWN EE.
With the new Mac external drive format this is no longer an issue - assuming that NWN EE runs from Mac with that new format and cross platform for the drive is not being needed - since cross platform is still FAT32.
I have a suggestion here. It could perhaps be a bit OTT.
Could we use Linux package management as a source of inspiration for this? This approach is used a lot by programmers to help ensure that 3rd party code you depend on is up to date and everything is compatible with everything else.
The general way they work is that you, as the programmer, specify something your code depends on and what versions you support (usually something * for any, or 3.4.* etc.). When you fling your dependencies at the package manager, it downloads them from its list of available sources. After that, it checks the dependencies of the things it just downloaded and then downloads them. It repeats this until there's nothing left to download.
There's a bit more to it that the description above, but that's the general idea.
In practice they also do things like work out what's the best version of something to use in the case that two of your dependencies also depend on the same thing. If you have two packages that depend on "my_bestest_nwn_hak", but one package supports 3.4.*. and the other supports 3.*, it will pick the latest version which satisfies both packages.
The key advantages I see to using a similar approach would be:
Redundancy - you can have multiple sources to get files from. No repeat of nwvault.ign going down.
Versioning - you don't have to wrangle multiple versions for different servers, the package manager handles that.
Bandwidth - the game server doesn't have to splurge bandwidth that could be better used for playing the game in order to supply hak/mod files to random people.
https://www.linux.com/news/apt-get-primer introduction to Apt-get, the package manager used to install programs/libraries in Debian (and its derivatives like Mint and Ubuntu) Linux distributions.
I haven't read this thread properly, just skimmed it and done a few ctrl+f searches to see if this suggestion has been raised already. While "package", "manager" and "linux" did come up, skim reading for context seemed to suggest they weren't talking about my suggestion. Sorry if this has been suggested before.
My preference is to just get us a simple but functional download-on-connect system for the initial release. You can work on optimising the content duplication and other issues later.
I don't have anything substantive to add in terms of 'how' to get it done, but I wanted to throw my support behind the idea nonetheless. Apologies if I'm restating the obvious - the trello board says that more discussion is needed.
Custom content is one of the reasons why NWN is so great. However, the cumbersome method of having to pre-install content made having custom content a disincentive for people to try new persistent worlds, since they couldn't just hop on and play.
So, worlds can offer great content and rich worlds, but they will generally have fewer players because there are no 'drive by' drop-in players. People have to go to some amount of effort to get the content before they can even try the world. Then, worlds are generally reluctant to add or change their content frequently, since that puts their playerbase at risk every time a change is made (since all players would then have to update their haks).
By having haks auto-loaded in some form or fashion opens the door for all worlds to use custom content and continuing to add to their content, which in turn, fuels the development of more custom content, which pushes NWN to remain current for a longer period of time.
We've needed this functionality for a long time. I sincerely hope this feature is added to the development roadmap.
I asked in the last week twitch stream and was disappointed by the answer.
this feature seems to be in the long road. As I see this should be implemented as soon as possible. We dont need the perfwcr solution. Its a head start / beta.
The master server was a step in the right direction. But its only a index of names where you cant join any game if it requires custom content.
End users/players participating in the head start are precisely the kind of users that should expect some form of "handholding" in the form of having to pre-download content to experience what EE is like on these servers. That's the caveat with early access/beta programs. It's not meant to be seamlessly accessible from the get-go, and I would rather see Beamdog focus on other things in their current to-dos than something that, while I agree should be something definitely implemented pre-release, is more of a handy benefit than a mandatory/server-will-die-without-it feature in EE's current state.
As I brought up earlier, the addition of material (most importantly custom scripts) that do not need to be delivered to players and are in haks to reduce the module load would be a problem to have moved to players.
My questions: 1) An ability to designate to send an empty hak for the player.
I suppose the next point is not the same kind of question so not exactly number 2: I have custom versions of all the spells - I have a spell mastery system that players gain levels 1 - 100 based on a die roll every time they cast a spell. This system required some modification to every spell script yet the name of the script is the same as everyone (from Bioware/Beamdog distribution).
The problem would occur if our versions were distributed - which I do not wish at any rate - since the name of the script is identical to the stock script. Would a distribution system just look at the name of a script or would it look at the file length to determine a difference? This not only applies to scripts but to any material - there are different versions of tilesets and yet the tile names are the same - and imposing a change on a server by getting the wrong material with the same name would cause a lot of problems.
Anyway I am in favor of some sort of distribution system - I am concerned about the bandwidth issue with pulling the material from the server. Auto install is really needed however - with the difficulty and issues i have been constantly dealing with for players trying to simply install haks and tlk. The new location of the Neverwinter Nights directory under Documents, along with the extraction issues from a rar or 7zip are just too technical for some players. The most common mistake is extracting from a rar or 7zip and having everything in a folder that they drop in the hak folder thus having the files in the wrong place.
So yes some method that will auto install - I use to use the auto install system that Bioware sent but that is windows only and of course will not work now - and was a bit flaky then due to size and amount of files and not using the best of compression. That file (nwcontbuild) needs a Mac version - I am guessing linux would not auto install but if so, then a linux version as well.
None the less, an update of that file (nwcontbuild), using better tight compression that would execute and install all - portraits, music, haks, and tlk would be great. An auto download from the web location set in nwnplayer.ini - so that would relieve the server bandwidth issue, drawing the files from off server site locations (I have my own site but others could use NWvault or?).
I was just wondering if this (development thread) goes anyway to satisfying the needs of pws in this regard. I am not personally involved in any way with this BTW, just been watching it develop with interest.
On this subject, a thought on something that the community could do to make life better for everyone with this as well.
Currently how modules and haks are structured is to have one module (sometimes more if it's a story campaign rather than a PW) which is usually in the 10-100 mb range and a few hakpaks that are sometimes several hundred mb in size.
It's currently done this way because all the hakpaks are downloaded from the nwn vault and it's annoying to have to download and install lots of haks individually. If the downloading is automatically handled by the game client, then that removes this irritation.
In such a world of auto-downloading, it would make much more sense to have *lots* of very small haks so as to not waste bandwidth and hdd space. It's really wasteful and dumb having to download a new 100mb hak because someone added a new creature model.
The smaller we can make hak files, the more likely they are to be reused across servers.
Two things - I have no affiliation to this project whatsoever. I do not know if it is any use for pw players, which is why I keep asking people to look at it and report back.
In my opinion, creating a launcher with hak selection would be a considerable option. You could have there a server list which allows you to connect with haks required by the server. If you don't have required files the steam workshop would handle missing files. If content is not present on workshop.. then you have to add them manually. Just like in Arma 3. Add a key check for every hak and you probably will have a working system in which launcher tells the game what files need to be loaded.
There needs to be something, it's so painfully tedious to track down what files are needed by a PW and mess around putting them in the right place.
Quake 2 had auto downloading of content, it shouldn't be beyond the realms of man and science to get it implemented in EE. Even something like the NWN2 version would do for now.
Ideally allow multiplayer servers to prep and assign download location with server info along side to read (or possibly check for faster downloading links), allow single player modules to link to locations after a check for its required haks.
Comments
Other content I have in Haks I do not wish to distribute - Areas - yes areas are 3 container resources count for 1 area - keeping a module container lower in content make it faster and keeps from going over limit.
Conversations - yes i have some conversations in haks as well that I do not want to distribute.
There are a few other odds and ends that are normally in a module that i have removed to haks for the above reason - module max container limit.
I had also experienced OS failure with the files for an overfull hak no longer being accessible in a windows folder (files removed from the hak and in the folder). True I am talking about 32 bit WinXP experience but unless the game is not going to work on 32 bit machines with older OS systems, haks will have to work under those perimeters unless Beamdog makes some changes not ONLY to the module container limit with the stand-alone server but with hak limits as well. Win7 to Win10 supported but 32 bit as well as 64.
This is hard lessons learned by years of CEP Team management - and we all (the CEP Team) struggled with these limitations and received guidance from Bioware as well.
Unless something has drastically changed, these limitations are still in place.
At this time Neverwinter Nights: Enhanced Edition is only available on Windows 7 and up.
You will forgive me friend but I am going to go with Beamdog on the subject of what they can and can't do to the engine. They have the source code.
Perhaps if Neverwinter was open source we could have fixed the issue ourselves 12 years ago.
See what I did there?
- Jamie
Let's break this down by file system:
FAT aka FAT16
FAT, for File Allocation Table, is the successor to the original FAT12 filesystem that shipped with MS-DOS many, many years ago.
Maximum disk size: 4 gigabytes
Maximum file size: 4 gigabytes
Maximum number of files on disk: 65,517
Maximum number of files in a single folder: 512 (if I recall correctly, the root folder "/" had a lower limit of 128).
FAT32
"There's no practical limit on the combined sizes of all the files in a folder, though there may be limits on the number of files in a folder."FAT32 was introduced to overcome some of the limitations of FAT16.
Maximum disk size: 2 terabytes
Maximum file size: 4 gigabytes
Maximum number of files on disk: 268,435,437
Maximum number of files in a single folder: 65,534
NTFS, or "New Technology File System" introduced with Windows NT, is a completely redesigned file system.
Maximum disk size: 256 terabytes
Maximum file size: 256 terabytes
Maximum number of files on disk: 4,294,967,295
Maximum number of files in a single folder: 4,294,967,295
Note that when I say "disk" above, I'm really talking about "logical" disks, not necessarily physical. No one makes a 256 terabyte disk drive, but using NTFS you can treat an array of disk drives as a single logical disk. Presumably if you have enough of them, you can build a huge logical drive.
Also note that the NTFS's 256 terabyte limitation may well simply be an implementation restriction - I've read that the NTFS format can support disks up to 16 exabytes (16 times 1,152,921,504,606,846,976 bytes).
from: https://stackoverflow.com/questions/4944709/windows-limit-on-no-of-files-in-a-particular-folder
Yes if from Windows7 on everyone is using NTFS - and NWN EE updated the container limitations so the server runs with extended room, then much change can be make - however to remain backward compatible to all older modules, the CEP would need to have dummy haks with the required names if all, for instance, core haks were combined into one core hak.
Under FAT32: Maximum number of files in a single folder: 65,534 - we met that limit working with CEP - I know win7 onward should be NTFS but some external drives (maybe just older ones but made for win7) use FAT32.
It is too early to know what is being done concerning this and I am not rushing to change things until I get word that these changes have been made - even then I will test before doing any such changes.
This isn't t a limitation that should be fixed. If you're hitting it, consider changing your workflow.
The practical lifespan of a mechanical hard drive is 3-5 years before it begins to develop faults. 4-6 for Enterprise drives.
So this is an argument for a theoretical hard drive that could have failed 5 times over in the lifespan you are talking about. Possibly with a processor over the 12 year life span and a motherboard with it's capacitors replaced. Being argued in defense for the future direction of a game that requires a Windows 7 in it's minimum specs.
My experience with NWN:EE should not be lessened because someone else didn't find their data important enough to move to a newer file system (using one of a hundred ways to do so) and left their hardware to rot for over a decade.
If I am to be frank, sod that imaginary person in that made up scenario. If they want to update to the Enhanced Edition they should get a new hard drive that isn't full of faults and format it to a file system from this decade. It costs all of $40 and there are numerous free file managers and cloud solutions to aid in transferring that data.
Yes, even on Windows XP.
And maintaining 100% backwards compatibility isn't going to happen if the game is to move forward. With the removal of bink NWN:EE is already incompatible with far more single player content than your proposed fringe scenario of "Luddite with phobia of copy/paste".
We know its a beta, but if the nwn golden gooze is the custom multiplayer content. We are not opening doors right now for newcomers to see it.
Old players know the trick. Search server forums. Get the downloads. Even knowing that. Is tedious steps than a simply connect to a server and wait for content being downloaded.
For those nwn ee pw up there. Just check how many players have tried to connect your server and how many finally achieved it.
I've seen in my server nwplayer.ini many accounts trying to connect.
I think people who own EE want to play in EE. But there's no approach to ease the multiplayer experience.
Since installing the game on an external drive is not the best option - yet it IS an option - the there is that ONE limitation is this modern age of Mac not keeping up with the rest of us.
Anyway - in principle I agree with you - I would also point out I have a 10 year old external HD that is rarely plugged in - it is backup - and uses FAT32 - so such a drive has a far longer life span than sited - but that is immaterial for our basic purposes - just saying, they exist and are just fine.
I'm not saying that fringe case can't exist, just that it is a very specific issue that will be encountered by so few people that it does not seem like a good reason to hold back the other 99.99% of the userbase.
This fact, until very recently with the release of a new (for Apple ONLY) external drive file format, restricts Mac users to the single internal drive. I don't have a problem with that but then I am not using a Mac. I have an iMac and I know an external drive would be needed if I was using the computer as most people do. Just pointing out that all situations (IN modern day, if purchased off the shelf NOW) must be looked at since Mac IS a supported platform for NWN EE.
With the new Mac external drive format this is no longer an issue - assuming that NWN EE runs from Mac with that new format and cross platform for the drive is not being needed - since cross platform is still FAT32.
Could we use Linux package management as a source of inspiration for this? This approach is used a lot by programmers to help ensure that 3rd party code you depend on is up to date and everything is compatible with everything else.
The general way they work is that you, as the programmer, specify something your code depends on and what versions you support (usually something * for any, or 3.4.* etc.). When you fling your dependencies at the package manager, it downloads them from its list of available sources. After that, it checks the dependencies of the things it just downloaded and then downloads them. It repeats this until there's nothing left to download.
There's a bit more to it that the description above, but that's the general idea.
In practice they also do things like work out what's the best version of something to use in the case that two of your dependencies also depend on the same thing. If you have two packages that depend on "my_bestest_nwn_hak", but one package supports 3.4.*. and the other supports 3.*, it will pick the latest version which satisfies both packages.
The key advantages I see to using a similar approach would be:
- Redundancy - you can have multiple sources to get files from. No repeat of nwvault.ign going down.
- Versioning - you don't have to wrangle multiple versions for different servers, the package manager handles that.
- Bandwidth - the game server doesn't have to splurge bandwidth that could be better used for playing the game in order to supply hak/mod files to random people.
See:- https://en.wikipedia.org/wiki/Package_manager for more information.
- https://en.wikipedia.org/wiki/Dependency_graph - general description of the class of algorithms package managers use to get the correct version
- https://www.linux.com/news/apt-get-primer introduction to Apt-get, the package manager used to install programs/libraries in Debian (and its derivatives like Mint and Ubuntu) Linux distributions.
- https://docs.npmjs.com/getting-started/using-a-package.json#manually-editing-your-packagejson - See the grey box below this paragraph for an example of a package with dependencies in the NPM package manager for Javascript/NodeJS.
P.S:I haven't read this thread properly, just skimmed it and done a few ctrl+f searches to see if this suggestion has been raised already. While "package", "manager" and "linux" did come up, skim reading for context seemed to suggest they weren't talking about my suggestion. Sorry if this has been suggested before.
You can work on optimising the content duplication and other issues later.
Custom content is one of the reasons why NWN is so great. However, the cumbersome method of having to pre-install content made having custom content a disincentive for people to try new persistent worlds, since they couldn't just hop on and play.
So, worlds can offer great content and rich worlds, but they will generally have fewer players because there are no 'drive by' drop-in players. People have to go to some amount of effort to get the content before they can even try the world. Then, worlds are generally reluctant to add or change their content frequently, since that puts their playerbase at risk every time a change is made (since all players would then have to update their haks).
By having haks auto-loaded in some form or fashion opens the door for all worlds to use custom content and continuing to add to their content, which in turn, fuels the development of more custom content, which pushes NWN to remain current for a longer period of time.
We've needed this functionality for a long time. I sincerely hope this feature is added to the development roadmap.
this feature seems to be in the long road. As I see this should be implemented as soon as possible. We dont need the perfwcr solution. Its a head start / beta.
The master server was a step in the right direction. But its only a index of names where you cant join any game if it requires custom content.
How can we make devs prioritize this?
My questions:
1) An ability to designate to send an empty hak for the player.
I suppose the next point is not the same kind of question so not exactly number 2:
I have custom versions of all the spells - I have a spell mastery system that players gain levels 1 - 100 based on a die roll every time they cast a spell. This system required some modification to every spell script yet the name of the script is the same as everyone (from Bioware/Beamdog distribution).
The problem would occur if our versions were distributed - which I do not wish at any rate - since the name of the script is identical to the stock script. Would a distribution system just look at the name of a script or would it look at the file length to determine a difference? This not only applies to scripts but to any material - there are different versions of tilesets and yet the tile names are the same - and imposing a change on a server by getting the wrong material with the same name would cause a lot of problems.
Anyway I am in favor of some sort of distribution system - I am concerned about the bandwidth issue with pulling the material from the server. Auto install is really needed however - with the difficulty and issues i have been constantly dealing with for players trying to simply install haks and tlk. The new location of the Neverwinter Nights directory under Documents, along with the extraction issues from a rar or 7zip are just too technical for some players. The most common mistake is extracting from a rar or 7zip and having everything in a folder that they drop in the hak folder thus having the files in the wrong place.
So yes some method that will auto install - I use to use the auto install system that Bioware sent but that is windows only and of course will not work now - and was a bit flaky then due to size and amount of files and not using the best of compression. That file (nwcontbuild) needs a Mac version - I am guessing linux would not auto install but if so, then a linux version as well.
None the less, an update of that file (nwcontbuild), using better tight compression that would execute and install all - portraits, music, haks, and tlk would be great. An auto download from the web location set in nwnplayer.ini - so that would relieve the server bandwidth issue, drawing the files from off server site locations (I have my own site but others could use NWvault or?).
TR
As it is now i just simply do not want to bother finding/downloading 1000 different hak packs manually.
Now imagine a new player... I guess, the average new player won't even bother with the multiplayer part, it is just too complicated.
Currently how modules and haks are structured is to have one module (sometimes more if it's a story campaign rather than a PW) which is usually in the 10-100 mb range and a few hakpaks that are sometimes several hundred mb in size.
It's currently done this way because all the hakpaks are downloaded from the nwn vault and it's annoying to have to download and install lots of haks individually. If the downloading is automatically handled by the game client, then that removes this irritation.
In such a world of auto-downloading, it would make much more sense to have *lots* of very small haks so as to not waste bandwidth and hdd space. It's really wasteful and dumb having to download a new 100mb hak because someone added a new creature model.
The smaller we can make hak files, the more likely they are to be reused across servers.
Two things - I have no affiliation to this project whatsoever. I do not know if it is any use for pw players, which is why I keep asking people to look at it and report back.
TR
Quake 2 had auto downloading of content, it shouldn't be beyond the realms of man and science to get it implemented in EE. Even something like the NWN2 version would do for now.
Ideally allow multiplayer servers to prep and assign download location with server info along side to read (or possibly check for faster downloading links), allow single player modules to link to locations after a check for its required haks.