What is the state of the internal SetCampaign style commands in NWN:EE? Are they still super slow?
dwinblood
Member Posts: 38
I love the fact that NWNX exists and for persistent worlds it is great. However, there is a great potential benefit to using databases for stand alone modules as well. The biggest problem is the built in functions are incredibly slow. In fact, to try to squeeze as much speed out of it as possible I know the internal DB supports writing objects instead of just variables, so to speed things up we would write all of our variables onto an object and just periodically serialize that. This was not without issue. It can introduce a lurch into the module. It does work, but that lurch can be an issue. What would be nice is if NWN:EE actually had Sql Lite or something integrated that would allow us to make modules for the AVERAGE person and they could just use them without knowing anything about setting up NWNX or something like that. Now if NWNX were somehow OFFICIALLY integrated into the NWN:EE game that too could mitigate this need. The biggest factor here is adding functionality without expecting the average user to jump through hoops. Some of them already have issues with installing HAKs and other things. Their loss, but still a loss.
2
Comments
On the subject of NWNX being an official Beamdog NWN:EE thing, I don't think that will ever happen, and that it's a good thing. Beamdog is for adding things to base game that make sense for their scope, while NWNX is for extending the capabilities of said base. It's meant to be cracked open, tinkered with, and added to by the community. That they're supporting it so strongly is fantastic, but it'll always be for those things out of scope for NWN.
That doesn't mean none of the features of its plug-ins can be integrated if they decide it does make sense for core, though. Some of them already have (eg area instancing)
You can get very good performance if you realize two things:
- you can have as many "CampaignDatabases" as you like (they are essentially just files), meaning dont put everything in file as you can have many fast, small files as you like
- and you realize that old data is never deleted on updates it is just marked as invalid and a new record is created with new value, which bloats out file size quickly if you do frequent updates.
For the first, I've found that if you break out your variables as much as possible to different files it is fine. Size is often under 16k per file, which is no negative performance on an SSD.
For the second, the key is that when you are ready for an update, you first delete the old database file and write the new values to a new file. That way your file is always "packed" by virtue that it is always fresh.
see below, which creates a refreshed, fully packed, single and new file for each PC henchman. If I were to simply keep the same db file and update the variable with new object to store, size and perf would go bad
string sHenchmanTagId = GetTag(oHenchman);
string sDBFile = GetPlayerCharTag(oPC) + "_" + GetTag(oHenchman);
//Destroy the old file first to prevent the char file from exploding in size
DestroyCampaignDatabase(sDBFile);
//Now overwrite with the old file
int iSaveResult;
iSaveResult = StoreCampaignObject(sDBFile, sHenchmanTagId, oHenchman);
I store large store inventories, multiple companions with inventory, etc., and it is fine provided that I delete the db old file first, then re-store data in a net new file.
EE doesn't distribute these utilities (yet?) but the format didn't change so the 1.69 will work for 1.74 as well.
But it's running fine for me ;-)
One final note though.... The packer does not work when a database gets too large in size so there is in fact a limit window associated with it. I noticed it when using databases with lots of repeat entries such as storing and deleting bank chests. I think the size limit is somewhere around a gig or more though so you have a lot of leeway
If the deletion performance is not terrible, maybe we can get a `SetMutableCampaignObject` that deletes the older object entry for good when called?
For NWN, a secondary thread that occasionally does the packing when resource use is low would be more than enough. But, that's not really different from a cronjob that calls the packer utility, which you can do yourself.
I can read that your Persistent Facade 2 could help creators to easily switch from different types of persistencies.
How could it help me to convert my full NWNX PW into a Bioware DB PW? So I can launch it in NWN:EE
Thank you for your answer,
First, import only facade without any extension. If you use APS then you should import the part for it too. The second step would be the change of the functions named in your old scripts that use APS plus add the inclusion of the facade include.
In the main include of facade is the way which it uses the databases. There ist also the way discribed how to use it
Will try this. Thanks a lot for your answer!
Yet I am doing some pretty complex things. I had a module that was complex enough that the actual SAVE mechanism in the engine didn't actually save all data. So reloading would not always resume where it was at.
I therefore had to make my own system to store and restore things that the normal SAVE could not.
Thanks for the response.
I didn't try the destroy before updating technique before though. So that is worth looking into.
https://discord.gg/CukSHZq