// This will remove ANY campaign variable. Regardless of type.
// Note that by normal database standards, deleting does not actually removed the entry from
// the database, but flags it as deleted. Do not expect the database files to shrink in size
// from this command. If you want to 'pack' the database, you will have to do it externally
// from the game.
void DeleteCampaignVariable(string sCampaignName, string sVarName, object oPlayer=OBJECT_INVALID);
So, the bioware database will never shrink in size. If you keep adding and deleting variables, it'll just grow and bloat and get slower and slower. If you have a relatively fixed number of variables to store, that shouldn't be an issue.
There used to be a utility to process the database out of the game and repackage it, but I don't know if it was reliable.
Then, there's also the fact that the native DB is just plain slow. But, if it is fast enough for your use case, that's no big deal.
So Biowares DB just continues to bloat because it doesnt actually delete anything.
Why would Bioware not have made it work properly to begin with?
And how does NBDE get around that and how is it faster?
NWNX is obviously the king of the hill by a long shot for speed and functionaliy but having never hosted a server before I know enough to know it works but not exactly why.
Why would Bioware not have made it work properly to begin with?
Database management has come a long way since 1999. Even then, the database bioware used was slightly outdated. But generally databases are built to be as fast as possible for the caller (in this case nwscript), and then the remaining work was meant to be done when there are CPU cycles to spare. In case of NWN, that second part was meant to be done manually using the external utility.
And how does NBDE get around that and how is it faster?
There is a DestroyCampaignDatabase() function, which simply deletes the file in question. How NBDE works is that it spawns a placeable, and rather than writing your variables directly to the database, it writes them as locals on that placeable. Then when you call flush(), it deletes the old database and stores the entire placeable with StoreCampaignObject(). So, with NBDE, the database only ever has a single variable in it - an object with a bunch of locals on it.
It is faster because every time you call a SetCampaignXxx, NWN has to write to disk, which is slow. NBDE instead writes to local variables on the placeable, in memory which is fast, and then only does a single write to disk when you flush. So it will be slower if you flush after every variable, but if you need to store a group of variables at once, it's much faster.
EDIT: As for NWNX, it _is_ faster, but that's not the real selling point - NBDE is reasonably fast for medium sized servers (i.e. not Arelith). NWNX/SQL allows you to structure your database in such a way to model relations between various pieces of data and quickly do operations on all of those. For example, suppose you have a bank that stores gold for players, and charges 0.1% per day. With NBDE/BioDB, in order to do the accounting on all players, you have to Get() the stored gold for every player, calculate and deduct the payment and then Set() the new value. With NWNX it's just "UPDATE bank SET gold=0.999*gold;" And you can easily have much more complex logic without writing much code. Perhaps you want to roll Appraise and make the payment dependent on that? And also check player titles, and have lords pay less than knights? All of that comes down to more or less a single command, as opposed to what will probably grow to 100 lines of nwscript.
I've always had someone else hosting for me so I have never had to deal with the other side of things so this simply hasn't been part of my world and it's not always the easiest thing to find up to date info for. After 16 years there is a lot of information out there and not all of it is accurate or applicable today. Which really makes NWNX so amazing. It's still in active development which just boggles my mind.
Stumbling across this discussion, I'd like to chime in
I've been using NBDE for a long time and was concerned with the bloat until Beamdog solved the DestroyCampaignDatabase bug on Linux, making the bloat concern moot. Now, every time I save a database, it is destroyed first, which means bases are as slim as possible at all times.
Besides the extremely fast NBDE, I extensively use StoreCampaignObject which is very convenient provided you store your databases on SSD.
I was using NWNX 15 years ago but decided not to anymore because it didn't exist on Windows when I started developping my PW again, no clue whether this has been addressed. Not to mention it's a pain to install on Linux if you're not a geek. Sherincall has done a good job with his complete installation shell but I couldn't make it work last time I tried, so I decided to only use standard NWN code.
I don't miss the SQL relational aspect at all either. If I ever need to cross-reference tables, I can do so without SQL.
@Japualtah Interesting, but how do you delete player's undesired characters without nwnx? (my main concern) manually? and do you use an alternative to reboot the server in case of crash? thanks
And you can easily have much more complex logic without writing much code. Perhaps you want to roll Appraise and make the payment dependent on that? And also check player titles, and have lords pay less than knights? All of that comes down to more or less a single command, as opposed to what will probably grow to 100 lines of nwscript.
Still nwn allows to use about 5-10% of real database uses as we don't have any complex objects except structures. Every implementation of simple list is convoluted and requires a ot of manual fetching and mapping of columns.
Simple implementation of table would be a game changer really.
@Japualtah Interesting, but how do you delete player's undesired characters without nwnx? (my main concern) manually? and do you use an alternative to reboot the server in case of crash? thanks
Player speaks to an NPC, which adds a line to the logs with WriteTimestampedLogEntry Linux shell on cron then tail | grep the logs and deletes the character file.
In case of a crash, my screened shell simply runs
while [ 1 ]
do
./nwserver-linux -module althea -port 5121 -publicserver 1 -servername "FR] Althea, le Quadracle" -nwsyncurl http://japu.synology.me/nwsync
done
Not having any trouble, running a small PW though never more than 10 people online at any give time, I opened the servers last month.
@Japualtah Interesting, but how do you delete player's undesired characters without nwnx? (my main concern) manually? and do you use an alternative to reboot the server in case of crash? thanks
Player speaks to an NPC, which adds a line to the logs with WriteTimestampedLogEntry Linux shell on cron then tail | grep the logs and deletes the character file.
In case of a crash, my screened shell simply runs
while [ 1 ]
do
./nwserver-linux -module althea -port 5121 -publicserver 1 -servername "FR] Althea, le Quadracle" -nwsyncurl http://japu.synology.me/nwsync
done
Not having any trouble, running a small PW though never more than 10 people online at any give time, I opened the servers last month.
I use scripts someone made to delete characters. Linux bash script. Script puts an item on the bic. file. The bash script run every 24 hours to wipe bic. file with item. I'm testing now with EE because EE does cdkey or playername so not sure now.
Comments
// This will remove ANY campaign variable. Regardless of type. // Note that by normal database standards, deleting does not actually removed the entry from // the database, but flags it as deleted. Do not expect the database files to shrink in size // from this command. If you want to 'pack' the database, you will have to do it externally // from the game. void DeleteCampaignVariable(string sCampaignName, string sVarName, object oPlayer=OBJECT_INVALID);So, the bioware database will never shrink in size. If you keep adding and deleting variables, it'll just grow and bloat and get slower and slower. If you have a relatively fixed number of variables to store, that shouldn't be an issue.
There used to be a utility to process the database out of the game and repackage it, but I don't know if it was reliable.
Then, there's also the fact that the native DB is just plain slow. But, if it is fast enough for your use case, that's no big deal.
You can also take a look at NBDE, which is a library built on top of the bioware one, which removes the first problem (bloat) and significantly improves the speed, but at the cost of a slightly harder interface - you have to manually 'flush' it to store the data to disk.
https://neverwintervault.org/project/nwn1/script/nbde-natural-bioware-database-extension-v10
So Biowares DB just continues to bloat because it doesnt actually delete anything.
Why would Bioware not have made it work properly to begin with?
And how does NBDE get around that and how is it faster?
NWNX is obviously the king of the hill by a long shot for speed and functionaliy but having never hosted a server before I know enough to know it works but not exactly why.
Database management has come a long way since 1999. Even then, the database bioware used was slightly outdated. But generally databases are built to be as fast as possible for the caller (in this case nwscript), and then the remaining work was meant to be done when there are CPU cycles to spare. In case of NWN, that second part was meant to be done manually using the external utility.
There is a DestroyCampaignDatabase() function, which simply deletes the file in question. How NBDE works is that it spawns a placeable, and rather than writing your variables directly to the database, it writes them as locals on that placeable. Then when you call flush(), it deletes the old database and stores the entire placeable with StoreCampaignObject(). So, with NBDE, the database only ever has a single variable in it - an object with a bunch of locals on it.
It is faster because every time you call a SetCampaignXxx, NWN has to write to disk, which is slow. NBDE instead writes to local variables on the placeable, in memory which is fast, and then only does a single write to disk when you flush. So it will be slower if you flush after every variable, but if you need to store a group of variables at once, it's much faster.
EDIT: As for NWNX, it _is_ faster, but that's not the real selling point - NBDE is reasonably fast for medium sized servers (i.e. not Arelith). NWNX/SQL allows you to structure your database in such a way to model relations between various pieces of data and quickly do operations on all of those.
For example, suppose you have a bank that stores gold for players, and charges 0.1% per day. With NBDE/BioDB, in order to do the accounting on all players, you have to Get() the stored gold for every player, calculate and deduct the payment and then Set() the new value. With NWNX it's just
"UPDATE bank SET gold=0.999*gold;"
And you can easily have much more complex logic without writing much code. Perhaps you want to roll Appraise and make the payment dependent on that? And also check player titles, and have lords pay less than knights? All of that comes down to more or less a single command, as opposed to what will probably grow to 100 lines of nwscript.
However, for Windows servers, NBDB/BioDB are still our only realistic options, right?
https://github.com/mtijanic/nwnxlite
I've always had someone else hosting for me so I have never had to deal with the other side of things so this simply hasn't been part of my world and it's not always the easiest thing to find up to date info for. After 16 years there is a lot of information out there and not all of it is accurate or applicable today. Which really makes NWNX so amazing. It's still in active development which just boggles my mind.
I've been using NBDE for a long time and was concerned with the bloat until Beamdog solved the DestroyCampaignDatabase bug on Linux, making the bloat concern moot.
Now, every time I save a database, it is destroyed first, which means bases are as slim as possible at all times.
Besides the extremely fast NBDE, I extensively use StoreCampaignObject which is very convenient provided you store your databases on SSD.
I was using NWNX 15 years ago but decided not to anymore because it didn't exist on Windows when I started developping my PW again, no clue whether this has been addressed. Not to mention it's a pain to install on Linux if you're not a geek. Sherincall has done a good job with his complete installation shell but I couldn't make it work last time I tried, so I decided to only use standard NWN code.
I don't miss the SQL relational aspect at all either. If I ever need to cross-reference tables, I can do so without SQL.
Cheers
Simple implementation of table would be a game changer really.
WriteTimestampedLogEntryLinux shell on cron then
tail | grepthe logs and deletes the character file.In case of a crash, my screened shell simply runs Not having any trouble, running a small PW though never more than 10 people online at any give time, I opened the servers last month.
The bash script run every 24 hours to wipe bic. file with item.
I'm testing now with EE because EE does cdkey or playername so not sure now.