Skip to content

Persistent storage feedback/advice/nuance

NeverwinterWightsNeverwinterWights Member Posts: 339
edited February 2018 in Builders - Scripting
Like with most systems that already exist on the vault and what not I tend to take the "do it myself to learn" and "reinvent the wheel" approach. So for persistent storage I've taken the approach:

~Use NWNdatabase.
~A shared chest/object (set to plot) between players with scripts in the OnOpened, OnClosed and OnClicked events.
~when chest is opened it retrieves or creates(first time used) a campaign storage creature(skin item removed) that will spawn in an inaccessible area. All items from storage creature are copied to chest and destroyed on creature.
~when chest is closed it copies all items from chest to creature and destroys all items in the chest. Re-stores the campaign creature storage object and destroys the currently spawned one.
~when chest is clicked on it checks to see if it is open (in use) if so ClearAllActions and inform player that it is in use so they can't use the chest (Beamdog fixed the issue of placeables sometimes staying open).
~The item limit in the chest should never be a problem because you are only copying back and forth what the chest can hold.

So far in testing everything seems to work as intended. In one test I placed an item in the chest but intentionally shut down the NWN.exe while the chest was still open which did result in the loss of said item. I figured that would be the result.

One thing I have not been able to test yet: When a player is using the chest and crashes or logs out does the chest stay open? Does Beamdog's fix for open chests address this? Will it close and still fire it's closing script?

Anyway the reason for this topic was to get informed by the community members who have already worked with and ironed out bugs with persistent storage. To bring up any specific things that need to be addressed. Problems with duping, containers, stacks, etc and how to specifically address these. Things that I've not thought of or addressed in my approach.

Any advice would be much appreciated.
Post edited by NeverwinterWights on

Comments

  • HimmelweissHimmelweiss Member Posts: 72
    edited February 2018
    I load items with the OnOpened Event.
    I do this only one time for the entire duration the server is up.
    There is no need to load items from DB everytime a player opens the chest.
    Unless ofc. the chest gets deleted/recreated like in an player housing system or so.

    I save & remove items with the onDisturbed event by asking for the specific disturb type.
    INVENTORY_DISTURB_TYPE_ADDED
    INVENTORY_DISTURB_TYPE_REMOVED & INVENTORY_DISTURB_TYPE_STOLEN

    That way, even if the client crashes he looses nothing.
    Sure, it is more load for the Server because everytime a player drops or removes an item from the chest you get a query to the DB. But, and that's just my opinion, it is much safer that way for the player.

    I don't use the OnClosed or OnClicked Events at all.

    Well just my opinions :d
    There may be even better ways, dunno.
    Sherincall
  • NeverwinterWightsNeverwinterWights Member Posts: 339
    edited February 2018
    I should probably add that this is a shared chest/object between players. If I load the items once, every ones items will end up in the chest no?

    And are you saving each individual item? Or saving them all to a creature and then saving that creature to the database?
  • HimmelweissHimmelweiss Member Posts: 72
    Well if you want to share the same chest then ofc. you should always load the content of the chest.

    I am saving each individual item. More Control, but also more load to server.
    That's just my personal preference.
    I don't really like to deal with a creature or something when it comes to persistent chests.
    Too much risk loosing just everything if you only save on the OnClosed event.

    Now when it comes to saving a henchman where i can control its inventory, then sure, i just save the entire creature/henchman.

    Anyways:
    Imagine a player on your server is filling a new chest with lots of cool items and then he or even the server crashes so that your OnClosed event doesn't fire.
    The result = a player that will probably rage quit
    NeverwinterWights
  • NeverwinterWightsNeverwinterWights Member Posts: 339


    Imagine a player on your server is filling a new chest with lots of cool items and then he or even the server crashes so that your OnClosed event doesn't fire.
    The result = a player that will probably rage quit

    Yep. This is exactly why I started this topic. Have these things been worked around yet?

    What I'm wondering though is if the player crashes and exits will the chest close now and still fire the OnClosed script. If so then the items will still transfer and the creature storage object will still be saved. Since the PC object is still there, everything may still save to the DB ok? The problem is I don't have anyway to test in multiplayer EE yet.

    The server crashing would still be a problem though for sure.

    Personally I'm not sure I like the idea of saving every single individual item for every player though. Hmm. Something to think about though.
  • HimmelweissHimmelweiss Member Posts: 72
    Well, there is nothing wrong with your approach. Many do it that way.
    I also did the creature thing with my NWN2 Player Housing System.

    The Server rarely crashes anyway, i think it actually never happened to me once.
    But i never ran a huge PW with lots of active players playing, so i am definetly the wrong person to ask for experience.

    I actually have no clue either what happens if the client crashes while the chest is opened, hmm... might be worth it to make a test.


  • NeverwinterWightsNeverwinterWights Member Posts: 339
    edited February 2018
    Ok so the OnClosed script will still fire after a player crashes and I found a way to make sure the items in the chest are still saved for that player.

    Now the problem I'm having is that, while the chest is open by one player, another player can still use the radial menu to "use" the chest. I put check scripts in the OnClick event and the OnUsed event. The OnClick works fine. Checks to see if it's open and calls a ClearAllActions and sends a message to any players who click it while it's open stopping them and letting them know the chest is in use.
    I thought just doing something similar in the OnUsed event would give me the same result but I just can't seem to stop other players from "using" the chest if it's open/in use by another player.

    Any ideas or suggestions?
  • henesuahenesua Member Posts: 62
    edited February 2018
    I've found that a creature is a better persistent repository than a placeable. The larger inventory helps avoid lots of problems, and if you also need persistent inventory for a creature instead of a placeable then you've got parity. So you've stumbled on the most important part of all of this.

    An example that you can pull apart and study:
    https://neverwintervault.org/project/nwn1/script/maguss-persistent-inventory-examples
    NeverwinterWights
  • NeverwinterWightsNeverwinterWights Member Posts: 339
    @henesua Thanks I'll check that out for sure. In the meantime, are you saying there is no way to prevent a player from "using" the object when they use their radial menu?
  • henesuahenesua Member Posts: 62
    @NeverwinterWights Huh? No. I'm saying that your choice of using a creature as the mule for storing items in the DB is a very good choice.
    NeverwinterWights
  • NeverwinterWightsNeverwinterWights Member Posts: 339
    henesua said:

    @NeverwinterWights Huh? No. I'm saying that your choice of using a creature as the mule for storing items in the DB is a very good choice.

    Ooooh gotcha sorry. Totally read that wrong. I still am trying to figure out why exactly you can't stop someone from "using" a placeable the same way you can stop then in the OnClicked event. You would think a simple check of somekind (GetIsOpen, GetLocal, etc) could work the same way. As far as I can tell the only thing I might be able to do is teleport the 2nd "offending" character somewhere or maybe add a paralyzing effect or similar. Not very clean or desired effect though. I wonder if this is something I should bring up as a suggestion or ask during a stream. Hmm.
  • RifkinRifkin Member Posts: 141

    henesua said:

    @NeverwinterWights Huh? No. I'm saying that your choice of using a creature as the mule for storing items in the DB is a very good choice.

    Ooooh gotcha sorry. Totally read that wrong. I still am trying to figure out why exactly you can't stop someone from "using" a placeable the same way you can stop then in the OnClicked event. You would think a simple check of somekind (GetIsOpen, GetLocal, etc) could work the same way. As far as I can tell the only thing I might be able to do is teleport the 2nd "offending" character somewhere or maybe add a paralyzing effect or similar. Not very clean or desired effect though. I wonder if this is something I should bring up as a suggestion or ask during a stream. Hmm.
    You don't need to prevent users from accessing the chest, because you aren't using the chest's inventory as the location they put/retrieve items from.

    The chest is just to kick-off your script to return the DB(campaign) object, of which is an invisible, and plot creature that contains their items in it's inventory.

    When they click on the chest, you spawn the creature and use the 'OpenInventory' command to show the creatures inventory to the player.

    This means, multiple people can use the chest at the same time, and they only see their own inventory, because in reality, they aren't seeing the chest inventory, they are seeing their campaign creature's inventory.. Of which, you create one for each player, and store it via SetCampaignObject, and retrieve via GetCampaignObject...

    There are already versions of this exact setup on the NWVault, but if you're feeling smart you can definitely write one of your own.
    NeverwinterWights
  • NeverwinterWightsNeverwinterWights Member Posts: 339
    edited February 2018
    Rifkin said:


    The chest is just to kick-off your script to return the DB(campaign) object, of which is an invisible, and plot creature that contains their items in it's inventory.

    When they click on the chest, you spawn the creature and use the 'OpenInventory' command to show the creatures inventory to the player.

    Unfortunately this presents it's own set of problems:

    -You have to add the creature as a henchmen to open it's inventory.
    -I could be mistaken, but I don't believe vanilla NWN has a "real" invisible creature. When you tab you can still see a glow, nameplate, damaged.
    -You can also see all of the creatures item slots while in it's inventory which kinda takes away the idea that it is just supposed to be a chest. This is just a preference thing. Some might like that idea.
    -Also, is there an event that fires when you close the creatures inventory that you could use to remove the henchmen and re-store it as a campaign object?
    -According to the Lexicon "On creatures, it seems only the stolen distrubed type will fire.". I haven't tested to see if this is still true but I was thinking of using the OnDisturbed event to re-store the inventory creature campaign object in the event that a player crashes while in the creature inventory (I already solved this problem with the chest's OnClosed event).

    There are systems on the vault that create a "middle man" object. So you would use the pretend chest, fake the animations, spawn the campaign storage creature, spawn an invisible object, and then transfer the inventory back and forth between the creature and the invisible object. I will probably either go this route or try and find a way to prevent another player from "using" the chest (I would still prefer the 'only one person use the chest at a time' method but that might be out of the question).

    Edit: So everything is possible using the "middle man" object. And all doable, so far, with only 2 scripts.
    Post edited by NeverwinterWights on
  • NeverwinterWightsNeverwinterWights Member Posts: 339
    edited February 2018
    So this is what I came up with so far:

    OnUsed (useable chest with no inventory):
    void ForceInteract(object oPC, object oObject) { AssignCommand(oPC, ActionInteractObject(oObject)); AssignCommand(oPC, ActionDoCommand(SetCommandable(TRUE))); AssignCommand(oPC, SetCommandable(FALSE)); } void main() { object oPC = GetLastUsedBy(); if (!GetPlotFlag(OBJECT_SELF)) SetPlotFlag(OBJECT_SELF, TRUE);//safeguard if (!GetIsOpen(OBJECT_SELF)) { string sPCKey = GetPCPublicCDKey(oPC); string sName = GetName(oPC); location lNPCWP = GetLocation(GetWaypointByTag("STORAGE_NPC_WP")); object oMyStorageNPC = RetrieveCampaignObject("PERSISTENT_STORAGE", sPCKey+sName, lNPCWP); object oMiddleMan = CreateObject(OBJECT_TYPE_PLACEABLE, "invisiblestorage", GetLocation(OBJECT_SELF)); if (!GetPlotFlag(oMiddleMan)) SetPlotFlag(oMiddleMan, TRUE);//safeguard SetLocalString(oMiddleMan, "CURRENT_KEY", sPCKey); SetLocalString(oMiddleMan, "CURRENT_NAME", sName); SetLocalObject(oMiddleMan, "CHEST_TO_CLOSE", OBJECT_SELF); PlayAnimation(ANIMATION_PLACEABLE_OPEN); ForceInteract(oPC, oMiddleMan); if (GetIsObjectValid(oMyStorageNPC)) { SetLocalObject(oMiddleMan, "CURRENT_STORAGE_NPC", oMyStorageNPC); object oItem = GetFirstItemInInventory(oMyStorageNPC); while (GetIsObjectValid(oItem)) { //Add gold, stack, container stuff CopyItem(oItem, oMiddleMan, TRUE); DestroyObject(oItem); oItem = GetNextItemInInventory(oMyStorageNPC); } } else { oMyStorageNPC = CreateObject(OBJECT_TYPE_CREATURE, "nw_oldman", lNPCWP); SetPlotFlag(oMyStorageNPC, TRUE);//safeguard SetLocalObject(oMiddleMan, "CURRENT_STORAGE_NPC", oMyStorageNPC); } } else { SendMessageToPC(oPC, "This chest is currently in use by another player. Please wait until the player is finished or use another storage chest."); } }

    OnClosed (invisible object with inventory):
    void main() { string sPCKey = GetLocalString(OBJECT_SELF, "CURRENT_KEY"); string sName = GetLocalString(OBJECT_SELF, "CURRENT_NAME"); object oMyStorageNPC = GetLocalObject(OBJECT_SELF, "CURRENT_STORAGE_NPC"); object oChestToClose = GetLocalObject(OBJECT_SELF, "CHEST_TO_CLOSE"); object oItem = GetFirstItemInInventory(OBJECT_SELF); while (GetIsObjectValid(oItem)) { //Add gold, stack, container stuff CopyItem(oItem, oMyStorageNPC, TRUE); DestroyObject(oItem); oItem = GetNextItemInInventory(OBJECT_SELF); } StoreCampaignObject("PERSISTENT_STORAGE", sPCKey+sName, oMyStorageNPC); AssignCommand(oChestToClose, PlayAnimation(ANIMATION_PLACEABLE_CLOSE)); DestroyObject(oMyStorageNPC, 0.5); DestroyObject(OBJECT_SELF, 0.5); //Debug: check to see if OnClose ran if another player crashes SendMessageToPC(GetFirstPC(), "Persistent Storage Closing Script Ran! Items saved."); }

    If I missed or messed up something that could fubar everything, please let me know.
    I might still go with using the OnDisturbed event of the invisible object to store the campaign creature object each time an item is added or taken. Would be the only way to prevent loss in the event of a server crash.
    Post edited by NeverwinterWights on
  • zunathzunath Member Posts: 92
    You're welcome to take a look at how we do persistent storage.

    https://github.com/zunath/CyberpunkZombieSurvival_JVM/blob/master/src/main/java/GameSystems/StorageSystem.java

    It's in Java and using an ORM, so it won't be a direct copy-paste but maybe it'll help you get an idea.
    NeverwinterWights
  • hermythermyt Member Posts: 12
    edited February 2018
    I wrote this

    https://neverwintervault.org/project/nwn1/script/hermyts-persistent-chests

    Many moons ago still a good system though. I did quite a bit of testing at the time as the preferred method of storage then was using object reference IDs and nwnx storage but that resulted in loosing customized item properties. The issue then was that bulk writes to the nwn db were taxing on multiplayer servers and firing a script that will dump 200+ items to the nwn db WILL cause a solid lag spike on the server.

    Because of this, using the aforementioned 'ondisturbed' event is MUCH better for your server, a player cannot physically dump objects into a chest fast enough to cause any performance issues on a multiplayer server unlike doing a bulk dump of potentially 100's of items in one go.

    Reading from the nwn db is super fast too so loading works great it's just the writing to the db that's slow.

    So +1 for the 'ondisturbed'

    Also, even exporting a chest object singularly will be slow if it contains 100's of items as the actual detailed item information including model components and colors and item properties and powers and attached variables will all be exported for every single item in the chest.

    Also note zunaths system above uses a more recent nwnx system than back in the day and his post allows true item serialized storage using nwnx which is super awesome.

    Although I believe that implementation of nwnx is Linux only unless I'm mistaken.


    NeverwinterWights
  • NeverwinterWightsNeverwinterWights Member Posts: 339
    @Barry_1066 If you don't mind me asking, i know you are also not using NWNX, what method are you using for persistent storage? I'd like to get your take.
  • NeverwinterWightsNeverwinterWights Member Posts: 339
    edited February 2018
    @hermyt Looking over yours now. Got a couple questions. What happens when another player uses the same chest and tries to take stuff out or put stuff in?
  • zunathzunath Member Posts: 92
    @hermyt Correct - it's NWNX only, which is currently only available on Linux. But you could rewrite the thing in NWScript and use the Bioware DB just by swapping out a few calls. I do notice that I'm iterating over the chest on every disturbed though - I probably should fix that! :smile:

    You'd have to go a step further if you want private player storage. I've got a solution for that too, but unfortunately it's tied to a larger "player building" system I've developed. If you think it might be helpful I can drop you a link to the code.
  • NeverwinterWightsNeverwinterWights Member Posts: 339
    zunath said:


    You'd have to go a step further if you want private player storage. I've got a solution for that too, but unfortunately it's tied to a larger "player building" system I've developed. If you think it might be helpful I can drop you a link to the code.

    As I have it scripted above it can be used privately or publicly since it now creates an invisible "chest" for each player.
    zunath said:

    You're welcome to take a look at how we do persistent storage.

    https://github.com/zunath/CyberpunkZombieSurvival_JVM/blob/master/src/main/java/GameSystems/StorageSystem.java

    It's in Java and using an ORM, so it won't be a direct copy-paste but maybe it'll help you get an idea.

    Looking this over reminded me that I most likely need to add a storage item limit. Is there a limit on items you can put in a placeable inventory? Can a placeable inventory hold more than a creature? I don't remember.
  • zunathzunath Member Posts: 92
    Cool, glad you've got a solution for it already.

    I believe there's a limit on it but I don't know off hand what the amount is. Ours are generally capped at 20-30 items mostly for performance reasons (serialization is heavy).
  • BalanorBalanor Member Posts: 176
    Placeable inventory is limited to about 25 pages and then stuff past that will just "disappear". I ran into this problem with the SOU treasure system that uses chests for low, medium, and high treasure. While you can load them up with inventory in the toolset, when in game the inventory gets truncated to ~25 pages.

    If you want to implement a storage limit, which I would highly suggest, you could do so via a script in the OnDisturbed which gives the item back if it is over the limit, or in the OnClosed, simply end and don't save persistently when the PC closes the chest (and send them a message they need to take items out to save).
    zunathNeverwinterWights
  • hermythermyt Member Posts: 12

    @hermyt Looking over yours now. Got a couple questions. What happens when another player uses the same chest and tries to take stuff out or put stuff in?

    This system wasn't designed originally as a player exclusive storage but a quick drop in storage system, iirc you just had to unique tag the area and then the chest was made unique by its location coordinates so you didn't have to do any additional coding and can just drop them anywhere.

    You would need to add some code to make them player specific, accounting for the player name and cdkey reference in the chests ID and then modifying the onopen to only load the players specific contents, the ondisturbed event to check who's accessing the chest in case someone else is trying to loot it and/or store items in it then in the onclosed event you would empty the chest as well in preparation for another player accessing their own unique contents (ie: using it as a central bank).

    One issue with the onclosed event for processing though is that occasionally a chest will get 'stuck' open, many persistent world's had this issue. If your storing items in the onclosed event it will stall the storage system, the only known fix has been to recreate the chest by using a lever or something else which has been the usual solution.
  • hermythermyt Member Posts: 12
    zunath said:

    Cool, glad you've got a solution for it already.

    I believe there's a limit on it but I don't know off hand what the amount is. Ours are generally capped at 20-30 items mostly for performance reasons (serialization is heavy).

    This is why the ondisturbed event is handy, serializing a single object per player action is alot less intense than attempting to do 20-200+ in a response to a single event (on closed). It also bypasses that occasion stuck open chest bug that may or may not be fixed now.

    zunath
  • zunathzunath Member Posts: 92
    hermyt said:

    zunath said:

    Cool, glad you've got a solution for it already.

    I believe there's a limit on it but I don't know off hand what the amount is. Ours are generally capped at 20-30 items mostly for performance reasons (serialization is heavy).

    This is why the ondisturbed event is handy, serializing a single object per player action is alot less intense than attempting to do 20-200+ in a response to a single event (on closed). It also bypasses that occasion stuck open chest bug that may or may not be fixed now.

    Agreed - but you still need to retrieve the items which requires deserialization and can be lag inducing. If it becomes an issue though, just load the inventories on module load instead of on open.
  • hermythermyt Member Posts: 12
    zunath said:

    hermyt said:

    zunath said:

    Cool, glad you've got a solution for it already.

    I believe there's a limit on it but I don't know off hand what the amount is. Ours are generally capped at 20-30 items mostly for performance reasons (serialization is heavy).

    This is why the ondisturbed event is handy, serializing a single object per player action is alot less intense than attempting to do 20-200+ in a response to a single event (on closed). It also bypasses that occasion stuck open chest bug that may or may not be fixed now.

    Agreed - but you still need to retrieve the items which requires deserialization and can be lag inducing. If it becomes an issue though, just load the inventories on module load instead of on open.
    With the nwn db then deserialization load time is crazy fast, I can't recall specific numbers but I set up timers to record load times under high item load.

    Actually just checked I had posted the load times, it's about 200ms per loading 300 serialized items. I had mine set normally that it would just load that once per reset with my script.
Sign In or Register to comment.