Skip to content

nwsync feedback & discussion thread

@niv, I have some questions and comments based on reading your doc over here:

I figured I'd start a thread rather than PM to invite discussion, hope that's okay. Thank you for the documentation and work so far!


1) If a server is passworded, will the game check that the player knows the password before downloading assets?
2) If a player has assets in HAKs/TLKs locally already, I expect nwsync won't source stuff from there unless a player manually jumps through a lot of hoops to seed their repository? That's probably not a feature I'd spend time on, but the thought did occur.
3) Our server still runs v1.69 and has server-only hak full of assets players DON'T need. There's various reasons for this and I'd be 100% okay with players downloading the contents of this HAK, but they don't need it and it is a waste of space on the player's side. Would you be open to giving the ability to mark specific HAKs as "private" or "server-side" in the Module Properties Custom Content tab? This would tell nwsync to skip the contents of these particular HAKs, if any. It would be the module designer's responsibility to get these settings correct. I have the impression server HAKs are not particularly unique to our world, but I don't get out much to other worlds. :)

I'll play with the thing a bit over the next couple weeks as time permits, but I thought it wouldn't hurt to start a discussion in parallel.



  • shadguyshadguy Member Posts: 154
    Couple more thoughts and notes:

    * (minor nit) In 8182, in the server browser, the column for nwsync that replaced ping has no column name/header/label
    * (feature suggestion) Please add support for an nwsync Alias in nwn.ini so I can redirect where the files live, like we have for hak/tlk/modules/etc etc.

    I really like the design concept you chose to work with. Things look pretty reasonable at first blush. I'll have to either go mess with my security settings or wait for https support to proceed with actual testing though - I'm erroring out on loading the server with a reasonable/expected message about https. I'll not get into that tonight, however. I've already blown up my server with ill advised late night puttering once this week. :-)

  • nivniv Member, Moderator, Developer Posts: 409

    Thanks for your test and feedback.

    1) Yes.
    2) Local data isn't imported yet. Down the line, I want to provide a utility to do this but the core functionality of this has no business nosing into the hak directory. For now, anyways.
    3) You are in full control what goes in the manifest. The update utility will take any kind of path/container spec you want to expose. You don't need to add everything the mod has; the example just gives it as, well, example, since that's the most straightforward use case.
    4) The ping column should have a proper label. At least it does on my end! Can you specify where/what exactly?
    5) Alias is a good idea and also already on the list.
  • shadguyshadguy Member Posts: 154

    Thank you for the reply - that makes a lot of sense. I'll make a an empty Server_manifest.mod that references all the public haks and omits the server/private hak. That can generate the player manifests and we're good to go - thank you for pointing that out. I didn't see it last night.

  • nivniv Member, Moderator, Developer Posts: 409
    You don't even have to generate a empty .mod.

    You can just go: ./update root/ 1.hak 2.hak 3.hak my.tlk
  • shadguyshadguy Member Posts: 154
    That is very slick - thank you. I see you mention that towards the end of the doc under Technical Details. May I suggest adding the sample command from your previous post in section 1 of the document, right after the other example of how to use ./update? That may help highlight the tool's flexibility.

    FYI, here's what I meant by the ping label column. Maybe I'm confused or describing it poorly:

    I was expecting something different from the demo during Friday's twitch stream, but I'm generally listening/not watching while puttering away at work, so I may have misunderstood. I haven't gone back to double check the twitchstream video to check my assumptions.

  • nivniv Member, Moderator, Developer Posts: 409
    Hmm, looks like either some files in ovr/ weren't updated fully (the new GUI models with a new stringref), or the dialog.tlk is not updated and is missing the text for "Sync". But that's what it should read. "Sync". Sorry for the spoiler.
  • JapualtahJapualtah Member Posts: 165
    My comment isn't going to be very helpful but, since you asked for feedback, here is mine.

    I downloaded the 8182 package on a server dedicated to test this out, different from my production server, fired a module and proceeded to read the NWSync User Manual.
    I don't know whom this user manual is written for, but after scratching my head a few times I reckoned the best was to close the whole thing and wait for a mere mortals version.

    Still love you man, keep up with the good job, but this is not my level.
  • JuliusBorisovJuliusBorisov Member, Administrator, Moderator, Developer Posts: 22,130
    /developer / official status off

    So, I've been monitoring all the sites discussing NWN:EE over the weekend, and among all the feedback NWSync has been getting spotted these ugly parts from the Codex:

    "this is basically a feature for *** who can't put files into folders or follow simple step-by-step instructions"

    "NWN has always been one of the easiest games to mod or install modules. If you own a PC, there's no excuse for not being able to do it."

    Once upon a time, the Codex was considered a serious place with RPG nerds and old school players.

    I'm just a CM, not a developer, but I find these comments strange. Is it a real opinion by people who play the game, or an opinion formed by a blind hatred?
  • JapualtahJapualtah Member Posts: 165
    More shortsighted than anything else IMO.
    I know of close friends who own NWN but stay away from PWs because they're not tech savvy at all.
    This is not even a debate, TBH.
  • jonesr65jonesr65 Member Posts: 66
    There are several other reasons why I don't play on PW's and having to download files and hak's and getting installed correctly is one of those reasons. From what I saw on watching the live stream yesterday that Niv and the Beamdog team has done a great job of designing a great system that should benefit players and builders both. Sure it will need some refinement but no program can be written the first time perfect.
  • dunahandunahan Member Posts: 129
    In my years on PWs as player, dm or admin I saw much of everything. Some users could sort everything blindfolded, some where blindfolded and couldn't do a thing without proper help.
    But on every PW there where players, dms or admins that helped. Players developed tools to do a similar job, like WorldGate we used on my PW.
    And then, there is or three Facebook groups which are willing to help.

    Don't know what they want, they are "bossing" around there. About things that aren't finished yet, a WIP or some things that show what could be done...
    Even my few players where happy about the news about the EE version.
  • zordren3742zordren3742 Member Posts: 207
    edited October 2018

    /developer / official status off

    So, I've been monitoring all the sites discussing NWN:EE over the weekend, and among all the feedback NWSync has been getting spotted these ugly parts from the Codex:

    "this is basically a feature for *** who can't put files into folders or follow simple step-by-step instructions"

    "NWN has always been one of the easiest games to mod or install modules. If you own a PC, there's no excuse for not being able to do it."

    Once upon a time, the Codex was considered a serious place with RPG nerds and old school players.

    I'm just a CM, not a developer, but I find these comments strange. Is it a real opinion by people who play the game, or an opinion formed by a blind hatred?
    Nothing new there, it is just a continuation of what started 90 pages ago in that post. They have been hating on that page sense November of last year. The things they are saying are not just hate towards beamdog, some of the things people are saying in that post are just plain shitty altogether and it has nothing to even do with Beamdog. Sad there are people on earth that act and think like this and then they get on a forum and share their ignorance with the world.
    Post edited by zordren3742 on
  • JapualtahJapualtah Member Posts: 165
    I run away screaming every time I seed the name 'GitHub' mate, I fear command line frenzy like the plague, I'm not the target audience of that stuff and I run the servers alone atm.
    Of course, with a lot of time and frustration, could do, but this should still be a game for end-users ;)
  • TarotRedhandTarotRedhand Member Posts: 1,177
    Speaking as a person who never plays on a pw, my interest was piqued by the livestream. So I open the multiplayer tab to look at the demo shown on the stream and guess what - its not there. Disappointing or what.

    So how am I as a mere mortal even supposed to comment on a feature that is not yet used anywhere?

  • dunahandunahan Member Posts: 129
    edited October 2018
    @Japualtah: I did a test with this following cmd on windows.
    update --description=mymod /caddy/windows "C:/Users/dunahan/documents/neverwinter nights/modules/module000.mod"
    It worked and created in my caddy-installation, where I extracted the binaries from @niv's, a data-folder and a manifest-folder.

    I'll post more tomorrow ;) Now its 22:19 o'clock here, and that's laaaate for me :tongue:
  • dunahandunahan Member Posts: 129
    edited October 2018
    Okay, now my steps for that example
    in the docs.
    My example needs a module with one hak and a custom tlk linked in it. Because without these, the nwnsync won't work. Those files are within the documents folder for nwn:ee.

    1. Download and extract caddy. For the
    test, you don't need any settings.
    Extract it to C:\caddy.
    2. Download and extract the binaries
    for nwnsync. Extract it to C:\caddy\nwnsync.
    Eventually rename folders or move the
    3. Start CMD, change to caddys folder with cd c:\caddy. Run caddy with caddy, without options.
    4. Start another CMD, because caddy runs within the first. With cd c:\caddy\nwnsync
    change to nwnsync folder.
    5. Run nwnsync with the command from above.
  • JapualtahJapualtah Member Posts: 165
    Thanks for taking the time dunahan, will give it a shot eventually.
    Not very keen on installing Caddy though, my web server is working fine and I don't fancy messing with possibly conflictual installations.
  • dunahandunahan Member Posts: 129
    I think it should work with any webserver. All I'm asking myself, can it use sftp? Does it need it? Didn't test that out yet.

    I'm hosting most of our custom haks and the tlk in web, but not at home at my server it self.
  • nivniv Member, Moderator, Developer Posts: 409

    caddy is just mentioned in case you don't have a decent web server set up yet. As it's a single-command thing, it helps people get set up rapidly. But you can use any modern webserver.

    sftp is not supported as a transfer protocol. Just plain http right now. I might look into adding ftp or similar though if there's demand. https is coming as well.
  • AzanixAzanix Member Posts: 3
    When will there be a version us mortal men can figure out without being born with code in our veins? :)
  • JapualtahJapualtah Member Posts: 165
    edited November 2018
    Today is a great day.

    I would like to thank wholeheartedly Niv on behalf of the community, if I may, for his outstanding job.
    NWSync is a revolution IMHO and as far as I can tell, it's working as flawlessly as you can expect from a newly released piece of software.

    I have a few things in mind that I'll throw in here.

    What about the music?
    I make extensive use of custom music files and they're not included in the repository, are they?

    Second off, just because I care about the poor folks who struggle as much as I do with command line stuff, please modify in your documentation 'Try update --help' with 'Try ./update --help'. Of course it's obvious. For you ;)

    Third off: the override folder is ignored. Since all my 2DA files are in override, my first manifest didn't include any 2DA and my players reported the weird things you can imagine.
    It's not a problem, really, I just copied all my override files in a hak, ran the sync and removed the 2DAs afterwards, not a big deal.
    There's nothing you can do about that, can you ? I'm merely raising a warning flag for modders, it's easy to get caught absentminded.

    Lastly, but that only comes from my own ignorance, I still haven't found how to connect to my web server, I did it the dirty way, mounting my NAS from GNOME and hoping my symbolic link would do the trick, which it seems to do, but I truly blush in shame.
    Time to tinker with /dev/fstab and root/.permissions which didn't go very well on my first attempts :#

    Again, I'm very happy with this new tool and incredibly relieved I could make it work.
    *Virtually shakes hands*
    Post edited by Japualtah on
  • JapualtahJapualtah Member Posts: 165
    I just noticed a bug though.
    On one computer, I had deleted all resources, emptied the hak, override and tlk folders.
    Connected to my PW, downloaded everything and could connect.
    Once connected, none of my custom placeables were to be seen.

    I then manually copied my placeables.2da file into the override folder and that fixed the problem.
    I'm going to investigate the problem a bit more and will come back if I find anything valuable to add.
  • nivniv Member, Moderator, Developer Posts: 409
    Hey @Japualtah,

    thanks for your kind words! I really do appreciate that. There's still a lot to do on NWSync, it's nowhere near where it needs to be, so expect more stuff in the future.

    Let me try and address your questions:

    - Music doesn't work yet. Music is currently streamed directly out of music/, etc and doesn't even go through where all other data is loaded from (ResMan). We're going to have to bend some things into shape to make this work, but it's on the TODO.
    - You don't have to create a hak. ./update actually can take multiple sources. You can give it the mod, you can give it extra haks, or you can add arbitary directories and files; i.e. ./update modules/my.mod override/. Note that order of entries determines shadowing ("overwrite") behaviour.
    - Serving up NWSync from a network share will be really slow. Getting a decent web server in place is going to be useful when you grow stuff a bit. But there's no rush. Start with what works, and refine later.

    Let me know if you figure out what's going on with placeables.2da. Make sure to restart your game server when you update the repo, so the server knows to tell clients about the new manifest hash.
  • JapualtahJapualtah Member Posts: 165
    Nailed it, it was some discrepancy between my local work files and the server's, so it was all on my end, my bad.

    Thanks a freaking lot about the optional directories for ./update, I overlooked that, it's really sweet!
    You may want to add to your document that if you want to add a folder you have to end the entry with /* to include all files. Again, might be obvious for you, much less for your target audience.

    About the remote web server, you're off course right.
    I run a few web servers from my Synology NAS and the one dedicated to NWN even runs on a Synology Wordpress packet, which is awesome. I don't see me moving my web servers to one of my NWN servers but maybe I would be better off serving NWSync locally and having a symbolic link on the NAS pointing to the sync folder?
    I don't even know if the NAS will let me do that, it farts in my general direction when I even try to log in as root, but if you think that would be the right way to do it, I'll look into it.
    Not even going to think about routing 80 traffic to different machines, I know it can be done, but oh boy, that's punching way above my weight!

    Anyway, thanks again, NWSync (when used properly o:) ) is fantastic.
  • JapualtahJapualtah Member Posts: 165
    Could you please shed some light on the game start delay please?

    The game now takes around 10 secs to launch instead of being instant before 8186, be it with F9 (test module) or using the 'reload when empty' option with a running Windows server, or connecting to my Linux servers.
    This waiting time quickly adds up especially when setting up visual effects which need to be tweaked very often to get right, emitters being what they are.
    I guess it's checking the local nwsync folder against the web repository which takes time, is it possible to turn it off for development? We have all resources locally anyway.
    Luckily I work on an i7 8700 with SSDs, I dread to imagine what it's like on older machines :#

  • nivniv Member, Moderator, Developer Posts: 409
    How big is your nwsync/nwsyncmeta.sqlite3 file?
  • JapualtahJapualtah Member Posts: 165
    3.404 Ko

    Anyway, I remembered the old trick we were using back in the days when the server would slowly start to crawl before a crash: StartModule(GetModuleName()); which instantly reloads the module.
    Works like a charm.
  • AzanixAzanix Member Posts: 3
    we finally went on live with Nwsync on our server but we get this error
    i have no idea where to even begin looking for this error
  • nivniv Member, Moderator, Developer Posts: 409
    @Azanix There has been a bug in the Windows utilities that generated invalid data. Please grab the latest version from here: if you are on windows.

    If you are not on Windows, please link me the broken manifest and possibly the source files so I can investigate.
Sign In or Register to comment.