Skip to content

nwsync feedback & discussion thread

1235»

Comments

  • jpsweeney94jpsweeney94 Member Posts: 93
    edited January 2020
    Having some issues again with NWSync since new update to module :/ Hoping for some help.

    I ran NWSync and upload files to my webserver via FTP. FTP gives me this prompt about 5 times: https://www.dropbox.com/s/c8jq1sznlebrojk/nwsync_overwrite.png?dl=0 (I'm pretty sure i tried keeping either files and same error... If I'm remembering incorrectly, I atleast know i did the "overwrite" option to keep the source file)

    After upload, i try to join the server and get this error: https://www.dropbox.com/s/o1105p3zy7cazvh/join_error.png?dl=0

    Almost like NWSync has files named the same and they need to overwrite or something? Never had this issue until recently. Only recent change that *might* be causing it is a new hak (contents): https://www.dropbox.com/s/t0tfdl3397q6cjm/MEX10_TOP_2DA_hak.png?dl=0

    Also made some changes like adding music to a hak and what not, but those worked before this newly added hak.

    Any idea what could be the problem?

    Thanks!

    EDIT: Re-ran NWSync write and saw these in the log as well, could this be it?
    W, [2019-12-31T18:20:44] -- nwsync_write: Duplicate resref entry in erf, but differing offset/size: resref=m_crowishwizard_.tga offset=39323109 size=24620 (idx=1172) would collide with offset=39224761 size=98348; renamed to __erfdup__1172.tga
    W, [2019-12-31T18:20:44] -- nwsync_write: Duplicate resref entry in erf, but differing offset/size: resref=m_crowishwizard_.tga offset=39347729 size=6188 (idx=1173) would collide with offset=39224761 size=98348; renamed to __erfdup__1173.tga
    W, [2019-12-31T18:20:44] -- nwsync_write: Duplicate resref entry in erf, but differing offset/size: resref=m_crowishwizard_.tga offset=39353917 size=1580 (idx=1174) would collide with offset=39224761 size=98348; renamed to __erfdup__1174.tga
    W, [2019-12-31T18:20:44] -- nwsync_write: Duplicate resref entry in erf, but differing offset/size: resref=m_wisemalewizard.tga offset=44722885 size=24620 (idx=1336) would collide with offset=44624537 size=98348; renamed to __erfdup__1336.tga
    W, [2019-12-31T18:20:44] -- nwsync_write: Duplicate resref entry in erf, but differing offset/size: resref=m_wisemalewizard.tga offset=44747505 size=6188 (idx=1337) would collide with offset=44624537 size=98348; renamed to __erfdup__1337.tga
    W, [2019-12-31T18:20:44] -- nwsync_write: Duplicate resref entry in erf, but differing offset/size: resref=m_wisemalewizard.tga offset=44753693 size=1580 (idx=1338) would collide with offset=44624537 size=98348; renamed to __erfdup__1338.tga
    W, [2019-12-31T18:20:44] -- nwsync_write: Duplicate resref entry in erf, but differing offset/size: resref=po_hu_m_oarcher_.tga offset=102731633 size=24620 (idx=3111) would collide with offset=102633285 size=98348; renamed to __erfdup__3111.tga
    W, [2019-12-31T18:20:44] -- nwsync_write: Duplicate resref entry in erf, but differing offset/size: resref=po_hu_m_oarcher_.tga offset=102756253 size=6188 (idx=3112) would collide with offset=102633285 size=98348; renamed to __erfdup__3112.tga
    W, [2019-12-31T18:20:44] -- nwsync_write: Duplicate resref entry in erf, but differing offset/size: resref=po_hu_m_oarcher_.tga offset=102762441 size=1580 (idx=3113) would collide with offset=102633285 size=98348; renamed to __erfdup__3113.tga
    

    EDIT: Re-ran NWSync on old working-version as well (keeping separate NWSync data folders) and those lines appear on it as well. So don't think that's the issue?
    Post edited by jpsweeney94 on
  • WilliamDracoWilliamDraco Member Posts: 175
    edited January 2020
    Are you sure you are copying everything that nwsync creates to the server? Specifically are you also updating the new 'latest' or the new manifests subfolder? Something on the server isn't matching up with what NWSync is creating locally. It might even be the case where nwsync has deleted things in it's process and the server hasn't removed those - I thought that wouldn't be an issue but maybe.

    As to the 'Duplicate resref entry' notices: They are just telling you that your source files (haks) have the same file multiple times. And not only are they there multiple times, but they are DIFFERENT each time. The game has always handled this by just using the 'newest' one according to hak-ordering rules. NWSync is throwing those warnings so that you know, but it's not a 'problem' unless you were expecting to use both of the originals separately.

    You should go see about cleaning those up though, by removing all the unused duplicates that it mentions.

    As to Shadguy's comment: It's possible to run nwsync directly on the server if it allows any kind of remote terminal control. If yours is just ftp, probably no such luck.
    jpsweeney94
  • jpsweeney94jpsweeney94 Member Posts: 93
    thanks for the reply @WilliamDraco

    Using your NWSync tool, i target my "nwsyncarda" folder and i upload that entire folder to my web-server. I even tried a fresh NWSync write to see if it was old files or something, and same thing.

    I'm assuming it's something to do with the 1st screen shot i posted? Im not sure why it would be replacing files on a new folder upload. Or why the same file would be uploaded twice. Or why one has 0 bytes. Wonder if it's my FTP? Maybe ill try a different one.
    As to Shadguy's comment: It's possible to run nwsync directly on the server if it allows any kind of remote terminal control. If yours is just ftp, probably no such luck.

    Oh, cool. Ill have to look into this, would be nice. Would i be able to see the web-server directory in windows explorer? Thinking so i can target it with your tool.
  • jpsweeney94jpsweeney94 Member Posts: 93
    edited January 2020
    Turns out the issue was the FTP i was using (FileZilla) -_- .... Switched to Cyber Duck and all is well. Thanks
  • jpsweeney94jpsweeney94 Member Posts: 93
    Is https support still planned on being added to NWSync?? If so, would google drive be supported? lol would be awesome to not have to upload entire repo every time and just the changes with google drive on desktop...
  • WilliamDracoWilliamDraco Member Posts: 175
    edited February 2020
    https support was added to NWSync with 8193 - Both http and https are currently supported.

    As to Google drive... I doubt it that will ever happen. Maybe try to search for a ftp client that does differential uploads?
    Post edited by WilliamDraco on
    jpsweeney94
  • jpsweeney94jpsweeney94 Member Posts: 93
    edited February 2020
    https support was added to NWSync with 8193 - Both http and https are currently supported.

    As to Google drive... I doubt it that will ever happen. Maybe try to search for a ftp client that does differential uploads?

    @WilliamDraco Thanks for the suggestion. I finally managed to "mount" my FTP/server folder to my desktop in windows 10 through "add a network location" when right clicking on "This PC" in windows explorer. I did some small tests and it does work for adding/deleting files.

    However, when i try to select the network folder in your NWSync GUI program, it doesn't show up in the choose destination folder dialogue. Any ideas why? It does show up in choose source folder dialogue but not the destination one for some reason.
  • WilliamDracoWilliamDraco Member Posts: 175
    Probably a question more well suited to the github page for the GUI, rather than this thread about the CLI program itself.

    While here though: It must be a bug/issue in the library I'm using to build the GUI. Difference being that the Source is a file, while Destination is a folder, so they are different elements. I'll follow-up but it's not likely a matter I can directly resolve.
    jpsweeney94
  • piraopiropiraopiro Member Posts: 1
    I have this bug, tryed almost everything, dowloading the game again, deleting the folder, tryed configuring the firewall, copy the file from another player's PC, but does not solve!
    3wky6unq6qit.png
  • milormilor Member Posts: 5
    FIXABLE?

    neverwinter@host:~/nwsync$ ./nwsync_write -v --description="HOST Server Data V20200329" ~/www/ ~/.local/share/Neverwinter\ Nights/hak/host_v3.0.0.hak
    I, [2020-03-29T21:44:59] -- nwsync_write: Reindexing
    I, [2020-03-29T21:44:59] -- nwsync_write: Adding hak: /home/neverwinter/.local/share/Neverwinter Nights/hak/host_v3.0.0.hak
    I, [2020-03-29T21:44:59] -- nwsync_write: Preparing data set to expose
    E, [2020-03-29T21:44:59] -- nwsync_write: ResRef README.0 is not resolvable (we don't know the file type); origin: Erf:/home/neverwinter/.local/share/Neverwinter Nights/hak/host_v3.0.0.hak
  • WilliamDracoWilliamDraco Member Posts: 175
    edited March 2020
    Error is as it says on the tin. ",0" is not a supported filetype.
    See this file for all the supported filetypes: https://github.com/niv/neverwinter.nim/blob/master/neverwinter/restype.nim

    A readme is useless to deliver as part of nwsync. I'd therefore recommend removing it from the hak.

    If you are super desperate to keep it together for some reason, you can try using a text-like format that is supported, like .nss. Better to find an alternative method of storing the documentation, though.
  • milormilor Member Posts: 5
    edited March 2020
    Error is as it says on the tin. ",0" is not a supported filetype.
    See this file for all the supported filetypes: https://github.com/niv/neverwinter.nim/blob/master/neverwinter/restype.nim

    A readme is useless to deliver as part of nwsync. I'd therefore recommend removing it from the hak.

    If you are super desperate to keep it together for some reason, you can try using a text-like format that is supported, like .nss. Better to find an alternative method of storing the documentation, though.

    Thank You, I was doing just that when I read you.
    I had to remove README.res

    Now to continue with the guide ...

    Can a LOG be activated?
  • milormilor Member Posts: 5
    Do you have Round-robin or will you?
  • dunahandunahan Member Posts: 139
    Now that my world is on EE finally, I tried to set up nwsync. With the GUI that provides @WilliamDraco it is makeable.

    I stumbled upon some things, that can be a little cumbersome... I opened a issue at github for the help message window and a message window for errors.

    Because, at my first use, it didn't do its job and I didn't realized anything... Okay, later I took a look in the destination folder and saw the log with the error. I suggested to show a message, that something didn't work and to take a look at the log.
    The other thing I mentioned is, the help message window is a bit oversized. How about resizing or make it resizable?

    Next, two things I want to ask. Before I start, I run the task not from my server, i took my working machine. ;)
    First, I set up NWN:EE to use my old hak/tlk folders so I don't have to copy them to my user folder (I use more than one user on my Win10 laptop). The sync didn't realized that. I think that's how it should work?
    Second one: I have a huge amount of haks (mostly the CEP) and the write process is extremely slow. I think 10% after an hour.
    If I stop the process, would the sync begin at the start, or when I stopped it?

    Thanks for that program and the GUI. And of course, hopefully the answers to my questions.

    dunahan
  • WilliamDracoWilliamDraco Member Posts: 175
    edited May 2020
    I've responded to your github issue there - Thanks :)

    As to the other matters.
    1) My understanding was that nwsync should follow alias' in its resman the same way that the game does, although I've never tested that myself. That'd be something for Niv to comment on/resolve if so, as the GUI simply prepares the command and hands it off to nwsync to follow.

    2) There's some combination of system components that does cause a really slow write process, especially using a hard-drive (rather than SSD) but that doesn't seem to always be the core problem. For instance, I've prepared 12GB of haks->nwsync in ~30 minutes, but some report 1GB taking hours and hours...
    EDIT: I also remember: Any kind of virus-scanning program that might be actively scanning every file NWsync creates will make it excruciatingly slow, as it creates a *LOT* of files to scan. Temporarily disable any such 'real-time scanners'
    On the positive side, you can stop and resume the process and it should (mostly) pick up where it left off.
    dunahan
  • dunahandunahan Member Posts: 139
    edited May 2020
    I think it is good to talk about the suggestions on github. I also gave you more info there ;)

    For my question one, I will wait till niv can answer it. For now I'm using my user folder and delete those haks later, when the sync is finished.

    For number two, I think I'll do more tests. The most users will have CEP, so I will use a module with only the haks we provide... Then the down-/upload should be smaller.

    For the other reasons of a slowdown...
    Hmm, I use it on my HDD (only one). So it reads and writes at once (I had yet no success, to get win10 connected with my webspace).
    The only virus scanner I use, is the built in from win10. Don't know, if it is scanning the whole time...

    At least, I tried to start over where I was last night, but it began from the beginning. Okay, that could be a typical beginners problem :D Won't stop me! I got World Gate to run, then that sync will be possible, too.

    Edit:
    At least with that reduced hak-mod, about 3GB, it was fast. About one hour only. So indexing the CEP slowed down the process. Now it's time to upload the roundabout 45.000 files o.O XD
    Post edited by dunahan on
  • ValborValbor Member Posts: 1
    edited August 2020
    Hi, here is a video guide for setup NWSync if anyone else needs it:

    dunahan
  • dunahandunahan Member Posts: 139
    mrtwonder wrote: »
    I am getting a 503 time out error for my server, 'Town of Luce'. *update, it only seems to do it on some computers. It might be an error on the http server side as well.

    @niv I activated NwSync with my PW but most of my players do have problems with it. Some say, it even won't start to download or like me, I got this 503 timeout error...

    How does NwSync save the data actually? Doesn't it look at the hak-folder, if the data exists? I created the sync data on the same machine I develope and play. The data is the same as that on the server. Only, due file limit on my webserver, I removed CEP from sync. Could that be a problem?
  • WilliamDracoWilliamDraco Member Posts: 175
    The 503 timeout error is a http webserver thing, Not nwsync. Maybe your webserver is mis-understanding all of nwsync's requests as some sort of DOS attack and putting up temporary blocks or something. Whatever the case, it's something to sort out with your host and server, not nwsync itself.

    As to the second part, publishing an nwsync repo REPLACES haks, it is not in addition to. If you use CEP content, you must include CEP content in the nwsync repo or else it will not be loaded by clients.
    UNLESS
    you use the additional server flag to also publish the list of haks. per the docs
    -nwsyncpublishhaks
    This optional switch is purely for advanced users. Normally, once a server is told to advertise a NWSync URL, it will stop sending the list of required haks and tlk to clients (otherwise, clients would attempt to load them in addition to the nwsync manifest). If you have a very special setup, you can override this behaviour with this switch.
    For normal operation, it is strongly advised to not do this. Content should be in NWSync, otherwise it cannot deduplicate.
    Again though, that's not recommended and is a pathway to strange results.
    dunahan
  • dunahandunahan Member Posts: 139
    @WilliamDraco Ah , I understand. Okay, then I will have to switch to another option. Because my webspace is limited to a specified number of files and with CEP its run out of files...

    Then yet, NwSync isn't working for our PW, without problems. Thank you for your explanations!

    Now I think about using a old tool from 1.69 times, named WorldGate that worked relative simple and alike NwSync. Only disadvantage is, it has to run before the Client is started, not from within it.
  • RhamnetinRhamnetin Member Posts: 3
    Has anyone ever had this issue when setting up nwsync on Windows? For some reason it can't add my TLK file for this one module, but there's no such issue with other modules.

    Error: unhandled exception: 'custom.tlk' is not a resolvable resref [ValueError]
Sign In or Register to comment.