Help Setting up External Script Compiler

Balanor · December 2017

Since it appears we're back to compiling large/complex scripts outside of the toolset in EE (hopefully for just the time being), I'm trying to do a full build of all scripts in my EE module using either the Advanced Script Compiler or the PRC Neverwinter Script Compiler. However, I can't remember how to get these to run, and after quite a bit of internet and Vault searching it appears documentation on exactly how to setup either of these is non-existent, or else very well hidden.

I know we need to setup a .bat file to run either of them, but I can't find what the parameters are that are needed and/or if it does really need to be run in the temp0 folder, etc. Can someone please pretend I am a 5 year old (or perhaps just a newbie to NWN ) and walk me through the steps needed to get either of these compilers to work? Or just provide an example of either that works?

shadguy · December 2017

Here's a sample run script we use with SkyWing's compiler ( https://neverwintervault.org/project/nwn2/other/tool/advanced-script-compiler-nwn2 ):

---
pushd modules\temp0
..\..\utils\StandaloneCompiler\NWNScriptCompiler.exe -1glo -i ..\..\dnhak -i ..\..\override *.nss
popd
---

Details may vary depending on where you stick the file, but you can see we do push into temp0 to compile. We also include some files from a HAK build dir and the override dir; your needs may vary there.

I agree, the options are a bit of a pain to sort through.

-Dave

Gorgon · December 2017

Advanced Script Compiler for NWN1 release thread has the old topic about it, with some examples. Most will work if you avoid using the -n option to specify the game install folder (e.g., it can't find tlks without remapping both locations for them).

If you have the NWNScriptCompiler.exe in your %USERPROFILE%\\Documents\\Neverwinter Nights directory, and a module open in the toolset, the following should work for a general .bat/.cmd file (use .cmd if you can):

@ECHO OFF
set mask="%1*.nss"
CD "%USERPROFILE%\\Documents\\Neverwinter Nights\\modules\\temp0"
"%USERPROFILE%\\Documents\\Neverwinter Nights\\NWNScriptCompiler" -1ceglo -r . %mask% 
CD /d %~dp0
PAUSE

Adjust path locations to suit your needs.

You can add the -a option if you want it to analyze generated code and verify that it is consistent, but it takes a tad longer (not much on new hardware).
The -c option is the default "compile", so not really needed, and the -e is to enable non-BioWare extensions (up to you if you want/need them).
I don't know if the -h homedir setting for NWN2 is useful here (or ignored), but you can experiment with it, since EE now uses one.
The -v1.69 option will run like the NWNTX version, and fix minor errors when compiling, instead of halting with an error, but I never would recommend that. If it is broken, it should be fixed at the source.
Run NWNScriptCompiler -? for a full listing of command line options and their meanings.

I sometimes find it useful to pipe the output to a log file, by adding ' > "path\\to\\compiler_log.txt" ' at the end of the command line (or using the UnixUtil tee.exe port for win, to get both console and log output). You can set up all sorts of complicated include paths, and different ways of doing your compiling, so have fun experimenting.

*edits* I forgot to mention you can use the command line (or call it from another file), to compile only specific prefixed scripts (that is the %mask%). If you made some system where all the scripts began with "pud_", and you called that batch file above "compile_ee.cmd", then you could type compile_ee pud_ to only compile those scripts in the module.

Balanor · December 2017

Thanks guys, for the replies. Gorgon, your example is just what I needed. Hopefully this helps out other people looking to get modules converted to 1.74 right now.

fot1 · January 2018

Thanks to @Gorgon post, I almost got it working for me. But I'm stumble on some error, and any help would be appreciate! Let me explain what I have done:

When I do not specify the `-n`, I got an the error "'Unable to open NWN1 registry key". It is on this file:
https://github.com/SkywingvL/nwn2dev-public/blob/master/NWNScriptCompiler/Main.cpp#L555

I looked at my registry for Neverwinter Nights and no luck. I only found registry keys for beamdog installer. And the only way to the code not calling this function is passing the -n option:
https://github.com/SkywingvL/nwn2dev-public/blob/master/NWNScriptCompiler/Main.cpp#L2794

So I tried to experiment with that, and after messing a bit around, I found that if I copy the nwscript.nss, which is present on the beamdog directory `data/ovr` to my `tmp0` folder, I could compile some files. Great success! But no. Because it fails to find any included .nss that I have not edited, and that is all original untouched .nss files.

I also got some .tlk errors, but should be ok by now because I'm not using any custom one. This is the line I'm using to compile:


:: "%USERPROFILE%\\Documents\\Neverwinter Nights\\NWNScriptCompiler" -1cglo -n "%USERPROFILE%\\Documents\\Neverwinter Nights" -r . %mask%

So, my questions are:

1) Is there a way to make the compiler to look the original .nss somehow?
2) Is there a better way to make it use the `nwscript.nss` than manually copy to the temp0 directory?
3) Any chance I'm using the wrong compiler binary? (I haven't compiled the compiler code, I downloaded it form here: https://neverwintervault.org/project/nwn2/other/tool/advanced-script-compiler-nwn2).

fot1 · January 2018

Thanks to @Gorgon post, I almost got it working for me. But I'm stumble on some error, and any help would be appreciate! Let me explain what I have done:

When I do not specify the `-n`, I got an the error "'Unable to open NWN1 registry key". It is on this file:
https://github.com/SkywingvL/nwn2dev-public/blob/master/NWNScriptCompiler/Main.cpp#L555

I looked at my registry for Neverwinter Nights and no luck. I only found registry keys for beamdog installer. And the only way to the code not calling this function is passing the -n option:
https://github.com/SkywingvL/nwn2dev-public/blob/master/NWNScriptCompiler/Main.cpp#L2794

So I tried to experiment with that, and after messing a bit around, I found that if I copy the nwscript.nss, which is present on the beamdog directory `data/ovr` to my `tmp0` folder, I could compile some files. Great success! But no. Because it fails to find any included .nss that I have not edited, and that is all original untouched .nss files.

I also got some .tlk errors, but should be ok by now because I'm not using any custom one. This is the line I'm using to compile:

:: "%USERPROFILE%\\Documents\\Neverwinter Nights\\NWNScriptCompiler" -1cglo -n "%USERPROFILE%\\Documents\\Neverwinter Nights" -r . %mask%

So, my questions are:

1) Is there a way to make the compiler to look the original .nss somehow?
2) Is there a better way to make it use the `nwscript.nss` than manually copy to the temp0 directory?
3) Any chance I'm using the wrong compiler binary? (I haven't compiled the compiler code, I downloaded it form here: https://neverwintervault.org/project/nwn2/other/tool/advanced-script-compiler-nwn2).

fot1 · January 2018

Thanks to @Gorgon post, I almost got it working for me. But I'm stumble on some error, and any help would be appreciate! Let me explain what I have done:

When I do not specify the `-n`, I got an the error "'Unable to open NWN1 registry key". It is on this file:
https://github.com/SkywingvL/nwn2dev-public/blob/master/NWNScriptCompiler/Main.cpp#L555

I looked at my registry for Neverwinter Nights and no luck. I only found registry keys for beamdog installer. And the only way to the code not calling this function is passing the -n option:
https://github.com/SkywingvL/nwn2dev-public/blob/master/NWNScriptCompiler/Main.cpp#L2794

So I tried to experiment with that, and after messing a bit around, I found that if I copy the nwscript.nss, which is present on the beamdog directory `data/ovr` to my `tmp0` folder, I could compile some files. Great success! But no. Because it fails to find any included .nss that I have not edited, and that is all original untouched .nss files.

I also got some .tlk errors, but should be ok by now because I'm not using any custom one. The line I'm using to compile is as follows:

"%USERPROFILE%\\Documents\\Neverwinter Nights\\NWNScriptCompiler" -1cglo -n "%USERPROFILE%\\Documents\\Neverwinter Nights" -r . %mask%

So, my questions are:

1) Is there a way to make the compiler to look the original .nss somehow?
2) Is there a better way to make it use the `nwscript.nss` than manually copy to the temp0 directory?
3) Any chance I'm using the wrong compiler binary? (I haven't compiled the compiler code, I downloaded it form here: https://neverwintervault.org/project/nwn2/other/tool/advanced-script-compiler-nwn2).

fot1 · January 2018

I was not being able to make it work. Checking the source code for the error message, I discovered that the compiler reads some keys on the windows registry. The beamdog NWN don't seems to have similar registry keys, and I didn't had anything else installed, since I was setting up this box only to do neverwinter nights EE related stuff. So I tried and installed my original nwn from GOG, and then it worked.

So be warned: It is possible that the compiler is taking resources from the original neverwinter nights, and it while it works now, it might not work on the future/not inlcude beamdog improvements.

Gorgon · January 2018

That explains why some people had the same problem with it.

Rather than using the -l option to load base game resources, you might need to use -i and include the ovr directory (contains the latest nwscript.nss for now, but I'm assuming it'll only be in the bif files later). Then also extract all the scripts from the data\base_scripts.bif somewhere and include them as well (and continually update them with every version change that modifies them, so a PITA).

I'll give it a try in a clean Win 10 VM later, with only EE installed, and see what happens. Hopefully the compiler will get an update for EE eventually.

fot1 · January 2018

Thanks to @Gorgon post, I almost got it working for me. But I'm stumble on some error, and any help would be appreciate! Let me explain what I have done:

When I do not specify the `-n`, I got an the error "'Unable to open NWN1 registry key". It is on this file:
https://github.com/SkywingvL/nwn2dev-public/blob/master/NWNScriptCompiler/Main.cpp#L555

I looked at my registry for Neverwinter Nights and no luck. I only found registry keys for beamdog installer. And the only way to the code not calling this function is passing the -n option:
https://github.com/SkywingvL/nwn2dev-public/blob/master/NWNScriptCompiler/Main.cpp#L2794

So I tried to experiment with that, and after messing a bit around, I found that if I copy the nwscript.nss, which is present on the beamdog directory `data/ovr` to my `tmp0` folder, I could compile some files. Great success! But no. Because it fails to find any included .nss that I have not edited, and that is all original untouched .nss files.

I also got some .tlk errors, but should be ok by now because I'm not using any custom one. This is the line I'm using to compile:

:: "%USERPROFILE%\\Documents\\Neverwinter Nights\\NWNScriptCompiler" -1cglo -n "%USERPROFILE%\\Documents\\Neverwinter Nights" -r . %mask%

So, my questions are:

1) Is there a way to make the compiler to look the original .nss somehow?
2) Is there a better way to make it use the `nwscript.nss` than manually copy to the temp0 directory?
3) Any chance I'm using the wrong compiler binary? (I haven't compiled the compiler code, I downloaded it form here: https://neverwintervault.org/project/nwn2/other/tool/advanced-script-compiler-nwn2).

Gorgon · January 2018

fot1 said:
When I do not specify the `-n`, I got an the error "'Unable to open NWN1 registry key". It is on this file:
https://github.com/SkywingvL/nwn2dev-public/blob/master/NWNScriptCompiler/Main.cpp#L555

As that link shows, it is trying to find: HKEY_LOCAL_MACHINE\SOFTWARE\BioWare\NWN\Neverwinter

Beamdog doesn't use that (they have very different registry settings than the old BioWare ones). Using the -n option will either break things, or use old 1.69 install files (the -l option is the same, so thanks for pointing that out). The -h option won't work either, since it only looks for the NWN2 user folder.

--

I haven't had time to mess with this (meaning I could be totally wrong), but I am thinking the compiler isn't going to work well unless you extract all your required resources from haks and the base EE content now, until the compiler gets updated for EE. Then you can include them in order, using the -i option (and the data\ovr folder to get nwscript.nss until they toss it back in a bif). You may also have to include tlk locations, and whatever other resources it wants from your user/install directories.

The NWN CLI utilities can extract everything to a single folder with one command:
nwn_key_unpack NWN_ROOT\data\nwn_base.key target\directory\path
*Change NWN_ROOT and target\directory\path to fit your setup (thanks to @Sherincall for that tip in Discord BTW ).
That's overkill with it being around 2.24GB, but you can include just the base_scripts.bif and other folders you need (also include your regular and custom tlk path).

The neverwinter_utils.nim release page has compiled versions, so you don't have to do it yourself.

If you have 1.69 installed as well, you could probably keep using it the normal way from the EE toolset, and just include the base_scripts.bif files and nwscript.nss from ovr (and possibly the new dialog.tlk and your custom one). If you have any haks with scripts to include, then you may also need them extracted, unless you are sharing the same hak folder between EE and 1.69 (I don't know if it looks for haks relative to the modules\temp0 folder, one of the nwn.ini aliases, or uses the install path, so that might be worth checking).

If you do try this, you might want to use the -j option though, and verify where files are being included from, just to be sure of the order things are being loaded in. Personally, I'd stick with extracting everything and including it, rather than messing with it trying to load base game resources. I know it is a PITA with weekly updates, but better to be sure.

If someone else knows a better way, then I'm happy to hear it.

Glorwinger · January 2018

I have a working prototype of the nwnnnscomp compiler running native on my Mac for NWN EE. I need to clean up a few command line options and it stumbles when reading the NWN EE bif - it finds th scripts but is giving me a null. I unpacked everything and it runs so once I debug that issue I will puch code that should provide native compiler for Win / Mac / Linux... I should have time today to finish but I think byt the weekend I can wrap up a version.

dimecoin · January 2018

I was able to get it working with this:


@echo off

set mask="%1*.nss"

set INSTALL_DIR="%USERPROFILE%\\Documents\\Neverwinter Nights"
set SRC_DIR="%USERPROFILE%\\Documents\\Neverwinter Nights\\modules\\temp0"
set COMPILER=G:\NWN\StandaloneCompiler\NWNScriptCompiler.exe
set INCLUDE=G:\Beamdog\00840\ovr\

cd /D %SRC_DIR%
del /F /Q *.ncs

echo --------------
echo.

%COMPILER% -1cglo -i %INCLUDE% -n %INSTALL_DIR% -r . %mask%

echo.
echo --------------
dir *.ncs

cd /d %~dp0

Gorgon · January 2018

That still doesn't load base game scripts for me.

Try this simple script in a module, and test compile it in the toolset. Then try it with your batch file.

#include "x2_inc_switches"

void main()
{
    SetExecutedScriptReturnValue();
}

It compiles in the EE toolset, but gives an error without including the extracted scripts from the base_scripts.bif:
Error: NSC1085: Unable to open the include file "x2_inc_switches"

Try extracting the base_scripts.bif to a base_scripts folder, then adding it to your include line, like so (use whatever folder names you like):
set INCLUDE="G:\\Beamdog\\00840\\ovr";"G:\\NWN Stuff\\base_scripts"
The trailing backslash seems to be optional there, and using "quotes" is a good idea if you may have spaces in the path. Also the double backslashes \\ solve a few issues with special characters like leading underscores in names and so on (or use forward slashes /).

One other thing I did to see if it worked, is symlinked the "G:\\Beamdog\\00840\\data\\dialog.tlk" in the "%USERPROFILE%\\Documents\\Neverwinter Nights" location (my paths are different), which fixes:
WARNING: Exception 'Failed to open TLK file.' loading base TLK. STRREF lookup may be unavailable.

The -j options shows it is loading them from the correct haks and include locations, though it always gives priority to the -i locations over haks. That means if you want to override a base game script, it has to be in the module temp0 folder, or included before the base scripts in the -i list (first found is used in that list).

Something else to remember when compiling in the temp0 folder, is that it also is backwards for order when an include script is in both the module and a hak. The hak version should be the one used, but it includes the module version. This is probably because it will always use include files from your current folder, as if you included it with -i (see above). I've confirmed it by executing a simple script onload, that writes a log entry using a constant string value from an include file, then including that from both the mod and a hak using different values. The toolset compiler uses the hak version (which is correct for order of priority), but the external compiler run on temp0 uses the module version. If run from outside the temp0 folder, and the full path to temp0 is given for the -r option, then it includes the hak version correctly.
*Note* This behavior does make life easier when testing an updated include file intended for a hak (if you don't compile outside of temp0), as long as you are aware it is reversed from how the toolset would do it.

Deleting all .ncs files before every compile might not be for everyone, if they don't keep all of the source files in the module all the time, or do more complicated compiling, but whatever works for you.

*Edits* and if @Glorwinger can get it working in EE, all this won't matter, and we will be very happy.

Help Setting up External Script Compiler

Comments