[MOD] EEex (v0.6.0-alpha)

Bubb

Overview:

EEex is an executable extender for Beamdog's Enhanced Edition of the Infinity Engine. Its goal is to externalize certain parts of the engine to grant modders a greater degree of control over otherwise hardcoded mechanics. EEex does not make any gameplay changes itself - it merely enables other mods to do so.

Just give me the link!

Alpha versions of this project are now available on GitHub.

EEex will support MacOS and Linux on launch, however, the current alpha version only supports Windows. In addition, BG2:EE v2.5.16.6 is the only supported game at the time of writing.

What does it do?

EEex, when installed, directly modifies the game's executable in order to insert a loader. This loader is used to alter the game's code on startup; the modifications that are made depend on the version of EEex installed, and any installed mods that make use of EEex's capabilities.

I'm a user, what do I need to do to install?

EEex is installed just as any other WeiDU mod. Simply extract the archive's contents into your game's base folder, and run the setup - it will take care of the rest. Please note that since EEex alters the game executable itself, you should ensure that the game is closed before running the installer!

EEex is currently in alpha, and as such it is highly unstable. All immediate crashes are believed to be fixed, though you still risk a crash at any time. Use with care.

I'm a modder, how do I use EEex?

EEex makes changes to many different parts of the engine. Accessing and making use of new opcodes, actions, triggers, and objects is as simple as installing EEex.

The real power of EEex, however, comes from its ability to change hardcoded engine behavior. The hooks that enable these alterations are completely defined and controlled by the Lua environment, and as such, modders wishing to use these systems will have to either:

a) Insert a M_*.lua file into the override folder.
or
b) Edit UI.MENU directly.

Detailed documentation pending.



Previous post: [EEex v0.2.0-alpha] Who's up for some new LUA functions?

Edit: Alpha versions of this project are now available on GitHub. Documentation and post-overhaul pending™.

I have been working on creating some new LUA functions via hex editing for a mod I am currently working on, and it has been going very well so far. In case any of you are curious, here's the two new functions I've made so far:

Infinity_GetPCIdentifier(slot) - Gets the ID of the character in the given portrait slot. Can be used to access the characters table or in the following command...

Infinity_GetLocal(creatureID, local) - Returns the value of the given local variable on the creatureID. Can be used in conjunction with the previous command to get local values of PCs.

Currently in the works:

Infinity_SetLocal(creatureID, local, value) - Sets the value of the given local variable on the creatureID.

To give you guys some proof, here's a screenshot:


Here's my question to you all -
What new commands would you like to see? New UI behaviors are beyond me, but externalizing any other behavior the engine exhibits is definitely a possibility. I.E. any internal fields you would like to have access to, any existing function behaviors you would like changed.

Raduziel

I think that externalizing the amount of Fatigue would open a whole new world of Psionic-like mods. @subtledoctor and @Grammarsalad were requiring this, IIRC. Be able to see the current amount of Luck stacked would be nice too.

If possible, I would like to have some more information about Saving Throws in the combat log (bonuses, penalties and displaying the roll no matter if you failed or succeeded). It is painful to know if a modded effect is working or not because requires tons of trial-and-error to make a poor estimative (as you can see in a discussion here)

Aquadrizzt

Is it possible to have the UI aware of hte currently active character? For example, If I'm on Imoen, can the UI perform an Imoen specific action?

swit

All those 3 new functions will be very useful. Keep up the good work.

kjeron

If any of these are possible:
- Get value of characters stats (from STATS.IDS)
- Get is set of characters spellstate (from SPLSTATE.IDS)
- Get characters EA/GENERAL/CLASS/RACE/ALIGNMENT/GENDER/ALIGNMENT IDS index (as opposed to their current string references).
- Get Memorized spell table of characters.
- Get Known/memorized spell tables for innate abilities.
- MemorizeSpell(level,"resref"), as opposed to the current MemorizeSpell(level,index), same for Unmemorize.

Bubb

Raduziel said:

I think that externalizing the amount of Fatigue would open a whole new world of Psionic-like mods. @subtledoctor and @Grammarsalad were requiring this, IIRC. Be able to see the current amount of Luck stacked would be nice too.

@Raduziel: I believe Fatigue externalization would be covered by one of kjeron's broader requests: "- Get value of characters stats (from STATS.IDS)." Just to make sure, would this satisfy what would be necessary for Psionic mods, or is the ability to set Fatigue also required?

Raduziel said:

If possible, I would like to have some more information about Saving Throws in the combat log (bonuses, penalties and displaying the roll no matter if you failed or succeeded). It is painful to know if a modded effect is working or not because requires tons of trial-and-error to make a poor estimative (as you can see in a discussion

I agree that this would be very useful. I've already attempted to find the save checks in the exe, but they are currently eluding me. The way that they print the Save vs. x is delayed, meaning I can't just use the string to track the location of the check. I'll keep working on it.

Aquadrizzt said:

Is it possible to have the UI aware of hte currently active character? For example, If I'm on Imoen, can the UI perform an Imoen specific action?

@Aquadrizzt: Depends on what sort of action you are talking about. I believe either the currentID or id field already externalizes the character id of the currently selected party member, (or if multiple party members are selected, the one in the highest portrait slot). You can use this id to access the characters table, but not much else. Could you clarify what actions you are wanting to do in relation to a selected party member?

swit said:

All those 3 new functions will be very useful. Keep up the good work.

@swit: Thank you! . Me working on this hasn't stopped me from looking into the other features, btw.

kjeron said:

If any of these are possible:
- Get value of characters stats (from STATS.IDS)
- Get is set of characters spellstate (from SPLSTATE.IDS)
- Get characters EA/GENERAL/CLASS/RACE/ALIGNMENT/GENDER/ALIGNMENT IDS index (as opposed to their current string references).
- Get Memorized spell table of characters.
- Get Known/memorized spell tables for innate abilities.
- MemorizeSpell(level,"resref"), as opposed to the current MemorizeSpell(level,index), same for Unmemorize.

@kjeron: Very good list; they appear to all be possible. I'll get to work on them.

AstroBryGuy

I'm not familiar with LUA, but is this platform-independent bytecode, or are you talking about editing platform-dependent binaries?

Raduziel

@Bubb If you have the ability to set it, I don't see why not. The more options, the better.

Bubb

@AstroBryGuy: Currently I am directly modifying the Windows binary assembly. I believe I will have to find another way of modifying the Mac and Linux binaries, but I haven't looked at this yet. Now that you have brought up that point, I am doubting whether the way I am going about this is the best. I'll investigate how to edit the Mac and Linux binaries...

Aquadrizzt

@Bubb, mostly I'm wondering if it possible to customize user interfaces to different classes. For example, adding a shapeshift menu above the hotbar for druids, or a toggleable button for rage (or fighting styles). Basically, adding UI elements that are only present when the currently selected character would have access to them. Thinking about it further, being able to change the UI interface based on the class/race/whatever of the currently active character opens up the possibility of soft-coding the hotbar...

Bubb

I am still working on all of your suggestions, but just to update you guys I've made another very powerful function:

Infinity_DoString(chunk) - Runs whatever LUA code is embedded in the given string. Self-modifying code anyone?

Bubb

Here's an interesting development,

I was wondering if I could take my LUA endeavors further and integrate them into BCS scripts. And well, I did just that.



The above is a BCS script that prints "No, I'm sorry, none of them sound familiar." to the log if the player has their cursor over the second portrait slot.

Raduziel

This thread is like reading some medical researches: I have no idea why you're doing, but I'm glad you're being successful.

Grammarsalad

...Infinity_GetLocal(creatureID, local) - Returns the value of the given local variable on the creatureID. Can be used in conjunction with the previous command to get local values of PCs.

Oh, this is awesome!

Edit:

@Raduziel: I believe Fatigue externalization would be covered by one of kjeron's broader requests: "- Get value pointof characters stats (from STATS.IDS)." Just to make sure, would this satisfy what would be necessary for Psionic mods, or is the ability to set Fatigue also required?

I'm not sure about psionics-- you would need to ask @subtledoctor about that-- but I would be so super happy with @kjeron 's list. So +1 to that

subtledoctor

a) Not really interested unless it works on all platforms. (Don't mean to be hostile, it's just that I mod on a Mac and play on iOS.)

b) To the extent I am interested, I don't fully understand what is being proposed. Is this just for scripting stuff, or UI stuff, or something like that? Or are "new functions" the kind of thing that could turn into new opcodes or modified opcodes for spell effects?

Because, my wet dream for modding the IE is a spell effect that could retrieve and make use of global or local variables. (I never understood why there is an effect to set or modify variables, but no effect to retrieve them...)

Bubb

@subtledoctor:

a) My plan is to make these modifications on every binary that isn't locked-down. So, Windows, Mac, and Linux. The mobile versions cannot be modified to the extent I am doing. I am currently using the Windows binary as my base, and I will port over my changes to the other binaries when I am done.

b) Mostly everything. My original purpose of this thread was to outline new LUA functions, (those that pertain to the UI), but I have recently been branching out into other behaviors of the engine. For example, in my last post I showed that I extended the LUA environment into the scripting environment, basically allowing for dynamic scripts; almost like ToBEx's Assign() and Eval() scripting triggers / actions, but more powerful. I'm not completely done with this concept, but I'm getting there. As for new opcodes, I might be able to create some that mirrors other parts of the engine, such as variable getting and usage, yes.

Bubb

I've been working on some new scripting triggers / actions in an attempt to make AI scripting much more powerful, and I wanted to outline some of that here -

New triggers / actions (they are implemented as both):

Bubb_LUA(S:Chunk*) - Runs some LUA code provided by the given string. The trigger version of this checks the "trigger" LUA boolean after running the chunk, and succeeds if it is set to true.

Bubb_StoreObjectStat(S:Variable*,O:Object*,I:Stat*STATS) - Stores the provided stat from the given object in the defined LUA variable.

Bubb_StoreGlobal(S:Variable*,S:Global*) - Stores the given global into the defined LUA variable.

Bubb_StoreLocal(S:Variable*,S:Local*) - Stores the given local from the provided object into the defined LUA variable.

And perhaps the most important change of them all is the ability to dynamically override triggers / action parameters at runtime. This is accomplished by setting special LUA variables, here's an example of how it works:



The above script brings the XP value of the script owner to the exact value of Player1's XP.

I believe this system is good already, but I wanted to ask you all a question: is there any other triggers / actions you can think of that would help expand it even further? Any other information types you would like to be able to store... etc.

Edit: Oh, and I haven't ditched all of your UI suggestions. I'll implement them after I've finished my scripting stuff, (which I'm almost done with, btw).

Grammarsalad

Bubb said:

...
I believe this system is good already, but I wanted to ask you all a question: is there any other triggers / actions you can think of that would help expand it even further? Any other information types you would like to be able to store... etc.

...

One thing that would be helpful is an "exclusive or" trigger (since you're asking). That is, a trigger that works exactly like OR(x) but only returns true if 1 but not more than 1 of the following x lines is true.

Also, totally out of left field: spell exclusion flags:
http://gibberlings3.net/forums/index.php?showtopic=28382

Now, bit 14 excludes trueclass bards/sorcerers/mages. Unfortunately, unlike with mage kits, it also excludes bard kits. Would it be at all possible to make it only exclude trueclass bards? That one thing would allow me to create a mod with bard exclusive spells, which I think would be fun. Also, is it possible to make those exclusions that don't work, work?

Just figured I'd ask

swit

The above script brings the XP value of the script owner to the exact value of Player1's XP.

Just for perspective, the above posted code is an equivalent to fjxpmooc.bcs script from Level1 NPCs mod - originally 3000 lines (sic!). And those 3000 lines are not even close to match above precision. Bubb's lua stuff is revolutionary when it comes to EE engine scripting.

Grammarsalad

swit said:

The above script brings the XP value of the script owner to the exact value of Player1's XP.

Just for perspective, the above posted code is an equivalent to fjxpmooc.bcs script from Level1 NPCs mod - originally 3000 lines (sic!). And those 3000 lines are not even close to match above precision. Bubb's lua stuff is revolutionary when it comes to EE engine scripting.

I don't quite understand the significance, except that I'm excited that you're so excited. Lol

MoonWolf

What is the performance impact of one of your custom functions, especially the arbitrary lua evaluation compared to normal script functions ?

I realize that the EE engine on the whole is not nearly as prone to it but I'm still a little concerned that we might have the old stutter bugs again when a lot of these things are being called at once.

swit

I don't quite understand the significance, except that I'm excited that you're so excited. Lol

well, you've requested this trigger in your previous post:

One thing that would be helpful is an "exclusive or" trigger (since you're asking). That is, a trigger that works exactly like OR(x) but only returns true if 1 but not more than 1 of the following x lines is true.

it's not really needed because this is already possible with Bubb_LUA (Bubb, correct me if I'm wrong). For example this OR section:

OR(2)
	Global("LoveTalk","LOCALS",5)
	Global("HateTalk","LOCALS",10)

can be rewritten like this to work as you requested:

Bubb_StoreLocal("myVar1",Myself,"LoveTalk")
Bubb_StoreLocal("myVar2",Myself,"HateTalk")
Bubb_Lua("if myVar1 == 5 and myVar2 == 10 then trigger = false elseif myVar1 == 5 or myVar2 == 10 then trigger = true else trigger = false end")

trigger = false means that the Bubb_Lua will return false and end the block. For easier understanding here is the above Bubb_Lua trigger code with formatting instead of 1 line (it's normal lua code)

if myVar1 == 5 and myVar2 == 10 then
	trigger = false
elseif myVar1 == 5 or myVar2 == 10 then
	trigger = true
else
	trigger = false
end

Alternatively, if you're doing lots of checks like this in your scripts you can for example prepare M_*.lua file with pre-made function that will do above mentioned stuff and just call it like this instead of repeating the same code over and over:

Bubb_Lua("trigger = exclusiveOR(myVar1, 5, myVar2, 10)"

This is just an example, far more complicated stuff can be done with access to lua from within BCS.

Bubb

@Grammarsalad: I'll look into whether what you listed is possible.

@MoonWolf: I am watching performance very closely to make sure my new functions don't impact the game. You are correct in that I am currently having the engine compile and run the LUA chunk every time it executes it; not very efficient, but the engine actually does this itself in several places as well. To test performance I spawned 500 creatures running my script, and I felt no noticeable slowdowns:



If problems do occur with arbitrary LUA execution I can easily have a script shove all the LUA code into a M_*.LUA file, and change my Bubb_LUA() function to execute already compiled functions. This would mirror exactly how the GUI code works, so the game will run just as fast as it would without my functions in this scenario.

@swit: You are correct that an exclusive OR can be replicated with my Bubb_LUA() function, but I believe this would only work IF the triggers you are working with deal with fields that you can store. I believe @Grammarsalad wanted any combination of triggers to be dealt with in this manner. In this situation, I think a dedicated exclusive or is the only way. I'll look into it, but I don't know if I can accomplish that, as it has to do with changing how scripts are fundamentally processed.

Grammarsalad

Yes, so I'm clear, I'm definitely looking for an exclusive or for any combination of triggers (though, it's definitely not something I would want to greatly delay the wonderful work you are doing here. It's just A convenience/sanity thing.I can always duplicate the functionality in other ways, e.g.:
[Block 1]
OR(3)
A
-B
-C
...
[Block 2]
OR(3)
-A
B
-C
...
[Block 3]
OR(3)
-A
-B
C

Heh, it adds up

Edit: I'm really interested in the exclusion flag stuff, though...

Aquadrizzt

@Bubb, maybe I'm being dense, but does the xp example mean that the UI can now detect the stats of the currently selected character? Because that's revolutionary if it is the case...

Bubb

@Aquadrizzt: TLDR; Not by using the trigger / action in my example, but yes by using a dedicated LUA function.

The implementation you see in the XP example is a special trigger / action which could technically be used by the LUA environment by calling
C:Eval('Bubb_StoreObjectStat("xp",Player1,STATS.XP)')The problem with trying to use Eval is that any concept of the "currently selected party member" is nonexistent.

The good news is that I have all of the internal shenanigans to do with fetching stats done, so all I have to do is implement a LUA-side function which uses a Creature ID instead of a Object IDS value. This way, you could pass currentID into the function and get the stats of the selected party member. It would work like this:
local xp = Infinity_GetStat(currentID, STATS.XP)

Aquadrizzt

That's awesome and opens up so much opportunity with respect to class UIs. Cheers.

kjeron

swit said:

The above script brings the XP value of the script owner to the exact value of Player1's XP.

Just for perspective, the above posted code is an equivalent to fjxpmooc.bcs script from Level1 NPCs mod - originally 3000 lines (sic!). And those 3000 lines are not even close to match above precision. Bubb's lua stuff is revolutionary when it comes to EE engine scripting.

Just because I have a Bias (Opcodes > Actions):

OUTER_SET XP_BIT_SET = row# // Add entry to SPLPROT: row# 44 -1 8
OUTER_SET XP_BIT_NOT = row# // Add entry to SPLPROT: row# 44 -1 9
OUTER_SET SPLSTATE = ids# // Add a new one to use
COPY_EXISTING ~SPIN101.SPL~ ~override\XPMATCH.spl~ WRITE_LONG NAME1 ~-1~ LPF DELETE_EFFECT END
 FOR (k = 0; k < 24; ++k) BEGIN
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 326 target = 2 parameter1 = (2**k) parameter2 = XP_BIT_SET duration = 0 STR_VAR resource = EVAL ~XPBLCK%k%~ END
 END
 SET k -= 1
 LPF ADD_SPELL_EFFECT INT_VAR opcode = 326 target = 9 parameter1 = (2**k) parameter2 = XP_BIT_SET STR_VAR resource = EVAL ~XPBSET%k%~ END
 LPF ADD_SPELL_EFFECT INT_VAR opcode = 326 target = 9 parameter1 = (2**k) parameter2 = XP_BIT_NOT STR_VAR resource = EVAL ~XPBNOT%k%~ END
OUTER_FOR (i = 23; i >= 0; --i) BEGIN OUTER_SET j = i - 1
 COPY_EXISTING ~SPIN101.SPL~ ~override\XPBLCK%i%.spl~ WRITE_LONG NAME1 ~-1~ LPF DELETE_EFFECT END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 318 target = 2 duration = 1 STR_VAR resource = EVAL ~XPSETB%i%~ END
 COPY_EXISTING ~SPIN101.SPL~ ~override\XPMORE%i%.spl~ WRITE_LONG NAME1 ~-1~ LPF DELETE_EFFECT END
  FOR (k = 0; k < i; ++k) BEGIN
   LPF ADD_SPELL_EFFECT INT_VAR opcode = 318 target = 2 parameter1 = (2**i) parameter2 = XP_BIT_SET duration = 0 STR_VAR resource = EVAL ~XPSETB%k%~ END
  END
 COPY_EXISTING ~SPIN101.SPL~ ~override\XPBSET%i%.spl~ WRITE_LONG NAME1 ~-1~ LPF DELETE_EFFECT END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 318 target = 9 duration = 1 STR_VAR resource = EVAL ~XPSETB%i%~ END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 326 target = 2 parameter1 = 0 parameter2 = 0 STR_VAR resource = EVAL ~XPSETB%i%~ END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 326 target = 9 parameter1 = (2**j) parameter2 = XP_BIT_SET STR_VAR resource = EVAL ~XPBSET%j%~ END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 326 target = 9 parameter1 = (2**j) parameter2 = XP_BIT_NOT STR_VAR resource = EVAL ~XPBNOT%j%~ END
 COPY_EXISTING ~SPIN101.SPL~ ~override\XPBNOT%i%.spl~ WRITE_LONG NAME1 ~-1~ LPF DELETE_EFFECT END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 326 target = 2 parameter1 = SPLSTATE parameter2 = 111 STR_VAR resource = EVAL ~XPMORE%i%~ END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 326 target = 2 parameter1 = SPLSTATE parameter2 = 110 STR_VAR resource = EVAL ~XPLESS%i%~ END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 326 target = 9 parameter1 = (2**j) parameter2 = XP_BIT_SET STR_VAR resource = EVAL ~XPBSET%j%~ END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 326 target = 9 parameter1 = (2**j) parameter2 = XP_BIT_NOT STR_VAR resource = EVAL ~XPBNOT%j%~ END
 COPY_EXISTING ~SPIN101.SPL~ ~override\XPSETB%i%.spl~ WRITE_LONG NAME1 ~-1~ LPF DELETE_EFFECT END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 318 target = 9 duration = 1 STR_VAR resource = EVAL ~XPSETB%i%~ END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 104 target = 2 timing = 1 duration = 0 parameter1 = (2**i) END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 328 target = 2 duration = 1 parameter2 = SPLSTATE special = 1 END
 COPY_EXISTING ~SPIN101.SPL~ ~override\XPLESS%i%.spl~ WRITE_LONG NAME1 ~-1~ LPF DELETE_EFFECT END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 318 target = 2 parameter1 = (2**i) parameter2 = XP_BIT_NOT duration = 1 STR_VAR resource = EVAL ~XPLESS%k%~ END
  LPF ADD_SPELL_EFFECT INT_VAR opcode = 104 target = 2 timing = 1 duration = 0 parameter1 = (0 - 2**i) END
END

The target of "XPMATCH.spl" will have their XP increased (but not decreased) to match the casters XP.

Bubb

@kjeron: Wow, I didn't believe that was possible in the vanilla EE engines. Hopefully I am not wasting my time recreating too many things that can already be done...

Also, I've been working on some more scripting stuff:



The above script shows how objects can be stored and referred to using the LUA environment. The script doesn't really do anything other than having the script owner pick a random target once and then move to that object for all of eternity.

I believe this recreates IWD2's SetMyTarget action and MyTarget object.

swit

Just because I have a Bias (Opcodes > Actions):

@kjeron, while this is extremly impressive peace of code (like damn, I’m saving it right away - I’m sure something here will be useful for me in future) but the example XP matching code is something that can be written in a minute by anyone, while your opcode solution is a riddle that few people in the entire modding scene would be able to solve Let's say I'd like to match 75% of player1 XP for party members XP adjustment. With the above posted Bubb_LUA example code it would be a matter of adding:

Player1XP = Player1XP * 0.75

Doing the same with opcodes would be another riddle.

@Bubb, absolutely fantastic.

Cahir

Damn, I love you geeks talk about this alien code I absolutely have no idea about. But the fact you are all so excited about this, makes me feel equally excited. Cool mods may come to life because of this

*shivers*