Skip to content

BGEE-SoD Directory Structure

AstroBryGuyAstroBryGuy Member Posts: 3,437
edited April 2016 in BG:EE Mods
@Dee - I was doing some tests of BG1NPC on SoD, and it all seems to work find on the OSX version of SoD downloaded from Beamdog. Everything is integrated into the main game, with a moddable dialog.tlk file in the lang folder. But I'm seeing bug reports on that say installing on an SoD game from Steam does not work. (https://forums.beamdog.com/discussion/comment/730399/#Comment_730399)

Then, I saw this comment from @argent77 (https://forums.beamdog.com/discussion/comment/730863/#Comment_730863)
Important information for Steam users:
Beamdog decided to offer Steam users the "Siege of Dragonspear" expansion as a DLC in the form of a single zipped file named "dlc/sod-dlc.zip". All SoD-related resources have been included in the zip archive, including an expanded dialog.tlk which is used by the game instead of the one found in "lang/xx_YY". The current Near Infinity development snapshot can not access this content. As a result the game will be detected as plain "Baldur's Gate: Enhanced Edition" with the original resources of the base game. I'm currently working on a solution, but it requires to rework a major portion of NI's code base. I can't estimate when it will be done.
What's going on with the directory structure on SoD on Steam? Is the dialog.tlk file on Steam unmoddable now that it is tucked into a zip file? Is this going to affect GOG and Mac App Store too?
Post edited by AstroBryGuy on
Aedan
«1

Comments

  • tortuosittortuosit Member Posts: 63
    I think Beamdog should free the dlc content from the zip file.
    - For any dlc change (I guess) Steam needs to update the whole 2 GB file.
    - For any change I do, I need to repack (but with zip level 0). Its clunky.
  • BillyYankBillyYank Member Posts: 2,768
    Has anyone tried unzipping the zip file and dropping the contents in override?
  • The user and all related content has been deleted.
    Aedan
  • argent77argent77 Member Posts: 3,431
    edited April 2016
    You can already integrate the SoD content into the main game manually, but it's a pretty complicated process. I would wait for Beamdog or someone else to offer a better solution however.
    AstroBryGuyAedan
  • BillyYankBillyYank Member Posts: 2,768
    I almost feel like reporting it as a bug. "Game content should not be stuffed into a zip file that squats over the directory structure like a toad on a pie."
  • UlbUlb Member Posts: 295
    Just wanted to add that this doesn't just affect the Steam version, the GOG version also uses the .zip file.

    @CamDawg

    Can you guys weight in on this and whether we can expect any help/solution from Beamdog on this? As of right now SoD makes modding night impossible.. which is not a good thing for a game that thrives so much of of its modding community (as you well know).
  • CamDawgCamDawg Member, Developer Posts: 3,438
    edited April 2016
    BillyYank said:

    Has anyone tried unzipping the zip file and dropping the contents in override?

    Like @argent77 says, it's more complicated than that. He knows this stuff better than I do, so if I get something wrong below I hope he'll correct me.

    The IE has always kept its assets in archives known as biffs, e.g. data/ar06xx.bif will contain several files needed for ar0600 such as the tileset, search maps, weds, etc. while data/NPCSOUND.bif contains all of the sound files for dialogue, and so on and so forth. What file lives in which biff is delineated in a master key file, chitin.key. Even if you unpack the SoD zip file, chitin.key doesn't know about the assets in the SoD biff files since it just has regular BGEE assets. The zip file contains its own mod.key with that info.

    The rub is that mod.key lists only the SoD assets. So now you have an incomplete key (mod) while the main key (chitin) is no longer authoritative. You'd need to unzip everything, then use mod.key to dump everything out of the SoD biffs into the override. That would allow you to use the original chitin.key with all of the SoD stuff available via the override folder.

    And all that still doesn't address dialog.tlk issues.

    edit: Ulb posted while I was writing this. Hope it answers where we are right now.
    AstroBryGuy
  • UlbUlb Member Posts: 295
    CamDawg said:

    Hope it answers where we are right now.

    Ahm.. not very far it seems. :wink:

    Anyway, if I could tell weidu to leave a "gap" or a number of junk lines in the original dialog.tlk I could at least mod anything that doesn't directly touch SoD assets. (So kits and such theoretically work as long as we use dialog.tlk lines that are not already used in the mod dialog.tlk file, did I get that right?)

    AstroBryGuy
  • Sam_Sam_ Member Posts: 172
    CamDawg said:

    The rub is that mod.key lists only the SoD assets. So now you have an incomplete key (mod) while the main key (chitin) is no longer authoritative. You'd need to unzip everything, then use mod.key to dump everything out of the SoD biffs into the override. That would allow you to use the original chitin.key with all of the SoD stuff available via the override folder.

    I bet you could accomplish all of this manually with this WinBif, but I have SoD from Beamdog so I can't try it atm.
  • CamDawgCamDawg Member, Developer Posts: 3,438
    Ulb said:

    Anyway, if I could tell weidu to leave a "gap" or a number of junk lines in the original dialog.tlk I could at least mod anything that doesn't directly touch SoD assets. (So kits and such theoretically work as long as we use dialog.tlk lines that are not already used in the mod dialog.tlk file, did I get that right?)

    No, unfortunately. The problem with the tlk file is that, unlike other files, there is no override. If you play SoD on Steam, the one in the zip is the tlk file. If you put a new one in the lang or override folder the game will continue to use the one from the zip. I'm pretty sure you can pull it out of the zip, make your update, then rezip but I'll confess I'm not 100%.
    AstroBryGuyUlb
  • BillyYankBillyYank Member Posts: 2,768
    CamDawg said:

    Ulb said:

    Anyway, if I could tell weidu to leave a "gap" or a number of junk lines in the original dialog.tlk I could at least mod anything that doesn't directly touch SoD assets. (So kits and such theoretically work as long as we use dialog.tlk lines that are not already used in the mod dialog.tlk file, did I get that right?)

    No, unfortunately. The problem with the tlk file is that, unlike other files, there is no override. If you play SoD on Steam, the one in the zip is the tlk file. If you put a new one in the lang or override folder the game will continue to use the one from the zip. I'm pretty sure you can pull it out of the zip, make your update, then rezip but I'll confess I'm not 100%.
    Is that true of everything in the .zip? I tried to pull the bgee.lua out of the zip to the override, but it didn't pick up the changes I made.
  • AstroBryGuyAstroBryGuy Member Posts: 3,437
    CamDawg said:

    Ulb said:

    Anyway, if I could tell weidu to leave a "gap" or a number of junk lines in the original dialog.tlk I could at least mod anything that doesn't directly touch SoD assets. (So kits and such theoretically work as long as we use dialog.tlk lines that are not already used in the mod dialog.tlk file, did I get that right?)

    No, unfortunately. The problem with the tlk file is that, unlike other files, there is no override. If you play SoD on Steam, the one in the zip is the tlk file. If you put a new one in the lang or override folder the game will continue to use the one from the zip. I'm pretty sure you can pull it out of the zip, make your update, then rezip but I'll confess I'm not 100%.
    Is there a setting that tells the game to check the DLC for the zip, or is that just automatic? (i.e., if it sees a zip file in the DLC folder, it automatically uses it) One could extract the files from the zip, unbiff the BIF files to the override, replace the dialog.tlk file, and maybe some other stuff (new soundsets, movies, music, etc..). Then either alter the setting to ignore the DLC zip file, or if reading the file is automatic, move it to a backup location (dlc-bak or something) so the game won't see it.
  • BillyYankBillyYank Member Posts: 2,768

    CamDawg said:

    Ulb said:

    Anyway, if I could tell weidu to leave a "gap" or a number of junk lines in the original dialog.tlk I could at least mod anything that doesn't directly touch SoD assets. (So kits and such theoretically work as long as we use dialog.tlk lines that are not already used in the mod dialog.tlk file, did I get that right?)

    No, unfortunately. The problem with the tlk file is that, unlike other files, there is no override. If you play SoD on Steam, the one in the zip is the tlk file. If you put a new one in the lang or override folder the game will continue to use the one from the zip. I'm pretty sure you can pull it out of the zip, make your update, then rezip but I'll confess I'm not 100%.
    Is there a setting that tells the game to check the DLC for the zip, or is that just automatic? (i.e., if it sees a zip file in the DLC folder, it automatically uses it) One could extract the files from the zip, unbiff the BIF files to the override, replace the dialog.tlk file, and maybe some other stuff (new soundsets, movies, music, etc..). Then either alter the setting to ignore the DLC zip file, or if reading the file is automatic, move it to a backup location (dlc-bak or something) so the game won't see it.
    Wouldn't you also need to combine the mod.key with the chitin.key? Or is the mod.key like the dialog.tlk and contains the info the whole game?
  • The user and all related content has been deleted.
    AstroBryGuy
  • AstroBryGuyAstroBryGuy Member Posts: 3,437
    edited April 2016
    No. The chitin.key only needs to contains what's in the BIF files in the data directory. By un-biffing the BIF files in the DLC zip file and dropping them into the override, they don't need to be listed in the chitin.key.

    In theory, you'd need to do the following:

    1. Unzip DLC file
    2. Save current dialog.tlk to dialog.tlk.bak
    3. Replace dialog.tlk with DLC dialog.tlk
    4. unbiff .BIF files to override
    5. copy movies, sounds, and music to appropriate folders (and whatever else in the DLC zip)
    6. prevent the game from using the DLC zip file after extracting the files from it. So, disable it or move it.

    ninja'd by @subtledoctor
  • AstroBryGuyAstroBryGuy Member Posts: 3,437

    BillyYank said:

    is the mod.key like the dialog.tlk and contains the info the whole game?

    I don't think so, from what I've read. chitin.key + mod.key = the whole game. Either one, on its own, is incomplete.

    On the other hand, how about this procedure:
    1) Install BGEE 2.0
    2) Make a backup of the entire game
    3) Install SoD
    4) Extract the SoD game assets into /override (how to do this efficiently, I don't know)
    5) Extract dialog.tlk to /lang
    6) copy the /override and /lang folders to the BGEE backup
    7) Delete the BGEE+SoD game
    8) Restore the backup non-SoD BGEE game
    9) Run Generalized Biffing
    10) ... ? Now you have all of SoD in a non-SoD game? And chitin.key is aware of all the assets? And dialog.tlk is in the right place? And it won't look for that .zip archive, because SoD is technically not installed?

    Maybe?

    Probably not...
    I neglected to include Generalized Biffing. Good idea.

    I was hoping that maybe just moving the DLC zip somewhere (rename the directory to dlc-bak?) could make the game think the dlc isn't there and just use the resources in data and override. Or there's a flag in the .lua file to de-activate the DLC.
  • ALIENALIEN Member Posts: 1,269
    edited April 2016
    @Dee @Avenger_teambg I'm reading this thread and I'm shocked. BeamDog made huge mistake here or (it's even worse) ignore the whole problem from the start. You distribution model broke every mod and also BWS installer. There are no arguments which can support such disaster. For the love of Bhaal, please fix this!
    Post edited by ALIEN on
    So_LoWCaradocUlbAstroBryGuy
  • UlbUlb Member Posts: 295
    edited April 2016
    I have to agree with @ALIEN

    This is a really big deal.
    Beamdog just went ahead and completely broke modding for BGEE/SOD without any real need to do so.

    Even if a smart person from the community (like @argent77 ) would decide to step up and sacrifice his own time and talent to create a fix for this, handling the Steam/GOG versions would still always mean a lot of extra effort for modders and players alike.

    Beamdog claims to respect the modding community that has kept the BG series alive and well for more than the last decade. Yet, you have just made a huge unmistakable statement that says: "WE DO NOT CARE."

    Given that there are quite a few modders working for Beamdog and since Beamdog's own version (the one everyone else got to see during the beta) works perfectly fine, I have to assume that somehow this just slipped through under the radar. Maybe there were only a few coders involved on Beamdog's side that weren't really aware of modding? I certainly hope so.

    This doesn't just hurt the modding community, this will hurt Beamdog and this will be bad press, so you have to step up and do something about it ASAP.
  • argent77argent77 Member Posts: 3,431
    edited April 2016
    If you can't wait for an official solution to this mess you can try to integrate the SoD expansion into the main game manually. But be warned, it's a rather complicated process and prone to errors:
    [spoiler]
    1. Make a copy of the whole game directory without the "dlc" folder somewhere on your system (let's call it bgee_dir)
    2. Create a new directory somewhere on your system and unpack "dlc/sod-dlc.zip" into it (let's call it sod_dir)
    3. Rename mod.key to chitin.key in sod_dir
    4. Download the latest Near Infinity snapshot and unpack it into sod_dir
    5. Copy the characters, lang, movies and music folders from sod_dir to bgee_dir and accept to overwrite any existing files
    6. In sod_dir rename the folders characters -> bak-characters, movies -> bak-movies and music -> bak-music (to avoid exporting files from these folders)
    7. In sod_dir create a new empty folder "movies" and copy the file "sodcin01.wbm" from bak-movies into it (needed for Near Infinity to detect the correct game)
    8. Rename the home folder directory "Baldur's Gate - Enhanced Edition" to something else (to avoid exporting any files from there)
    9. Start Near Infinity in sod_dir
    10. Choose menu Tools -> Mass Export...
    11. Select output directory: bgee_dir/override (create the folder if it doesn't exist), untick any options and select all file types
    12. Press the "Export" button and hope for the best
    13. If the export process ended successfully you can close Near Infinity
    14. Go into bgee_dir/override and delete the file "sodcin01.wbm"
    15. Rename the home folder of the game back to "Baldur's Gate - Enhanced Edition"
    And that's it. The game in the new "bgee_dir" directory should be operational. Don't start it via Steam though. I'd imagine that Steam would try to "repair" it and revert everything back into the original state.

    Optionally, you can try to biff everything. However, WeiDU v239 doesn't recognize the new resource types yet and sets an incorrect type for WBM files.
    [/spoiler]
    Post edited by argent77 on
    AstroBryGuyUlbmf2112
  • GrunkerGrunker Member Posts: 46
    Hi guys, sorry for posting here, but I can't for the life of me figure out how to make a new thread?

    I'm new to the EEs but a BG-vet, and I want to play BG:EE + SoD. I'd like to, as a minimum, use Item Randomiser and and BG1NPC. I assume this whole folder debacle means that I currently can't since I have the Steam version?
  • AstroBryGuyAstroBryGuy Member Posts: 3,437
    edited April 2016
    Grunker said:

    Hi guys, sorry for posting here, but I can't for the life of me figure out how to make a new thread?

    I'm new to the EEs but a BG-vet, and I want to play BG:EE + SoD. I'd like to, as a minimum, use Item Randomiser and and BG1NPC. I assume this whole folder debacle means that I currently can't since I have the Steam version?

    You can play BGEEv2 for now and load SoD after beating Sarevok, then import your final save to SoD.

    Alternately, you can try @argent77's method above to create a BGEE+SoD that is moddable.
  • chilvencechilvence Member Posts: 50
    Argh, this situation is a total nightmare! Mods are the bread and butter of this series, I can't believe something so important would be overlooked :(
  • megamike15megamike15 Member Posts: 2,666
    it will be fixed at some point most likely with a patch that moves the tlk file.
  • chilvencechilvence Member Posts: 50
    It's just ... the principle? I'm a PC gamer, I've always been. That means 'Give me mods or give me death' . I can't be the only one? Even on my first play of SoD, I've been thinking every five minutes "that would be a good mod...." "that could be backported to BG1".... etc

    What exactly has kept the BG series alive and kicking for the last 18 years?
  • megamike15megamike15 Member Posts: 2,666
    edited April 2016
    chilvence said:

    It's just ... the principle? I'm a PC gamer, I've always been. That means 'Give me mods or give me death' . I can't be the only one? Even on my first play of SoD, I've been thinking every five minutes "that would be a good mod...." "that could be backported to BG1".... etc

    What exactly has kept the BG series alive and kicking for the last 18 years?

    I understand. the main reason i did not get the enhanced editions until last year was becuse i was waiting for all the mods i used to be compatible. now that the bg1's mods are broken [ including one i need to even go through the game] i'm kinda annoyed.

    but atlest [ i hope] bg2 is fine so i can just play sod and go right into bg2 until this is fixed.

    edit: anyway has any one reported this issue to beamdog yet? this is something that needs to be fixed asap.
  • chilvencechilvence Member Posts: 50
    It's ok, I am calm now :)

    Trying to make the most of this controversy, is it conceivable that this new system could be taken advantage of to make custom mod 'packs'?

    For example, if you wrap up a load of mods together, then zip them in a file with their respective dialog.tlk and mod.key, then you have almost every pre-requisite to being able to hot-swap them.

    If Beamdog play it right, this could be a new golden age of BG modding!

  • The user and all related content has been deleted.
  • elminsterelminster Member, Developer Posts: 16,315
    edited April 2016
    Can't comment on the QA process, but speaking personally I don't generally do much of my modding work using Steam because its default installation directory is a pain to get to (yea you can change it but all my other steam games are there so might as well keep them all together). The default is like Program Files/Steam/Steamapps/Common/Baldur's Gate Enhanced Edition

    (but if you are using something like Near Infinity you generally have to scroll down to get to the Steam folder and then you have to scroll down to get to the Steamapps folder).

    Beamdog's client/installer by comparison uses a much shorter path that is a lot easier to get to. Its just a time saver.

    (That said I'm not the person in charge of these sorts of distribution decisions. So I can't comment on anything about this being DLC)
  • UlbUlb Member Posts: 295
    edited April 2016


    Really, everyone complaining here should go to GOG and Steam and demand your money back. That might get their attention and lead to speedier cooperation.

    I disagree with you on that one. It is ultimately Beamdog's responsibility to make sure their game is in proper condition on all platforms it is sold. And there is no way this method of delivery was not at least co-developed and sanctioned by Beamdog.

    *edit* To clarify: I don't necessarily disagree about asking GOG/Steam for a refund (I'm only holding off on that for now until we get an official answer from Beamdog) but I disagree about the sentiment that this is all Steam/GOG's fault and Beamdog is not to blame.





    Yamcha
  • megamike15megamike15 Member Posts: 2,666
    asking for a refund is alittle extreme.

    there are only two ways to fix this problem.

    1. inform beamdog of the situation and wait for a hot fix and or patch that fixess this.
    2. wait for someone else to fix it with a mod or the newest version of weidu.
Sign In or Register to comment.