Skip to content

Creating a Modding Tool

klatuklatu Member Posts: 108
edited October 2012 in General Modding
Hey, y'all!

I am currently working on writing (or attempting to, anyway) a modding tool for BG. I'm still in the (middle/end)phase of establishing a solid basis for extracting resources from the source files.

There are a few things on which I would like to get some general thoughts from the community, though.

Regarding file formats: Is there anyone in the know on whether the basic file formats like TLK, BIF, DLG, ITM, etc... are likely to change or be altered for BG:EE? Since the devs seem to be using WeiDU a lot, I doubt it, but a heads-up might save me some headaches in the future.
(Extracting resources from Bifs is complicated enough considering the different compression methods for various versions, etc.)


I am planning on integrating a visual editor for writing dialogs. Instead of having to write WeiDU code by hand, the user would be given the option to stitch together and directly edit a sort-of flow-chart that represents the states and responses of different dialogs. The software would then be able to compile that structure or changes to it into WeiDU code.
However, considering how much control WeiDU offers in terms of dialog editing, I am a litle worried that making dialog assembly too simplistic/abstract will prevent that level of control.
On the other hand, I feel like a lot of talented would-be mod authors will never actually finish or even attempt to write dialogs because WeiDU can be really scary in its complexity. Not to mention that writing complex dialog trees is a pain in the ass.

I may have extensively meddled with my own BG games, but I am not a modder. To those of you who are:
Would such a feature be welcome? Does it even make sense? Any thoughts?
Or are there even already existing implementations of this idea?

Cheers;)
Post edited by klatu on
«13

Comments

  • MilochMiloch Member Posts: 863
    DLTCEP already works with BGEE, as does Near Infinity, so basic file structures have not changed. Dialog.tlk is in a different location; that's probably the biggest change. Of everything you mentioned, the most useful feature would be a visual editor for dialogues. I would be interested if you could actually do something like that. There is also Infinity Explorer but I am not sure if it does anything quite to the extent you're proposing.

    http://www.shsforums.net/topic/31285-infinity-explorer-v085/
  • klatuklatu Member Posts: 108
    OK, thanks @Miloch!
    Is there a specific reason for the relocation of the tlk?

    From the tools you mentioned I'm only really familiar with NI (heard of the others), and while it is powerful software, parts of it are pretty outdated by now. In the years since its creation even the Java standard for accessing files has changed extensively.
    And I am unsure whether newer compilations are any better in that regard.

    IE seems somewhat useful for reading dialogues, but I am not sure how useful it is for editing. I will look into it.

  • MilochMiloch Member Posts: 863
    @klatu - the game is delivered with language-specific dialog.tlks now, so en_US/dialog.tlk, fr_FR/dialog.tlk etc. NI is still somewhat maintained and DLTCEP is quite actively maintained.

    http://www.shsforums.net/topic/45358-nearinfinity/
    http://forums.gibberlings3.net/index.php?showforum=137
  • CuvCuv Member, Developer Posts: 2,535
    nice job!
  • CamDawgCamDawg Member, Developer Posts: 3,438
    edited October 2012
    klatu said:

    Regarding file formats: Is there anyone in the know on whether the basic file formats like TLK, BIF, DLG, ITM, etc... are likely to change or be altered for BG:EE? Since the devs seem to be using WeiDU a lot, I doubt it, but a heads-up might save me some headaches in the future.
    (Extracting resources from Bifs is complicated enough considering the different compression methods for various versions, etc.)

    Miloch's already posted the major change, which is the path change for dialog.tlk. The only other somewhat major change will be ogg support and the externalization of a few previously hardcoded features, but most of those aren't finalized.
    klatu said:

    I am planning on integrating a visual editor for writing dialogs. Instead of having to write WeiDU code by hand, the user would be given the option to stitch together and directly edit a sort-of flow-chart that represents the states and responses of different dialogs. The software would then be able to compile that structure or changes to it into WeiDU code.
    However, considering how much control WeiDU offers in terms of dialog editing, I am a litle worried that making dialog assembly too simplistic/abstract will prevent that level of control.
    On the other hand, I feel like a lot of talented would-be mod authors will never actually finish or even attempt to write dialogs because WeiDU can be really scary in its complexity. Not to mention that writing complex dialog trees is pain in the ass.

    Infinity Explorer always seems to be popular for non-modders to follow dialogue due to its tree structure (and I know it's been requested as a feature for DLTCEP). A better way to visualize complex dialogues is good (and a flow chart seems a really good idea) but I'm not sure how well that would translate into actual writing of dialogues.

    Rather than re-invent the wheel, I'm sure both projects would benefit from new features such as these without having to re-invent the wheel. NI's on Github and DLTCEP is on sourceforge and I'm sure both would accept contributions.
  • klatuklatu Member Posts: 108
    @CamDawg I agree... in theory.

    But in practice, I am not sure if I can do it at all. Maybe once I have something functional, I can try to integrate it into existing software.

    Re: translation into WeiDu, my basic notion is that the software keeps tabs on what you're doing. E.g. if you simply add a new player response to an existing dialogue state, that would translate to something like a COPY_TRANS command block. Stuff like that.
    Do you think that would be possible?
  • MilochMiloch Member Posts: 863
    Very cool @klatu. Now if you could make a tool that could generate WeiDU code for patching a tileset (i.e. replacing a single tile in it with another) without overwriting the whole thing, you would be a demigod at least. :D
  • salomonkanesalomonkane Member Posts: 48
    edited November 2012
    Hello, @klatu,

    "Would such a feature be welcome? Does it even make sense? Any thoughts?"

    I think the better way it's to follow the Miloch's Guidelines,
    It's the best way to succed for an useful & efficient tool .
    On my side, I would like to see a Campaign/Map Editor/Tool in WYSIWYG .
    Cheers, ^^ .

    Some sources about Editors from the past or in WIP :
    Infinity Campaign Maker :
    http://www.baldursgatemods.com/forums/index.php?topic=7325.0
    MergeDccV2 :
    http://www.baldursgateworld.fr/lacouronne/menace-sur-le-royaume-de-diamant-eternel/24576-conversion-des-animations-de-diablo-2-a-14.html#post298119
    Cocrane's BG Editor [in French] :
    http://www.baldursgateworld.fr/lacouronne/editeur-baldurs-gate/23262-versions-de-test-2.html#post293202
    Discussion about Toolset Wishs :
    http://www.baldursgatemods.com/forums/index.php?topic=7108.msg73219#msg73219
    http://www.baldursgatemods.com/forums/index.php?topic=7114.msg73243#msg73243
    Tools List :
    http://www.baldursgatemods.com/forums/index.php?action=downloads;cat=27
    Acronyms :
    http://www.baldursgatemods.com/forums/index.php?topic=3684.0
    Post edited by salomonkane on
  • klatuklatu Member Posts: 108
    edited November 2012
    Oh, btw, it seems a few (4 or 5 in around 10.000) of the source BAMs are sorta... buggy (negative or zero dimension images, images without a color palette, pointers to outside the file itself, stuff like that). They can still be read for the most part, but some parts must be skipped to avoid errors.

    The most puzzling are negative indices into the frame lookup table, I mean... why? Bug or feature? Or am I doing it wrong?

    Now I have to wonder how the engine itself deals with those files?
  • MilochMiloch Member Posts: 863
    @klatu - very nice. Would be cool if you could make a proper BAM editor that does import/export and pixel editing. We sort of have that with BAM Workshops (I and II) but they are a bit clunky. Also, you should have a window or status line that shows the frame, sequence, dimensions and offsets whenever you click on a frame.

    I noticed issues with some animations too, mainly with offsets being way out of bounds. Sometimes the engine can handle these (maybe by ignoring the frames); sometimes it can handle them... might also depend on video settings or something. Anyway, I wrote some code to fix it for the wyvern and tanar'ri (the two I noticed had such issues).

    http://forums.gibberlings3.net/index.php?showtopic=20900&view=findpost&p=173334
  • moody_magemoody_mage Member Posts: 2,054
    This looks amazing. I have no modding or real programming experience but the pictures look very pretty ;)
  • klatuklatu Member Posts: 108
    @Miloch - The import/export I can definitely get behind. Perhaps I will include even some basic editing features like manual animation creation, rotating, mirroring, copy-pasting, and a few basic filters. But anything beyond that would be too much, I'm afraid. There are many powerful (free / open source) software packages for high level image manipulation out there.

    Also, the GUI is very basic and largely preliminary. My main concern right now is actually being able to read all the file formats. Once I can do that, I will fine-tune the representations for each.

    Re: anim issues and your fix:
    can you explain this line to me? 
    READ_ASCII 6 bn (1) //Read BAM series (1-4)
    So you read the bytes starting at offset 6 into a ASCII-encoded string? What does the (1) signify?
    My guess is you are reading out the version number.
    If so, how does the format differ in regards to that? The IESDP is pretty vague when it comes to BAMs.

  • MilochMiloch Member Posts: 863
    @klatu - that code is written in WeiDU: http://www.weidu.org/~thebigg/README-WeiDU.html

    In the case of that line, I am just reading the 7th character of the filename, 0 would be the first and (1) meaning just one character, into the variable bn. So in the case of: 
                              ~mwyvg21.bam~  ~override~ //Wyvern
                              ~mwyvg22.bam~  ~override~
                              ~mwyvg23.bam~  ~override~
                              ~mwyvg24.bam~  ~override~
    There are four BAMs for the animation, because it's so big it has to be split into quarters for the engine. BAMs 1, 2, 3 and 4 have different offsets, being the animations in the top left, top right, lower left and lower left quadrants of the wyvern. And then the rest of the code decompresses the BAM (most are compressed), adjusts the offsets to the correct ones based on the placement and frame dimensions, then recompresses the BAM.

    If you want to see more documentation on creature animations specifically, check the documentation and reference picker in the Infinity Animations core mod. And I also wrote a batch utility for handling BAMs - assembling/dissembling them etc. It would be great to have a GUI that does that kind of stuff.

    http://www.shsforums.net/files/file/859-ia-infinity-animations-core-mod-weidu-beta-5/
    http://www.shsforums.net/files/file/837-bam-batcher/
  • klatuklatu Member Posts: 108

    OK, I have implemented a mechanism that makes it possible to (somewhat) conveniently inspect a file's abstract data structure. The goal is to implement all the known formats as they are described on the IESDP.

    I did this first and foremost to be able to inspect and compare files, to make sure I'm reading them correctly.

    However it might prove useful to newbies to get a basic idea of how each format is structured and what sort of information it stores.

    image
    image

    As you can see, the UI is a little different now. I integrated a docking framework that allows the user to customize where each component of the application is located, what size they have, etc..

    I have also started to implement a basic localization scheme that will allow me to easily provide a multi-language version in the future.

    Cheers!
  • MilochMiloch Member Posts: 863
    @klatu - That looks very cool, keep up the good work. What programming language are you writing it in? It might be worth looking at Infinity Editor as far as string and dialogues. I think the latest version was in C++ but I don't know if the source code is out there (it might be).
  • CorianderCoriander Member Posts: 1,667
    @Miloch It's Java.
  • AndreaColomboAndreaColombo Member Posts: 5,530
    @klatu - You rule big time.

    Perusing the IESDP Update History, it looks like it still does not contain the stuff @ScottBrooks revealed in this thread, so you may want to have a look at it, too.
  • klatuklatu Member Posts: 108
    @AndreaColombo
    Bookmarked. Thanks!
  • MilochMiloch Member Posts: 863
    Coriander said:

    @Miloch It's Java.

    @Coriander, I meant Infinity Explorer, which is written in Delphi actually, though bigmoshi claimed he was working on a C# version at some point.

    http://www.shsforums.net/topic/31285-infinity-explorer-v085/

    (Near Infinity is Java if that's what you were talking about.)
  • klatuklatu Member Posts: 108
    Can anyone privy to the technical details of BG:EE ( @ScottBrooks, @KeithS, @Nathan, @CameronTofer ) explain the new override system to me and others?
    In the old games, overrides override each other in a specific order.
    That order will no doubt be extended by BG:EE to allow localization.
    As I understand it, BG:EE will have several localized overrides, e.g. "...\en_US\Sounds\" will contain English sound files while "...\de_DE\Sounds\" will contain German sound files.
    My questions are these:

    1.) How does the game decide which localized override to use, ie. where in the game resources is the language set (baldur.ini, user profiles, etc...)?

    2.) Will localized overrides override each other based on the currently set language or are all files available to the engine at the same time?

    3.) What is the new override order/priority?

    Any anser is much appreciated. Thanks!
  • ScottBrooksScottBrooks Member Posts: 687
    1) the game uses the ini file to load the language.
    2) the game checks the language, and adds that lang/, and only that lang to the search path
    3) The existing order has been unchanged, the only real difference is that when we load data from a .bif, we look in the lang//data/ for the bif first, then we look in data/file.bif
  • ScottBrooksScottBrooks Member Posts: 687
    Also, the tool looks good, keep up the good work!
  • klatuklatu Member Posts: 108
    @ScottBrooks
    Thanks, this will definitely help!
  • klatuklatu Member Posts: 108
    @ScottBrooks
    Oh, I forgot to ask: Will the localized bifs be listed in the CHITIN.KEY?
    Because currently certain bits are set that specifiy to some extent where the bifs can be found relative to the game directory. Or is the file structure otherwise identical to the old one?
  • DevilSextonDevilSexton Member Posts: 5
    Let us hope tools such as this, help us create more diverse, expansive and rich mods to the Baldur's Gate Universe... to date still my favourite campaign setting. I'll be watching this with interest :) Keep doing what you do!
  • AndreaColomboAndreaColombo Member Posts: 5,530
    @klatu - it's actually a good thing that you're reworking it from scratch now that BG:EE is being released, 'cause it means that your tool will be conceived, developed and born with BG:EE compliance in mind. Now, if you could also create a BG:EE-compliant version of WeiDU ... modders and users alike would be much obliged ;D

    Also, you rule.
Sign In or Register to comment.

2024 Overhaul Games, a division of Beamdog. © 2024 Hasbro, Inc. All Rights Reserved. Baldur's Gate, Dungeons & Dragons, D&D, Forgotten Realms, Baldur's Gate, Wizards of the Coast and their logos are trademarks of Wizards of the Coast LLC in the U.S.A. and other countries, and are used with permission. Hasbro and its logo are trademarks of Hasbro, Inc. and are used with permission. © 1998 BioWare Corp. All Rights Reserved. BioWare, the BioWare Infinity Engine and the BioWare logo are trademarks of Bioware Corp. Black Isle Studios and the Black Isle Studios logo are trademarks of Interplay Entertainment Corp. Atari and the Atari logo are trademarks owned by Atari Interactive, Inc. All other trademarks are the property of their respective owners.