Tool To Extract Or Export Lists, Names, Descriptions, Resources, Etc. Of Files
Abel
Member Posts: 785
Is there such a thing out there?
Right now, I'd like to export a list of items (not the items files but a list of them) from a specific store.
I know EEKeeper can do so in a txt or csv but with all items and only for items, spells and creatures.
I tried DLTCEP, NI and some older tools but they can only read and show in a GUI the information I need. They can extract files too, but that's all. I gave a quick look at Weidu too, but it doesn't seem to be able to process anything deeper than the file level.
Right now, I'd like to export a list of items (not the items files but a list of them) from a specific store.
I know EEKeeper can do so in a txt or csv but with all items and only for items, spells and creatures.
I tried DLTCEP, NI and some older tools but they can only read and show in a GUI the information I need. They can extract files too, but that's all. I gave a quick look at Weidu too, but it doesn't seem to be able to process anything deeper than the file level.
0
Comments
Using Near Infinity is probably easier if you want to process only a few files. You can print the content of the file when you're in the Edit tab of the store. If you install a PDF printer such as PDF Creator, you can print the content of the window (it will "scroll" to print the full content) in a PDF file. Then in your PDF viewer, you'll be able to select the list of item among the rest.
You're right! Actually, it's the only way in NI. I found that I could select and copy/paste the content of the edit tab indeed. Alas, not all information where there. I also tried printing, but when the line is too long, it prints "..." and leaves some content.
I'm intrigued about the method you describe for Weidu. I'm looking into the documentation. I see there's a BYTE_AT offset function. Also READ_BYTE offset variable [ ELSE value ]. Is that what you're thinking about?
I have some WeiDU code that extract shop details like this, if you want: It was written for Baldur's Gate / Baldur's Gate II, but I believe there is no change in BGEE.
I wasn't interested in extracting what the shop buys, but it's not big deal to add.
You can indeed extract a lot of data from the game files using WeiDU, provided you learn the files structures. I used similar code to automate almost completely generation of articles about items (sample) and spells for an encyclopedia on a French website.
I started from the forum topic you found, especially from Miloch's code.
Now it's just thousands of line of probably bad WeiDU code though.
I would gladly have a look at what code you still hold and are willing to share.
The website is very well made. Still misses some information, but I see there's an extensive bestiary!
Here is the shop data extraction code.
// --------------------------------------------------------------------------------------------------- // --------------------------------------------------------------------------------------------------- // --------------------------------------------------------------------------------------------------- // Liste de magasins/sacs // --------------------------------------------------------------------------------------------------- // --------------------------------------------------------------------------------------------------- // --------------------------------------------------------------------------------------------------- BEGIN ~Liste des magasins~ NO_LOG_RECORD OUTER_INNER_PATCH ~12~ BEGIN WRITE_BYTE 1 0x0a READ_ASCII 1 lnl(1) WRITE_BYTE 0 0x0d READ_ASCII 0 mnl(1) READ_ASCII 0 nl(2) END OUTER_SPRINT ~list~ ~~ OUTER_SPRINT ~article~ ~~ OUTER_SPRINT ~stoitemlist~ ~~ // Partie tableau <<<<<<<< .../extractSortObjetBG-inlined/t-stotmp.txt ResRef;Name;Flags;Type >>>>>>>> COPY ~.../extractSortObjetBG-inlined/t-stotmp.txt~ ~override/stolist.csv~ // Partie description <<<<<<<< .../extractSortObjetBG-inlined/t-stodesctmp.txt Description et contenu des magasins et sacs, en enrichir si on envisage de faire des articles >>>>>>>> COPY ~.../extractSortObjetBG-inlined/t-stodesctmp.txt~ ~override/stodesc.txt~ // Liste d'objets en magasin <<<<<<<< .../extractSortObjetBG-inlined/t-stoitemtmp.txt Liste d'objets présents en magasin : non classée, avec doublons (accumulation magasin par magasin) >>>>>>>> COPY ~.../extractSortObjetBG-inlined/t-stoitemtmp.txt~ ~override/stoitemlist.txt~ // Extraction proprement dite des données des magasins // Liste établie à partir d'une recherche de dialogues et de créatures ouvrant le magasin (StartStore) SPRINT ~crg~ ~\(ERDANE\|FRIEND\|HIGHHEDG\|INN0103\|INN0105\|INN0119\|INN2616\|INN3304\|INN3307\|INN3351\|INN3357\|INN4801\|STO0703\|STO1112\|STO1115\|STO4803\|STO4901\|STO4906\|STO4908\|STO4909\|STOBLACK\|STOCHEAP\|STOSILEN\|STOVAI\|TAERUM\|TAV0114\|TAV0154\|TAV0705\|TAV0721\|TAV1215\|TAV4809\|TEM0002\|TEM0131\|TEM0132\|TEM2304\|TEM2601\|TEM3402\|TEM4802\|ULGOTH\)\.sto~ COPY_EXISTING_REGEXP GLOB ~%crg%~ ~override~ PATCH_IF (~%SOURCE_SIZE%~ > 0x9b) BEGIN // Les manipulations ultérieures vont écraser SOURCE_RES, hors on veut garder le nom du magasin SPRINT ~nomSto~ ~%SOURCE_RES%~ // Type READ_LONG 0x08 type PATCH_IF (type = ~0~) BEGIN SPRINT ~strtype~ ~Magasin~ END ELSE PATCH_IF (type = ~1~) BEGIN SPRINT ~strtype~ ~Taverne~ END ELSE PATCH_IF (type = ~2~) BEGIN SPRINT ~strtype~ ~Auberge~ END ELSE PATCH_IF (type = ~3~) BEGIN SPRINT ~strtype~ ~Temple~ END ELSE PATCH_IF (type = ~5~) BEGIN SPRINT ~strtype~ ~Conteneur~ END ELSE SPRINT ~strtype~ ~inconnu~ // Nom READ_LONG 0x0c strrefname READ_STRREF 0x0c name // Flags READ_LONG 0x10 flags SPRINT ~strflags~ ~~ PATCH_IF (((flags & 0b00000000000000000000000000000001) >> 0) = 1) BEGIN SPRINT ~strflags~ ~%strflags%vend, ~ END PATCH_IF (((flags & 0b00000000000000000000000000000010) >> 1) = 1) BEGIN SPRINT ~strflags~ ~%strflags%achète, ~ END PATCH_IF (((flags & 0b00000000000000000000000000000100) >> 2) = 1) BEGIN SPRINT ~strflags~ ~%strflags%identifie, ~ END PATCH_IF (((flags & 0b00000000000000000000000000001000) >> 3) = 1) BEGIN SPRINT ~strflags~ ~%strflags%vol possible, ~ END PATCH_IF (((flags & 0b00000000000000000000000000010000) >> 4) = 1) BEGIN SPRINT ~strflags~ ~%strflags%accepte don, ~ END PATCH_IF (((flags & 0b00000000000000000000000000100000) >> 5) = 1) BEGIN SPRINT ~strflags~ ~%strflags%vend soin, ~ END PATCH_IF (((flags & 0b00000000000000000000000001000000) >> 6) = 1) BEGIN SPRINT ~strflags~ ~%strflags%vend boisson, ~ END PATCH_IF (((flags & 0b00000000000000000001000000000000) >> 12) = 1) BEGIN SPRINT ~strflags~ ~%strflags%receleur~ END READ_LONG 0x14 pourcentvente READ_LONG 0x18 pourcentachat READ_LONG 0x1c depreciation READ_SHORT 0x20 echecvol READ_LONG 0x40 coutident // Chambres SPRINT chambres ~~ PATCH_IF type = 2 THEN BEGIN READ_LONG 0x5c roomflags PATCH_IF (((roomflags & 0b00000000000000000000000000000001) >> 0) = 1) BEGIN READ_LONG 0x60 prix SPRINT ~chambres~ ~%chambres%Paysan (%prix%), ~ END PATCH_IF (((roomflags & 0b00000000000000000000000000000010) >> 1) = 1) BEGIN READ_LONG 0x64 prix SPRINT ~chambres~ ~%chambres%Marchand (%prix%), ~ END PATCH_IF (((roomflags & 0b00000000000000000000000000000100) >> 2) = 1) BEGIN READ_LONG 0x68 prix SPRINT ~chambres~ ~%chambres%Noble (%prix%), ~ END PATCH_IF (((roomflags & 0b00000000000000000000000000001000) >> 3) = 1) BEGIN READ_LONG 0x6c prix SPRINT ~chambres~ ~%chambres%Royal (%prix%), ~ END END // PATCH_IF SPRINT listeobjet ~~ PATCH_IF (~%strflags%~ STRING_CONTAINS_REGEXP ~vend,~) = 0 THEN BEGIN // Vend des objets ~strtype~ ~Magasin~ SPRINT listeobjet ~Objets à vendre : Code Charges Nombre à vendre Stock~ // Lecture des objets // repris du tutorial 9.24 de la doc WeiDU, et adapté au fichier STO GET_OFFSET_ARRAY item_array 0x34 4 0x38 4 0 0 0x1c PHP_EACH item_array AS int => item_off BEGIN // READ_ASCII (item_off + 0x00) codeObjet READ_SHORT (item_off + 0x0a) charge READ_LONG (item_off + 0x14) stock READ_LONG (item_off + 0x18) reappro PATCH_IF reappro = 0 BEGIN SPRINT strreappro ~Stock limité~ END ELSE PATCH_IF reappro = 1 BEGIN SPRINT strreappro ~Stock réapprovisionné~ END ELSE BEGIN SPRINT strreappro ~Réapprovisionnement invalide~ END SPRINT listeobjet ~%listeobjet%%nl%%codeObjet% %charge% %stock% %strreappro%~ SPRINT stoitemlist ~%stoitemlist%%nl%%codeObjet%~ END // PHP_EACH END // PATCH_IF // Ecriture seulement si le nom et la description de l'objet sont définis // Pas applicable pour les magasins car certains n'ont pas de nom //PATCH_IF (strrefname >= 0) AND (strrefname <= 24123) PATCH_IF (type = ~0~) OR (type = ~1~) OR (type = ~2~) OR (type = ~3~) THEN BEGIN // Liste de magasins SPRINT ~list~ ~%list%%nomSto%;%name%;%strflags% (%flags%);%strtype% (%type%) %nl%~ // Information pour magasin, taverne, auberge, temple, ou sac SPRINT ~nouvmag~ ~[HR][/HR] Nom : %name% Code : %nomSto% Type : %strtype% Flags : %strflags% Taux vente : %pourcentvente% Taux achat : %pourcentachat% Taux dépréciation : %depreciation% Echec du vol à la tire : %echecvol% Coût d'identification : %coutident% Prix des chambres : %chambres% Soins proposés : non extrait Types d'objets achetés : non extrait Boissons : non extrait %listeobjet%~ // Ajout de l'article SPRINT ~article~ ~%article%%nl%%nl%%nl%%nouvmag%~ END // PATCH_IF (strrefname >= 0) AND (strrefname <= 24123) END // PATCH_IF (~%SOURCE_SIZE%~ > 0x9b) BUT_ONLY_IF_IT_CHANGES // Ecritures des fichiers APPEND ~stolist.csv~ ~%list%~ APPEND ~stodesc.txt~ ~%article%~ APPEND ~stoitemlist.txt~ ~%stoitemlist%~It's very much inspired by Miloch's code and by my attempts at extracting item and spell data.
This part is somehow readable, as it's purely informative.
In comparison, my item extraction code is a mess, partly due to including a lot of formatting constraints for the articles.
For the rest of the code, if you have a use, I'll either attach or send by mail due to size.
How exactly do you use this?
<<<<<<<< >>>>>>>>What do you use this for?
OUTER_INNER_PATCH ~12~ BEGIN WRITE_BYTE 1 0x0a READ_ASCII 1 lnl(1) WRITE_BYTE 0 0x0d READ_ASCII 0 mnl(1) READ_ASCII 0 nl(2) ENDI see you somehow copied txt into csv. Didn't know it was possible:
COPY ~.../extractSortObjetBG-inlined/t-stotmp.txt~ ~override/stolist.csv~It's amusing to see how you use SPRINT to set variables. Didn't think about it:
SPRINT ~crg~ ~\(ERDANE\|FRIEND\|HIGHHEDG\|INN0103\|INN0105\|INN0119\|INN2616\|INN3304\|INN3307\|INN3351\|INN3357\|INN4801\|STO0703\|STO1112\|STO1115\|STO4803\|STO4901\|STO4906\|STO4908\|STO4909\|STOBLACK\|STOCHEAP\|STOSILEN\|STOVAI\|TAERUM\|TAV0114\|TAV0154\|TAV0705\|TAV0721\|TAV1215\|TAV4809\|TEM0002\|TEM0131\|TEM0132\|TEM2304\|TEM2601\|TEM3402\|TEM4802\|ULGOTH\)\.sto~If it's no bother, yes, could you attach the other files?
You could without if you included the same content in a file in your "mod". Let's say it's either for lazyness or to keep everything in the tp2 file.
The 0x0a and 0x0d are characters used in text files to mark the end of a line. They differ on Linux, old Mac (new use the same as Linux) and Windows. This code is borrowed from Miloch and provides a way to create a variable that you can use in strings to insert end of lines, as for generating the lists, using the %lnl% variable. Don't ask me more, I just know it works. ;-)
I used csv extension because the formatting of data I used separates data with a semi colon. It's just a text file but using the csv extension makes it easier to open with Excel/Open Office, that's all. The "magic" of creating the csv format comes from the way the code writes the data with the ; separator. Again this is borrowed from Miloch's code on G3.
The file is created into the override folder as this is the only way to proceed with WeiDU (I tried other directories without success).
Actually the whole mess (written for Baldur's Gate file format, especially for extracting from ITM and CRE) is in a single file. Don't forget your spelunking equipment before diving!