File association will be asked every time if you move ProjectInfinity.exe in another location, which happens some times if you has this tool copied in different places to not re-enter different path's (in ini).
The checks is preformed against application path. Why you wan't re-enter different path's inside ini? From what I can think of it, it's very customized user-case. Anyway, I will try to improve this process later.
What is really terrible now is that there is no pause / retry / error indication.
Say, just now, i'm put some premade install sequence with SCS... once all is installed i'm found that iwdification is not installed, because somewhy PI copied it as `setup-iwdification iwdfication.exe`, - so components doesn't get installed. And what i'm can do? Right - just begin from start with manual intrusion... ugh.
ADD: For clearness, i'm install iwdfication right after spell_rev, but not as part of SCS.
Pausing installation is planned.
Reaction to proces execution errors can take a while but I will eventually get there. But I cannot reproduce iwdification bug. Are you sure that install sequence wan't malformed?
You should allways try to isntall all mods separatley, one by one, to make sure that they works independly. Only then you can try to execute much bigger installation.
And please send/post logs. I should probably stop asking about them and implement a button which will gather all logs and open issue at github etc.
File association will be asked every time if you move ProjectInfinity.exe in another location, which happens some times if you has this tool copied in different places to not re-enter different path's (in ini).
The checks is preformed against application path. Why you wan't re-enter different path's inside ini? From what I can think of it, it's very customized user-case. Anyway, I will try to improve this process later.
Mods usually has much lesser quality than original game, this always happens with any mods in any game forever. However modern weidu-based setup is painful in terms of reapplying fixes, because... if you lucky enough - you will fix something in middle of setup, and this is counter-productive to reapply rest of mods like SCS which installing still slow just to check something. So, in reality i'm use one setup where i'm play in game, while having few additional copies installs to play with mods / fixes. So, i'm prefer keep different copies with different ini files which already has right paths to game or mod source folders. Surely this is specific case, but not so uncommon for everyone who do a bit of modding.
Reaction to proces execution errors can take a while but I will eventually get there. But I cannot reproduce iwdification bug. Are you sure that install sequence wan't malformed?
As for setup-iwdfication... I'm deleted logs... i'm tried to reproduce now but no luck. So looks like you are right, may be it was definitely glitch with install sequence, but this is strange, cause i'm just copy-paste sequence from file.
Few suggestions / proposals:
1. It would be nice to use normal windows (e.g. non-dialog), because they look bit differently.
2. Allow user to resize / maximize window. Users may want to see bit more.
3. May be having visible path(s) for all games are bit too much in terms of screen space. EE users probably never want see paths for non-EE. However them can be hidden over collapsible panel, or move to configuration dialog or always show paths. Alternatively, you can permit select all known games from combobox and then show relevant path for this game (e.g. reverse logic from current, where you can fill all paths and only then choose game type for non-empty paths). Last option i think bit better, because it is kind of direct logic, when user choose game type, and then PI can deduct path from registry for example (just imaginations).
4. Please, show window, and then perform any background tasks (ideally with process indication). Just now it is happens big lag betwer PI run and reaction (window shown).
5. I hope that auto-update check at least daily, but not on each run.
Phase 3 - so PI can read and apply internal dependence (requirements/conflicts) between components of the same mod
Phase 4 - so PI can read and apply external mod dependence (requirements/conflicts) between external mods
(I know that you thinking about REQUIRE_PREDICATE for Phase 3/4 but currently there is no way for me to extract this information, more specifically, weidu can't do it and after so much time, I'm losing my faith that it will ever can do it.)
It will be up modders to support it or not.
How can modders support this in their mods? Is there a way I can do this/prepare for it now?
@HalfCelestial Start with posting logs. Do you have installed "Git for Windows" ?
Whoops. I totally missed that part somehow. ?♂️
I installed it now. However, it showed many warnings when installing it, and now when I try to download, a command prompt window opens up and closes quickly, but doesn't download anything.
@HalfCelestial So it must be a local problem with Git. Because my application depends on git, I also inherited it's problems. You can ask for support at git-related board like this: https://github.com/git-for-windows/git/issues or you can wait until I will implement build-in version of git.
1. It would be nice to use normal windows (e.g. non-dialog), because they look bit differently.
2. Allow user to resize / maximize window. Users may want to see bit more.
3. May be having visible path(s) for all games are bit too much in terms of screen space. EE users probably never want see paths for non-EE. However them can be hidden over collapsible panel, or move to configuration dialog or always show paths. Alternatively, you can permit select all known games from combobox and then show relevant path for this game (e.g. reverse logic from current, where you can fill all paths and only then choose game type for non-empty paths). Last option i think bit better, because it is kind of direct logic, when user choose game type, and then PI can deduct path from registry for example (just imaginations).
4. Please, show window, and then perform any background tasks (ideally with process indication). Just now it is happens big lag betwer PI run and reaction (window shown).
5. I hope that auto-update check at least daily, but not on each run.
@Lamiar
1. Please explain?
2. It's planned GUI redesign
3. It's also It's planned GUI redesign, I would go for 'configuration dialog', I've seen many examples of it, never had time to implement it
4. Multithreading is one of the goal, the tricky part is that I can't use NET BackgroundWorker/Task
5. It's "Spotify"-like model so auto-update checks are executed each time when application starts.
@Lamiar
1. Please explain?
2. It's planned GUI redesign
3. It's also It's planned GUI redesign, I would go for 'configuration dialog', I've seen many examples of it, never had time to implement it
4. Multithreading is one of the goal, the tricky part is that I can't use NET BackgroundWorker/Task
5. It's "Spotify"-like model so auto-update checks are executed each time when application starts.
1. Normal window, means normal/top-level/overlapped resizeable window. I don't know how explain more. Dialog windows has different close button size... for nothing. Amyway i say more about ability to resize window.
4. It is strange, with any modern framework versions this available everywhere.
@Lamiar
1. Will do.
4. It's because I'm not using c#, I'm using Powershell which is single thread. That's why implementing multi-threading is tricky and time consuming but for BETA/prototyping phase I can accept such drawback for other benefits.
@Lamiar
4. It's because I'm not using c#, I'm using Powershell which is single thread. That's why implementing multi-threading is tricky and time consuming but for BETA/prototyping phase I can accept such drawback for other benefits.
Oh. Madness. Clear. You probably know, technically pwsh can interact with almost any .NET code, so i think is possible to get it worked with threads. However i personally avoid pwsh, had deep exp in past - enough for me.
Oh. Madness. Clear. You probably know, technically pwsh can interact with almost any .NET code, so i think is possible to get it worked with threads. However i personally avoid pwsh, had deep exp in past - enough for me.
It can invoke .NET code but it would still be single thread unless I spawn another process. But there is better way to do it: Jobs. Don't worry, at the end of the road it won't be a problem.
Oh. Madness. Clear. You probably know, technically pwsh can interact with almost any .NET code, so i think is possible to get it worked with threads. However i personally avoid pwsh, had deep exp in past - enough for me.
It can invoke .NET code but it would still be single thread unless I spawn another process. But there is better way to do it: Jobs. Don't worry, at the end of the road it won't be a problem.
I'm don't worry...
It is not better in any way, because jobs are ways to sandbox processes, but not for imitating threads. And technically *nothing*
can prevent thread spawning via Thread.Create or even native CreateThread. Some drivers do this, btw.
Any way, doesnt treat this as criticism, just for me looks like you choose bad/hardest tool, but this is only mine opinion.
Any way, doesnt treat this as criticism, just for me looks like you choose bad/hardest tool, but this is only mine opinion.
@Lamiar
I know, just trust me, if it wasn't the actual tool-set, this app would never have current features. My previous experience shows that the amount of c# equivalent code would be 2-4 times larger. I would be still at the point when I display mods at treeview
Any way, doesnt treat this as criticism, just for me looks like you choose bad/hardest tool, but this is only mine opinion.
@Lamiar
I know, just trust me, if it wasn't the actual tool-set, this app would never have current features. My previous experience shows that the amount of c# equivalent code would be 2-4 times larger. I would be still at the point when I display mods at treeview
BTW, i'm get accidentally reproduced situation with "setup-iwdfication iwdfication.exe". I'm put some mods into mod source folder, specifically: iwdfication and stratagems.
Then i'm just copy-paste install sequence and press install. PI already noticed what iwdfication is duplicated, but doesn't prevent install / and do this buggy thing.
In my opinion, once PI find top-level .tp file (like stratagems.tp2) - it may not scan inner directories, because stratagems in that case would care about inner mods alone. In other words, .tp2 files who are closer to mod directory root should win. And only after what duplication detection should occur.
This particular use case for peoples who doesn't care about what stratagems already are included iwdfication and want install it as standalone (and this have sense, because after iwd we want install other spell-related mods which go strictly before stratagems).
@Lamiar
Thanks for this feedback. You error occurs because for now, PI doesn't support multiple instances of the same mod. So install sequence tells PI to install mod with 'iwdfication' ID which happen to be the one from stratagems. I should probably prevent installation when there are duplicated mods found.
Solution for you problem:
-use iwdification from Stratagems + use install order sequence file for easy putting "scs iwdification' components where you want
or
-rename stratagems\test\setup-iwdification.tp2 to .tph
Regarding you proposal:
Detecting actual and correct weidu mod 'inner directory' is a big challenge, detection procedure also affects the startup of the application and other things. So unless you want to spend a week and driving into weidu madness and problems, accept the fact that It's much easier to implement support for 'multiple instances of the same mod' instead of the excluding of the mods which are placed inside other mods inner directories
@ALIEN i'm not sure what i'm explain correctly or you understood me. I only suggest stop scanning inner directories of x directory when mod found in x directory. Because properly-formed mods are follow basically a two patterns, - doesnt see nothing complex / mad / what might not work. This also will have positive impact: faster startup, because practically you will never scan most of inner directories.
if I stop scanning inner directory of setup-mymodA.tp2 which is "G:\IEMods\", Stratagems won't be detected. If the algorithm would need to detect correct mod source folder ( "G:\IEMods\mymodA" or "G:\IEMods\SCS\Stratagems\" ), it must preform 'detection proces' for every mod first, then do a foreach loop for every mod to compare and exclude those mods which 'mod source folder' happens to be inside other, correctly detected ' mod source folder'. That's lot of extra processing which can be avoided simply by adding support for multiple instances of the same mod.
Because properly-formed mods are follow basically a two patterns
That's not correct, there are 6 types of properly-formed weidu mods, PI can properly detect and copy all of them, no matter of where/how deep folder structure is. The price for it is that 'detection proces' must be executed via specific way, which involves reading entire tp2 file content and executing weidu, which are main reasons for the long startup time/slowdowns.
This also will have positive impact: faster startup, because practically you will never scan most of inner directories.
Scanning of inner directories is not a problem, it doesn't add a lot delay time to startup time - it's the mod languages list and component list processing which is very slow. I'm currently exploring other ways to provide faster application startup, wait for next version to compare.
I’m with Lamiar here. There’s no reason to keep scanning once you’ve found the top-level tp2. A mod might have inert tp2 files in subdirectories for all manner of reasons, good and bad. (In this case, there’s a tp2 in test because I put all sorts of random files in test for lots of reasons and don’t always bother to clean them out before releasing RCs; it’s got nothing directly to do with the IWD spells that I ship with SCS.)
@DavidW Having extra tp2 files inside mod inner directory is fine, when I will implement support for multiple copies of the same mod, iwdfication-scs edge case won't be a problem either.
Refinements case is totally different: 20 tp2 files without weidu code so when I scan tp2 file content in order to find for eg README, it takes eternity.
As I demonstrated above, scanning correct 'mod source folder' and excluding inner mods (those tp2 files also requires detection of 'mod source folder' for comparison) task is something which takes lot of time. I have to make compromises when I take into consideration things like coping, reading mod languages, components, metadata etc and finally, displaying all of this at the GUI.
I don't think the algorithm should be as complicated or time-consuming as you suggest (and I think handling inert tp2 files by being able to support multiple copies gets the logic wrong: those tp2 files aren't mods at all). Try this:
Step 1:
Suppose you start in a folder that you know is not a mod's folder (in the first instance, the top-level folder in which the mods are kept). Check it for tp2 files. If you find any, process them as mod tp2 files, then parse them for a 'backup' line, and extract the mod's folder from that. (The only mods for which you can't find the mod's folder from the backup line are modern mods like SCS that keep the tp2 file in the mod folder.)
Step 2:
Now go through any folders that aren't the mod folders of tp2 files you've already found. These might or might not be mod folders. The reliable check is: does 'folder' contain either 'setup-folder.tp2' or 'folder.tp2'. If it contains either, it's (almost guaranteed to be) a mod folder; process (setup-)folder.tp2 as a mod tp2 file, then stop. If it doesn't, go back to step 2.
That should be pretty much 100% reliable (there are edge cases, but I think they're extremely unlikely to come up unless the end user intentionally builds a pathological file system). And there's hardly any processing required: I think I could code it in WEIDU in a few dozen lines and it would run in seconds, so C++ or whatever you're using should have no trouble with it. (And it doesn't require the existence or detection of the WEIDU executable.)
Some examples:
1) the example you gave above. The algorithm:
- starts in IEMods;
- detects tp2 files setup-mymodA, setup-mymodB, setup-mymodC, and processes them;
- extracts mymodA, mymodB, and ModFolderForModC from those tp2 files and ignores them;
- moves into SCS;
- finds no tp2 file, so recurses to all subdirectories of SCS (of which there's only one: Stratagems);
- moves into Stratagems;
- detects tp2 file setup-stratagems, so concludes that stratagems is a mod directory;
- processes setup-stratagems;
- halts, ignoring all subdirectories of stratagems (including test, so it never even finds setup-iwdification.tp2).
2) The Refinements case, say where it's been extracted to G:\IEMods as the only mod. The algorithm:
- starts in IEMods;
- finds no tp2 file, so recurses to all subdirectories of IEMods (of which there's only one: Refinements);
- detects tp2 file setup-refinements, so concludes that refinements is a mod directory;
- processes setup-refinements;
- halts, ignoring 'armor.tp2' in refinements and all of refinements' subfolders.
@DavidW First of all, many thanks for actual brainstorming of this topic and providing detailed step-by-step algorithm. Now I'm certain that you indeed spend some time to think through. I've spend some time on this specific topic once again (remember PPG discussion?), here are are the conclusions:
One sec ... I was confused by 'go back to step 2'.
Ok, you algorithm starts with extracting 'BACKUP' keyword for first pass, so it covers "Worldmap", "Dino QuestPack" and some other mods. Then, it try to detect 'one level-up directory name if the folder name match tp2 without setup-'. That is what PI is using since our discussion at PPG board, except the part when it saves matched mod directories and exclude them during the loop.
Reading 'BACKUP' keyword from file is not a problem, it's the getting list of mod languages and components which slows down whole scanning process. I had to look alternative ways in order to speed things up which I hope to introduce for next version.
Comments
Pausing installation is planned.
Reaction to proces execution errors can take a while but I will eventually get there. But I cannot reproduce iwdification bug. Are you sure that install sequence wan't malformed?
You should allways try to isntall all mods separatley, one by one, to make sure that they works independly. Only then you can try to execute much bigger installation.
And please send/post logs. I should probably stop asking about them and implement a button which will gather all logs and open issue at github etc.
Done Done
Mods usually has much lesser quality than original game, this always happens with any mods in any game forever. However modern weidu-based setup is painful in terms of reapplying fixes, because... if you lucky enough - you will fix something in middle of setup, and this is counter-productive to reapply rest of mods like SCS which installing still slow just to check something. So, in reality i'm use one setup where i'm play in game, while having few additional copies installs to play with mods / fixes. So, i'm prefer keep different copies with different ini files which already has right paths to game or mod source folders. Surely this is specific case, but not so uncommon for everyone who do a bit of modding.
As for setup-iwdfication... I'm deleted logs... i'm tried to reproduce now but no luck. So looks like you are right, may be it was definitely glitch with install sequence, but this is strange, cause i'm just copy-paste sequence from file.
Few suggestions / proposals:
1. It would be nice to use normal windows (e.g. non-dialog), because they look bit differently.
2. Allow user to resize / maximize window. Users may want to see bit more.
3. May be having visible path(s) for all games are bit too much in terms of screen space. EE users probably never want see paths for non-EE. However them can be hidden over collapsible panel, or move to configuration dialog or always show paths. Alternatively, you can permit select all known games from combobox and then show relevant path for this game (e.g. reverse logic from current, where you can fill all paths and only then choose game type for non-empty paths). Last option i think bit better, because it is kind of direct logic, when user choose game type, and then PI can deduct path from registry for example (just imaginations).
4. Please, show window, and then perform any background tasks (ideally with process indication). Just now it is happens big lag betwer PI run and reaction (window shown).
5. I hope that auto-update check at least daily, but not on each run.
Whoops. I totally missed that part somehow. ?♂️
I installed it now. However, it showed many warnings when installing it, and now when I try to download, a command prompt window opens up and closes quickly, but doesn't download anything.
Where can I find the relevant logs?
@Lamiar
1. Please explain?
2. It's planned GUI redesign
3. It's also It's planned GUI redesign, I would go for 'configuration dialog', I've seen many examples of it, never had time to implement it
4. Multithreading is one of the goal, the tricky part is that I can't use NET BackgroundWorker/Task
5. It's "Spotify"-like model so auto-update checks are executed each time when application starts.
1. Normal window, means normal/top-level/overlapped resizeable window. I don't know how explain more. Dialog windows has different close button size... for nothing. Amyway i say more about ability to resize window.
4. It is strange, with any modern framework versions this available everywhere.
1. Will do.
4. It's because I'm not using c#, I'm using Powershell which is single thread. That's why implementing multi-threading is tricky and time consuming but for BETA/prototyping phase I can accept such drawback for other benefits.
Oh. Madness.
I'm don't worry...
It is not better in any way, because jobs are ways to sandbox processes, but not for imitating threads. And technically *nothing*
can prevent thread spawning via Thread.Create or even native CreateThread. Some drivers do this, btw.
Any way, doesnt treat this as criticism, just for me looks like you choose bad/hardest tool, but this is only mine opinion.
I know, just trust me, if it wasn't the actual tool-set, this app would never have current features. My previous experience shows that the amount of c# equivalent code would be 2-4 times larger. I would be still at the point when I display mods at treeview
BTW, i'm get accidentally reproduced situation with "setup-iwdfication iwdfication.exe". I'm put some mods into mod source folder, specifically: iwdfication and stratagems.
Then i'm just copy-paste install sequence and press install. PI already noticed what iwdfication is duplicated, but doesn't prevent install / and do this buggy thing.
In my opinion, once PI find top-level .tp file (like stratagems.tp2) - it may not scan inner directories, because stratagems in that case would care about inner mods alone. In other words, .tp2 files who are closer to mod directory root should win. And only after what duplication detection should occur.
This particular use case for peoples who doesn't care about what stratagems already are included iwdfication and want install it as standalone (and this have sense, because after iwd we want install other spell-related mods which go strictly before stratagems).
Thanks for this feedback. You error occurs because for now, PI doesn't support multiple instances of the same mod. So install sequence tells PI to install mod with 'iwdfication' ID which happen to be the one from stratagems. I should probably prevent installation when there are duplicated mods found.
Solution for you problem:
-use iwdification from Stratagems + use install order sequence file for easy putting "scs iwdification' components where you want
or
-rename stratagems\test\setup-iwdification.tp2 to .tph
Regarding you proposal:
Detecting actual and correct weidu mod 'inner directory' is a big challenge, detection procedure also affects the startup of the application and other things. So unless you want to spend a week and driving into weidu madness and problems, accept the fact that It's much easier to implement support for 'multiple instances of the same mod' instead of the excluding of the mods which are placed inside other mods inner directories
I understood you correctly. You see nothing complex/not working because you actually didn't do it.
Example common mod folder structure: if I stop scanning inner directory of setup-mymodA.tp2 which is "G:\IEMods\", Stratagems won't be detected. If the algorithm would need to detect correct mod source folder ( "G:\IEMods\mymodA" or "G:\IEMods\SCS\Stratagems\" ), it must preform 'detection proces' for every mod first, then do a foreach loop for every mod to compare and exclude those mods which 'mod source folder' happens to be inside other, correctly detected ' mod source folder'. That's lot of extra processing which can be avoided simply by adding support for multiple instances of the same mod.
That's not correct, there are 6 types of properly-formed weidu mods, PI can properly detect and copy all of them, no matter of where/how deep folder structure is. The price for it is that 'detection proces' must be executed via specific way, which involves reading entire tp2 file content and executing weidu, which are main reasons for the long startup time/slowdowns.
Scanning of inner directories is not a problem, it doesn't add a lot delay time to startup time - it's the mod languages list and component list processing which is very slow. I'm currently exploring other ways to provide faster application startup, wait for next version to compare.
Refinements case is totally different: 20 tp2 files without weidu code so when I scan tp2 file content in order to find for eg README, it takes eternity.
As I demonstrated above, scanning correct 'mod source folder' and excluding inner mods (those tp2 files also requires detection of 'mod source folder' for comparison) task is something which takes lot of time. I have to make compromises when I take into consideration things like coping, reading mod languages, components, metadata etc and finally, displaying all of this at the GUI.
Next version startup time should be faster
I don't think the algorithm should be as complicated or time-consuming as you suggest (and I think handling inert tp2 files by being able to support multiple copies gets the logic wrong: those tp2 files aren't mods at all). Try this:
Step 1:
Suppose you start in a folder that you know is not a mod's folder (in the first instance, the top-level folder in which the mods are kept). Check it for tp2 files. If you find any, process them as mod tp2 files, then parse them for a 'backup' line, and extract the mod's folder from that. (The only mods for which you can't find the mod's folder from the backup line are modern mods like SCS that keep the tp2 file in the mod folder.)
Step 2:
Now go through any folders that aren't the mod folders of tp2 files you've already found. These might or might not be mod folders. The reliable check is: does 'folder' contain either 'setup-folder.tp2' or 'folder.tp2'. If it contains either, it's (almost guaranteed to be) a mod folder; process (setup-)folder.tp2 as a mod tp2 file, then stop. If it doesn't, go back to step 2.
That should be pretty much 100% reliable (there are edge cases, but I think they're extremely unlikely to come up unless the end user intentionally builds a pathological file system). And there's hardly any processing required: I think I could code it in WEIDU in a few dozen lines and it would run in seconds, so C++ or whatever you're using should have no trouble with it. (And it doesn't require the existence or detection of the WEIDU executable.)
Some examples:
1) the example you gave above. The algorithm:
- starts in IEMods;
- detects tp2 files setup-mymodA, setup-mymodB, setup-mymodC, and processes them;
- extracts mymodA, mymodB, and ModFolderForModC from those tp2 files and ignores them;
- moves into SCS;
- finds no tp2 file, so recurses to all subdirectories of SCS (of which there's only one: Stratagems);
- moves into Stratagems;
- detects tp2 file setup-stratagems, so concludes that stratagems is a mod directory;
- processes setup-stratagems;
- halts, ignoring all subdirectories of stratagems (including test, so it never even finds setup-iwdification.tp2).
2) The Refinements case, say where it's been extracted to G:\IEMods as the only mod. The algorithm:
- starts in IEMods;
- finds no tp2 file, so recurses to all subdirectories of IEMods (of which there's only one: Refinements);
- detects tp2 file setup-refinements, so concludes that refinements is a mod directory;
- processes setup-refinements;
- halts, ignoring 'armor.tp2' in refinements and all of refinements' subfolders.
One sec ... I was confused by 'go back to step 2'.
Ok, you algorithm starts with extracting 'BACKUP' keyword for first pass, so it covers "Worldmap", "Dino QuestPack" and some other mods. Then, it try to detect 'one level-up directory name if the folder name match tp2 without setup-'. That is what PI is using since our discussion at PPG board, except the part when it saves matched mod directories and exclude them during the loop.
Reading 'BACKUP' keyword from file is not a problem, it's the getting list of mod languages and components which slows down whole scanning process. I had to look alternative ways in order to speed things up which I hope to introduce for next version.
>> 1. Download and extract all mods into one folder
Some mods don't come with a setup-modname.exe. Do I need to provide one or does PI take care of it?
>> 2. Optional step: extract BWFixpack into the same folder and install it one time only
Optional? What do I base my decision on? What do I gain and/or lose by using BWFixpack?