Skip to content

Upscaling Infinity Engine Maps with AI

2»

Comments

  • KilivitzKilivitz Member Posts: 1,459
    edited January 2019
    So I've cut the original 1360x1282 image into four slices of 680x641, converted each one into a separate BAM and placed them next to each other with dimensions of 340x320 (so, scaled to 50% of the original size).

    It works - here's a screenshot with "Scale User Interface" turned on. To see the quality improvement described by Pecca in a previous post, compare the quality of the stone textures in the sidebars and in the options menu background (open the full-sized image).


    the "options" title text is a separate MOS file that's being upscaled and therefore also looks blurry

    So there it goes - we have a workaround for blurry UI elements when using Scale User Interface.

    However, and I really hate to be Debbie Downer, there's another roadblock: while making larger window backgrounds and interface buttons is perfectly doable, you'd also need to upscale the item and spell icons.

    I can currently think of two ways of upscaling existing BAMs:

    The first is to open the file on NearInfinity and use the post-processing "resize BAM frames" filter to upscale it. Then, you have to either use the Nearest Neighbor method (which makes the icon too pixelated) or turn the file into a PVRZ-based BAM, which allows you to use the bilinear/bicubic method (which makes it too blurry). Even if the resized BAM looks good enough, you'd have to do that for every single file - and we're talking about thousands of them.

    The second way is to export all BAM frames as PNG files, upscale each frame and then convert them back to BAMs. However, the only part of the process I know how to automate would be the upscaling of the frames (using a program like A Sharper Scaling that has batch processing). And even then, upscaling transparent frames adds a "green screen" background that is not fully eliminated by the game engine, regardless of the BAM being PVRZ-based or not. So now there's an added step of editing each individual frame in order to remove the green pixels. Check out the Magic Missile icon in this screenshot:


    In this screenshot, the Magic Missile icon is upscaled to 125% of its original size.

    Unless it's possible to automate every part of the process (I don't even know enough about programming to know if coding something like that would be feasible), we're talking about a painstaking process that would take several months at best.

    It's basically the same problem @the_sextein pointed out in a previous post, when talking about resizing character animations.
  • bob_vengbob_veng Member Posts: 2,308
    i suppose some script can be created to convert all such predominantly greenish pixels to the desired uniform green. can't really see that being a major obstacle.
  • the_sexteinthe_sextein Member Posts: 711
    edited January 2019
    XnConvert has a batch processor with a nearest neighbor option. I don't know if it would eliminate those green pixels but it might. Tell it to convert to PNG and use nearest neighbor. I would do it myself but I'm at work right now. I was using it the other night because I ran into a bunch of problems with the bam setup. Still haven't really figured it out as I just don't have time. Working long weeks at the moment.

    The upconversion via batch processing is pretty fast. Converting them back and getting them to run without some sort of script would be doable. If a couple people spent 8 hours a day doing nothing but that they could knock it out.
  • PeccaPecca Member Posts: 2,177
    Thinking more about it some more, it should also be possible to create a relative-size UI using
    local screenWidth, screenHeight = Infinity_GetScreenSize() Infinity_SetArea('UIElement', screenWidth/(a*b), screenHeight/(a*c), screenWidth/(a*x), screenHeight/(a*y))

    Then using the integer 'a' to specify the size of the UI relative to screen size. This would mean a large amount of work though.
  • bob_vengbob_veng Member Posts: 2,308
    edited January 2019
    Rhiyo said:
    i think the ditering artifacts need to be reduced on those images because literally individual pixels are getting "sharpened" and it just increases the dithering effect

    i could find this tool for that purpose, dunno how it could work: https://github.com/kornelski/undither
  • RhiyoRhiyo Member Posts: 6
    edited February 2019
    XnConvert has a batch processor with a nearest neighbor option. I don't know if it would eliminate those green pixels but it might. Tell it to convert to PNG and use nearest neighbor. I would do it myself but I'm at work right now. I was using it the other night because I ran into a bunch of problems with the bam setup. Still haven't really figured it out as I just don't have time. Working long weeks at the moment.

    The upconversion via batch processing is pretty fast. Converting them back and getting them to run without some sort of script would be doable. If a couple people spent 8 hours a day doing nothing but that they could knock it out.

    A user on the GameUpscale subreddit has had some success with upscaling and alpha, might be worth checking out. He's also made many scripts that could help in the process of upscaling maps. Can be found here.

    I'm glad to see that upscaling was acknowledged in Beamdogs latest update. Hopefully something comes of it :)
  • AndreaColomboAndreaColombo Member Posts: 5,525
    The way it was acknowledge, however, confuses in-game dynamic scaling with asset scaling before insertion in the game. That doesn’t make me especially hopeful—especially when we consider how the in-game scaling solution in question was already in their games from release until 2.0.
  • the_sexteinthe_sextein Member Posts: 711
    edited February 2019
    Rhiyo wrote: »
    XnConvert has a batch processor with a nearest neighbor option. I don't know if it would eliminate those green pixels but it might. Tell it to convert to PNG and use nearest neighbor. I would do it myself but I'm at work right now. I was using it the other night because I ran into a bunch of problems with the bam setup. Still haven't really figured it out as I just don't have time. Working long weeks at the moment.

    The upconversion via batch processing is pretty fast. Converting them back and getting them to run without some sort of script would be doable. If a couple people spent 8 hours a day doing nothing but that they could knock it out.

    A user on the GameUpscale subreddit has had some success with upscaling and alpha, might be worth checking out. He's also made many scripts that could help in the process of upscaling maps. Can be found here.

    I'm glad to see that upscaling was acknowledged in Beamdogs latest update. Hopefully something comes of it :)

    Agreed! It would be nice if they put some effort into taking full advantage of the latest upscale technology. Pecca has a very Nice UI that is better for 4K displays as well. I'm glad to see I didn't scare you off with my enthusiasm for the project. I'll check it out. I really am low on time at the moment and have a lot of work related stress but I may be able to throw some time and computing force towards a group project. On the other hand, if Beamdog is going to do it, I'm confident they can do a better job. I'm just worried because their efforts have been more focused on portable devices(blurring backgrounds with the default renderer) and now with console ports joining the scene it could be a very long wait with mixed results.

    My thoughts are that modern consoles are 4K capable machines and the standards for releasing a modern console game require a solid UI and stable gameplay so it could benefit PC users if they port some of that release to PC with specific DLC or patch that upgrades the backdrops and other bits for a machine with larger storage capacity and overall power. Either way, I think it's going to be a very long time if it happens. Still waiting on a fix for LOB mode and game breaking path finding problems and it's been like 6 months with nothing in sight. lol. I think a proper upscale would push the games to a level that could no longer allow it to work on a phone app site. This would cause them to splinter out a little more but the games are already separate and not always on the same patch either.

    At this point, I would be happy to have a 2.6 game with an EET compatible version of Pecca's UI and a final SCS v.33. I'm jonesing for some Baldur's Gate and that would be perfect gameplay wise. Graphics can wait. :) If we are lucky, after the dust settles, maybe work on Icewind Dale in EET will continue. If beamdog officially announces that they are not going to upscale the games then I will take what I can from the sub reddit and make a second go of it but I'm not confident that I have the time or knowledge to do everything. The game will never be able to truly benefit from true upscales without modification to the engine that I am not capable of. Training ERSGAN AI to better work with this game would make a much more worthwhile upscale for sure. The better the backgrounds look, the worse the sprites look. It would be a mountain of work to do it all without a dedicated team.
    Post edited by the_sextein on
  • RhiyoRhiyo Member Posts: 6
    Rhiyo wrote: »
    XnConvert has a batch processor with a nearest neighbor option. I don't know if it would eliminate those green pixels but it might. Tell it to convert to PNG and use nearest neighbor. I would do it myself but I'm at work right now. I was using it the other night because I ran into a bunch of problems with the bam setup. Still haven't really figured it out as I just don't have time. Working long weeks at the moment.

    The upconversion via batch processing is pretty fast. Converting them back and getting them to run without some sort of script would be doable. If a couple people spent 8 hours a day doing nothing but that they could knock it out.

    A user on the GameUpscale subreddit has had some success with upscaling and alpha, might be worth checking out. He's also made many scripts that could help in the process of upscaling maps. Can be found here.

    I'm glad to see that upscaling was acknowledged in Beamdogs latest update. Hopefully something comes of it :)

    Agreed! It would be nice if they put some effort into taking full advantage of the latest upscale technology. Pecca has a very Nice UI that is better for 4K displays as well. I'm glad to see I didn't scare you off with my enthusiasm for the project. I'll check it out. I really am low on time at the moment and have a lot of work related stress but I may be able to throw some time and computing force towards a group project. On the other hand, if Beamdog is going to do it, I'm confident they can do a better job. I'm just worried because their efforts have been more focused on portable devices(blurring backgrounds with the default renderer) and now with console ports joining the scene it could be a very long wait with mixed results.

    My thoughts are that modern consoles are 4K capable machines and the standards for releasing a modern console game require a solid UI and stable gameplay so it could benefit PC users if they port some of that release to PC with specific DLC or patch that upgrades the backdrops and other bits for a machine with larger storage capacity and overall power. Either way, I think it's going to be a very long time if it happens. Still waiting on a fix for LOB mode and game breaking path finding problems and it's been like 6 months with nothing in sight. lol. I think a proper upscale would push the games to a level that could no longer allow it to work on a phone app site. This would cause them to splinter out a little more but the games are already separate and not always on the same patch either.

    At this point, I would be happy to have a 2.6 game with an EET compatible version of Pecca's UI and a final SCS v.33. I'm jonesing for some Baldur's Gate and that would be perfect gameplay wise. Graphics can wait. :) If we are lucky, after the dust settles, maybe work on Icewind Dale in EET will continue. If beamdog officially announces that they are not going to upscale the games then I will take what I can from the sub reddit and make a second go of it but I'm not confident that I have the time or knowledge to do everything. The game will never be able to truly benefit from true upscales without modification to the engine that I am not capable of. Training ERSGAN AI to better work with this game would make a much more worthwhile upscale for sure. The better the backgrounds look, the worse the sprites look. It would be a mountain of work to do it all without a dedicated team.

    I'm happy you are enthusiastic! I just don't come to this forum much so don't reply often. I think if Beamdog were to implement upscaling officially, they'd have to do it in a similar way to the optional HD Texture packs that some games offer on Steam. As in, release the upscaled images as a free DLC or official workshop mod.

    Please feel free to join the game upscale discord (https://discordapp.com/invite/jkFHzGD) if you want to keep up with what everyone is working on :)
  • James_MJames_M Member Posts: 140
    Very nice! BG2 original, first, please!
  • yuyueyuyue Member Posts: 3
    What happened to this experiment? Is there any development projects going on with this awesome upscaling ?
Sign In or Register to comment.