hiDPI displays ("retina") and UI scaling in 2.5betas (Windows 10, but maybe others too?)
BelegCuthalion
Member, Mobile Tester Posts: 454
preamble:
i had filed this as a feature request for 2.3 / 1.4 games, but it's still an issue in the 2.5 betas, so i'm trying to file this again for those versions, but i'd like to discuss and clarify first where this is an issue ...
Until a few years, computer screens always had a ratio of physical dimensions and pixel dimensions that resulted in pixel densities of around 90-120 pixels per inch (let's say 100 pixels for simplicity). So if there was an element of 100x100 pixels on screen, it was (about) 1x1 inch in size. Programs and programmers relied on that, declaring sizes in pixels, not caring about real physical size (as it was always okay).
With mobile devices and their small screens, but same pixel dimensions, pixel density rose - some had around 200ppi, others around 300ppi, etc. so an element of 100x100 pixels is way smaller on those screens. So on mobile devices, dimensioning early on became relative to the physical screen size, not relying on pixel dimensions anymore. Fonts, images, etc. stayed the desired physical size, and the additional pixels available were used to render them sharper.
This finally arrived on notebooks and desktop computers as well the last years ... apple was defacto first with a usable variant of this (working for exactly their systems only though), microsoft was doing it "right" in the long run, which resulted in "wrong" for quite a while.
But at least with windows 10 creators update things are good now for almost all use cases.
One can set a scaling factor in the systems screen resolution settings (like "try to display everything 200% of the size usually calculated, and make things sharper instead of smaller" – this can be a setting between 100 and 500%, but values between 150 and 300 are the most common).
Then, there are basically 3 modes that can be set per program:
a) default and the best: let the program handle the high density (It has to know about the scaling factor set, and render sharper instead of smaller according to this).
b) let windows 10 do the best (the program does not know about scaling but id programmed well – windows does render everything sharper that can be done, like type, and scaling everything that can't, like images). this works well for most modern windows programs.
c) let windows 10 force a scaling up in a brutal way: make the program pixels double/triple size ... this makes the programs blurry / low resolution, but they work. this is for the old programs around – if this would not be done by force, the programs would render each of their pixel to one physical pixel, making it appear way too small.
So i'm currently testing the 2.5 betas on my windows 10 fall creators update system, that has a physical screen resolution of 3840x2160 Pixels (that is twice the standard 1920x1080 in each direction), a 15,6" size, resulting in a density of almost 300ppi. My system scaling factor is set to 200%, instructing aware programs to display the same size as on a standard 1920 screen, but sharper.
The Betas are somewhat, but not completely?, aware of this in default mode a):
They render the text sharp, the scale the mouse cursor right (well, at least in software cursor mode), etc. – so they "know". but there is one big issue: the UI scaling is not respecting the systems scaling factor. So if i disable the UI scaling option within in the program, it's displayed extremely small (assuming it's running on a 3840-screen without any scaling at all). If i activate the UI scaling option, then the UI is scaled to fit in ... this is okay and as intended by the programmers, but for my taste this is way too big now – my preference would be no UI scaling and appearing as if it was running on a 1920x1080 screen.
If i change the settings in the windows compatibilty tab of the programs to option b), things get "better".
Text etc. renders sharp as intended, and the UI scaling seems to "assume" it's running on a 1920x1080 screen, therefore doing what i'd want it to do – look big enough to use, but not as enormeous as in scaled mode. So everything is fine there ... except it's windows doing the job instead of the program – windows is telling the program it's running on a 1920 screen, and it's doing the sharper rendering for the program.
While this is okay, it's not said it's very future proof ... microsoft usually is keeping such features for very long, but it's always a complication compared to a clean solution.
just to test, i switched to option c) ...
the whole program then is defactor running as a 1920x1080 version – this works perfectly, but fonts are not sharp anymore. it's okay as a last resort, but contradicting the use of a highDPI screen.
So ... what should be done?
– first, make the 2.5 versions read the system scaling factor (if they don't already?).
– then, when the UI scaling option is disabled, don't just blindly set 1 program pixel to 1 display hardware pixel as currently, but respect the set system scaling factor. So if that is set to 200%, double the displayed size of the "non-scaled" UI, if the factor is set to 250%, scale it to that, etc.
i think it's not that much of a problem to make that work actually ... read the value, and handle the UI-scaling not as binary as currently (scale to max or scale not at all).
Extensions:
I'm having the same "issue" on my Pixel C android tablet – that has a pixel density of around 300ppi too. Disabling the UI scaling there results in tiny tiny UI elements as well, so the app is not reading the ppi value of the device either. As this is a touch device, i'm okay though with scaling the UI anyway to make it more touchable, so it's less of an issue there.
I know Apple is handing that topic diametrally to microsoft – it might be that this is no problem on their platforms. But but i still wonder what happens on iOS or MacOS if the UI scaling is disabled – does the UI stay "usable" or is it extremely tiny as well? As i don't have the games on that platforms, i can't test it.
As this topic is a bit difficult to describe, i'd like to discuss first to make a good bug ticket, so this can be done right ... it would serve the future proofness of the IE games a whole lot if they could handle pixel density right and natively.
i had filed this as a feature request for 2.3 / 1.4 games, but it's still an issue in the 2.5 betas, so i'm trying to file this again for those versions, but i'd like to discuss and clarify first where this is an issue ...
Until a few years, computer screens always had a ratio of physical dimensions and pixel dimensions that resulted in pixel densities of around 90-120 pixels per inch (let's say 100 pixels for simplicity). So if there was an element of 100x100 pixels on screen, it was (about) 1x1 inch in size. Programs and programmers relied on that, declaring sizes in pixels, not caring about real physical size (as it was always okay).
With mobile devices and their small screens, but same pixel dimensions, pixel density rose - some had around 200ppi, others around 300ppi, etc. so an element of 100x100 pixels is way smaller on those screens. So on mobile devices, dimensioning early on became relative to the physical screen size, not relying on pixel dimensions anymore. Fonts, images, etc. stayed the desired physical size, and the additional pixels available were used to render them sharper.
This finally arrived on notebooks and desktop computers as well the last years ... apple was defacto first with a usable variant of this (working for exactly their systems only though), microsoft was doing it "right" in the long run, which resulted in "wrong" for quite a while.
But at least with windows 10 creators update things are good now for almost all use cases.
One can set a scaling factor in the systems screen resolution settings (like "try to display everything 200% of the size usually calculated, and make things sharper instead of smaller" – this can be a setting between 100 and 500%, but values between 150 and 300 are the most common).
Then, there are basically 3 modes that can be set per program:
a) default and the best: let the program handle the high density (It has to know about the scaling factor set, and render sharper instead of smaller according to this).
b) let windows 10 do the best (the program does not know about scaling but id programmed well – windows does render everything sharper that can be done, like type, and scaling everything that can't, like images). this works well for most modern windows programs.
c) let windows 10 force a scaling up in a brutal way: make the program pixels double/triple size ... this makes the programs blurry / low resolution, but they work. this is for the old programs around – if this would not be done by force, the programs would render each of their pixel to one physical pixel, making it appear way too small.
So i'm currently testing the 2.5 betas on my windows 10 fall creators update system, that has a physical screen resolution of 3840x2160 Pixels (that is twice the standard 1920x1080 in each direction), a 15,6" size, resulting in a density of almost 300ppi. My system scaling factor is set to 200%, instructing aware programs to display the same size as on a standard 1920 screen, but sharper.
The Betas are somewhat, but not completely?, aware of this in default mode a):
They render the text sharp, the scale the mouse cursor right (well, at least in software cursor mode), etc. – so they "know". but there is one big issue: the UI scaling is not respecting the systems scaling factor. So if i disable the UI scaling option within in the program, it's displayed extremely small (assuming it's running on a 3840-screen without any scaling at all). If i activate the UI scaling option, then the UI is scaled to fit in ... this is okay and as intended by the programmers, but for my taste this is way too big now – my preference would be no UI scaling and appearing as if it was running on a 1920x1080 screen.
If i change the settings in the windows compatibilty tab of the programs to option b), things get "better".
Text etc. renders sharp as intended, and the UI scaling seems to "assume" it's running on a 1920x1080 screen, therefore doing what i'd want it to do – look big enough to use, but not as enormeous as in scaled mode. So everything is fine there ... except it's windows doing the job instead of the program – windows is telling the program it's running on a 1920 screen, and it's doing the sharper rendering for the program.
While this is okay, it's not said it's very future proof ... microsoft usually is keeping such features for very long, but it's always a complication compared to a clean solution.
just to test, i switched to option c) ...
the whole program then is defactor running as a 1920x1080 version – this works perfectly, but fonts are not sharp anymore. it's okay as a last resort, but contradicting the use of a highDPI screen.
So ... what should be done?
– first, make the 2.5 versions read the system scaling factor (if they don't already?).
– then, when the UI scaling option is disabled, don't just blindly set 1 program pixel to 1 display hardware pixel as currently, but respect the set system scaling factor. So if that is set to 200%, double the displayed size of the "non-scaled" UI, if the factor is set to 250%, scale it to that, etc.
i think it's not that much of a problem to make that work actually ... read the value, and handle the UI-scaling not as binary as currently (scale to max or scale not at all).
Extensions:
I'm having the same "issue" on my Pixel C android tablet – that has a pixel density of around 300ppi too. Disabling the UI scaling there results in tiny tiny UI elements as well, so the app is not reading the ppi value of the device either. As this is a touch device, i'm okay though with scaling the UI anyway to make it more touchable, so it's less of an issue there.
I know Apple is handing that topic diametrally to microsoft – it might be that this is no problem on their platforms. But but i still wonder what happens on iOS or MacOS if the UI scaling is disabled – does the UI stay "usable" or is it extremely tiny as well? As i don't have the games on that platforms, i can't test it.
As this topic is a bit difficult to describe, i'd like to discuss first to make a good bug ticket, so this can be done right ... it would serve the future proofness of the IE games a whole lot if they could handle pixel density right and natively.
1
Comments
I have seen that you have filed / combined this topic in redmine for the 2.5, thanks for that.
I've contemplated again about this, and while Thels is totally right that a UI size slider would perfectly solve this issue for UI scaling.
Anyway, being aware of the screen size scaling factor of the system still might make the IE:EE engine more futureproof for the years to come ... currently, those 4k highres screens may be exotic yet, but sooner or later they will be more and more common in laptops and desktop computers (just as they are on mobile devices now).
So, while it works somehow, it might be no bad idea if the engine does not think it is running on a (physically huge) 3840x2160 pixel 4k TV screen, but on a small 15" notebook screen with the same pixel resolution, but far higher PPI density and a scaling factor to adjust size.
Besides the UI scaling issue, one bad side effect of not really being aware of the systems scaling factor is that the hardware mouse cursor is only half the size it "should be" (compared to the software cursor). It's a sign something is not really right currently. And while it's just not as painful as it could be, looking some years down the road this could become a severe compatibility issue no one will fix anymore by then ...
It might be that text size calculation etc. is affected by this the one or the other way too (i need to investigate further though). My text size slider gives me 4 totally oversized font size options, and only the lowest three seem to be in a range that someone might want to consider to play with. Top of the three usable is "really large", second is "this is the right one", last is "i want to save space and my sight is good" ... all the others are way above anything that would be needed for better readabilty, and those 4 don't really fit into the UI. So it seems something is out of range there on my system at least (testing BG2:EE and IWD:EE 2.5 betas).Addition: i just checked font sizes, and it seems this is a problem too ...
I am testing with BG2:EE 2.5beta on Windows 10 latest version.
Screen has 15,6", 3840x2160 pixels resolution.
the windows scaling factor 200% (that equals 1920x1080 in size, but has "double sharpness").
1) Running the program without any windows compatibility settings:
- Program assumes it's running on a 3840x2160 screen (it is not reading the scaling factor)
- UI unscaled is totally unusable, UI scaled is way too large (personal taste).
- hardware mouse cursor is way too small. software mouse cursor is "right".
- font sizes considered to be "sanely useable": slider position 1, 2 and 3 (all above steps are enormeous to monstreous, none seems to be usable or useful)
2) setting the windows compatibily mode for the baldur.exe to let windows handle hiDPI- Program assumes it's running on a 1920x1080 screen (Windows is telling this so)
- UI unscaled is as i'd wish it would be, UI scaled is the same as above
- hardware mouse cursor has the right size. software mouse cursor is right.
- font sizes considered to be "sanely usable": slider position 2, 3, 4, 5, 6 (step 1 is too small even for 1920x1080, step 7 is starting to damage the UI)
So it's not only the UI scaling that is a problem, it's affecting more aspects that may appear minor right now, but might become a severe problem in the future, when no patches will be coming anymore to fix that ...To be future proof, whether 4k, 5k, fullHD or whatever shouldn't really matter, it would be good if the game was really pixel dimensions independent, by respecting the ppi values of the device, either directly (which is harder) or by the set scaling factor (thats not even windows specific, osx has a similar thing, your scaling factor on the iMac usually is set to 2x/200%, and linux has the same concepts, at least I know about gnome).
@JuliusBorisov : thanks for pointing to the right redmine thread, i will ad my suggestions there ... Didn't want to flood the bug tracker with just-another-thread-of-the-same.