Mutiple-Resolution Display Discussion

[Kratus]

@DCurrent A cool upgrade in this concept would be if we could have each screen with its own resolution. I tried to code it in the engine when I opened the poll for SORX resolution, but no success, maybe we could back to it in the future.
This will be useful to put the entire hud in a screen and reduce it without distortions (or reduce the gameplay area and maintain the hud).

 

[DCurrent]

@Kratus,

Sub-screen resolution is set at initialization, because the engine has to know exactly how much memory to allocate for it. It's a lot like an array in C.

You could still get a poly-resolution functionality, but you would need to first create a new screen and then transfer the old screen's sprite cache to it. Again, like an array in C.

DC

 

[Kratus]

@Kratus,

Sub-screen resolution is set at initialization, because the engine has to know exactly how much memory to allocate for it. It's a lot like an array in C.

You could still get a poly-resolution functionality, but you would need to first create a new screen and then transfer the old screen's sprite cache to it. Again, like an array in C.

DC

I tried to make something like that managing the drawscreen/drawspriteq/allocscreen functions but no matter what I do it seems that every screen always obeys the video.txt configuration and can't be changed, but your information about the memory explains why it happens.
I will make another try, thanks for the clarification

 

[DCurrent]

I tried to make something like that managing the drawscreen/drawspriteq/allocscreen functions but no matter what I do it seems that every screen always obeys the video.txt configuration and can't be changed, but your information about the memory explains why it happens.
I will make another try, thanks for the clarification

Maybe we are talking about different things. If you are talking about a sub-screen somehow breaking the limits of the main display resolution, no it can't do that. The main resolution is set by video text, and there's no changing it during runtime.

Screens exist completely independent of the main display resolution, just like a sprite. If the sub-screen has a higher resolution size than the main display, then the overhanging parts of the sub-screen just aren't visible.

DC

 

[Kratus]

Maybe we are talking about different things. If you are talking about a sub-screen somehow breaking the limits of the main display resolution, no it can't do that.

Exactly, my intent is to make every screen with its own resolutions independent from the video.txt. In the video below you can see a real example of what I'm talking about, this is the SOR2:NE project and in this game you are able to scale every element using its original resolution, even characters.

Of course we don't need to follow exactly the same, but having it on screens would be a huge step forward for the engine. Using SORX as an example, I would like to have the playable area using 420x240 and hud using 480x272 (the current resolution).

 

[DCurrent]

Exactly, my intent is to make every screen with its own resolutions independent from the video.txt. In the video below you can see a real example of what I'm talking about, this is the SOR2:NE project and in this game you are able to scale every element using its original resolution, even characters.

Of course we don't need to follow exactly the same, but having it on screens would be a huge step forward for the engine. Using SORX as an example, I would like to have the playable area using 420x240 and hud using 480x272 (the current resolution).

Now I understand, and actually you can do that. It will take some tricky math though, since you essentially need to embed a lower res screen into a higher res display. That means the lower res needs magnification to fill space correctly.

Not even that complex of a script really. You just have to work out the scaling factor, and set the "low res" screen accordingly. You'd also need to set up your stage boundaries and spawning to sync up correctly. Certainly possible, but definitely needs some good pre-planning.

DC

 

[Kratus]

Now I understand, and actually you can do that. It will take some tricky math though, since you essentially need to embed a lower res screen into a higher res display. That means the lower res needs magnification to fill space correctly.

It even works, but if the sub screen does not follow the video.txt resolution, it will be always distorted depending on the value you use to resize it.

In the image below I'm using a 396x224 screen for the battlefield in the video.txt, and the hud is shown by another screen rescaled to fit in the main screen using drawmethod scale function, once this hud was developed for the 480x270 resolution.

Currently the allocscreen function requires a resolution value but only changes the screen's size, not the pixel size.



However, if I double all the game's sprites to work on a resolution of 960x544 and then rescale it to 50% (to look like 480x270), it will fit perfectly. The idea is to have more freedom for custom resolutions instead of only round values of 200%, 50% or 25% (the only ones where sometimes we can avoid distortions).

 

[DCurrent]

It even works, but if the sub screen does not follow the video.txt resolution, it will be always distorted depending on the value you use to resize it.

In the image below I'm using a 396x224 screen for the battlefield in the video.txt, and the hud is shown by another screen rescaled to fit in the main screen using drawmethod scale function, once this hud was developed for the 480x270 resolution.

Currently the allocscreen function requires a resolution value but only changes the screen's size, not the pixel size.

View attachment 7094

However, if I double all the game's sprites to work on a resolution of 960x544 and then rescale it to 50% (to look like 480x270), it will fit perfectly. The idea is to have more freedom for custom resolutions instead of only round values of 200%, 50% or 25% (the only ones where sometimes we can avoid distortions).

Interesting. I'll be watching with interest. I can say right up front it's not really practical for the engine to support multi-resolution rendering natively. That would in effect, be duplicating the functionality of sub-screens.

In fact, I've never heard of any engine doing that, unless it was in entirely separate windows. Unity sort of can, but you have hack it with camera views, and it's really jankey. Unreal can pull it off using custom render targets, but in principal they are virtually identical to OpenBOR's sub-screens.

DC

 

[O Ilusionista]

In fact, I've never heard of any engine doing that, unless it was in entirely separate windows

Mugen and ikemen does that. You can have a character in one resolution, the hud in other, the stage on another and so on

 

[Kratus]

Interesting. I'll be watching with interest. I can say right up front it's not really practical for the engine to support multi-resolution rendering natively. That would in effect, be duplicating the functionality of sub-screens.

In fact, I've never heard of any engine doing that, unless it was in entirely separate windows. Unity sort of can, but you have hack it with camera views, and it's really jankey. Unreal can pull it off using custom render targets, but in principal they are virtually identical to OpenBOR's sub-screens.

DC

Mugen and ikemen does that. You can have a character in one resolution, the hud in other, the stage on another and so on

The SOR4 engine does something like that to rescale the classic characters to fit the same proportion of the hd characters, without distorting the original sprites.

 

[DCurrent]

Mugen and ikemen does that. You can have a character in one resolution, the hud in other, the stage on another and so on

Guys, we're not talking about the same thing.

The base resolution simply cannot change like that mid draw, especially around an uneven object. All that's going on under the hood is a carefully controlled rescale.

You cannot stick a 1080p thing into a 480p output and it just magically keep original resolution. Math says no. It either breaks boundaries or you downsample.

You can stick a 480p thing into a 1080p ouput and keep the quality, but obviously you have to upscale to maintain size. That's what Unreal, SOR4, Mugen, and whoever else does, and it's what we can do too.

I think what you're really asking for are different resample algorithms other than nearest neighbor so the scaling can be in different increments.

DC

 

[O Ilusionista]

I think what you're really asking for are different resample algorithms other than nearest neighbor so the scaling can be in different increments.

Ah in Mugen case, that is not what I was talking about. I can really change the resolution (not scale) - we have a thing called "local coords", which will take care of it.
It affects not only the size, but position and, if I am not mistaken, velocities too. Scale doesn't scale the collision boxes, but localcoord does.

For example, if my character uses a scale of 0.5 and I use code that doubles his size (2.0):
- if I use scale, its final proportion will be 2.0, as it is arbitrary (unless I use code to deal with it)
- if I use localcoord, the final ratio will be 1.0, as it takes into account

value = Const720p(12)
  Sets value 3 if the player has a coordinate space of 320x240 (240p).
  Sets value 6 if the player has a coordinate space of 640x480 (480p).
  Sets value 12 if the player has a coordinate space of 1280x720 (720p).

Converts a value from the 720p coordinate space to the player's coordinate space. The conversion ratio between coordinate spaces is the ratio of their widths

 

[DCurrent]

Ah in Mugen case, that is not what I was talking about. I can really change the resolution (not scale) - we have a thing called "local coords", which will take care of it.
It affects not only the size, but position and, if I am not mistaken, velocities too. Scale doesn't scale the collision boxes, but localcoord does.

For example, if my character uses a scale of 0.5 and I use code that doubles his size (2.0):
- if I use scale, its final proportion will be 2.0, as it is arbitrary (unless I use code to deal with it)
- if I use localcoord, the final ratio will be 1.0, as it takes into account

value = Const720p(12)
  Sets value 3 if the player has a coordinate space of 320x240 (240p).
  Sets value 6 if the player has a coordinate space of 640x480 (480p).
  Sets value 12 if the player has a coordinate space of 1280x720 (720p).

Converts a value from the 720p coordinate space to the player's coordinate space. The conversion ratio between coordinate spaces is the ratio of their widths

Right, but that's still scaling, just more comprehensive. You're not changing the amount of available pixels in an area. Display outputs just can't do that.

Incidentally, those properties (ex. movement ratio) are available in OpenBOR too. The only thing that really doesn't have a scale adjustment OpenBOR is collision.

DC

 

[Kratus]

I can really change the resolution (not scale) - we have a thing called "local coords", which will take care of it.

Interesting, in OpenBOR terms it's like having each screen with its own resolution, right? This explains why some Mugen games show low res characters and high res HUD/fonts.

You're not changing the amount of available pixels in an area

Hmm, in this case I think the SOR2:NE is more advanced than I thought, the dev is using Unity. Even if he is just using a different way of scaling, the method is so good to the point where rescaled sprites using any proportion (i.e. 50%, 30%, etc) are shown without any distortion, they look exactly as they are in the original games.
My point is to do the same thing but using screens.

 

[DCurrent]

Interesting, in OpenBOR terms it's like having each screen with its own resolution, right? This explains why some Mugen games show low res characters and high res HUD/fonts.

View attachment 7098


Hmm, in this case I think the SOR2:NE is more advanced than I thought, the dev is using Unity. Even if he is just using a different way of scaling, the method is so good to the point where rescaled sprites using any proportion (i.e. 50%, 30%, etc) are shown without any distortion, they look exactly as they are in the original games.
My point is to do the same thing but using screens.

View attachment 7097

That's admittedly going to be tough in OpenBOR because resampling is always nearest neighbor. Unity (IIRC) also has access to bicubic and bilinear.

You'll need to be really careful with your ratios to avoid distortion. On the other hand, the motion will be better in OpenBOR because OpenBOR "thinks" in floating points, whereas without some difficult tricks, Unity is in whole pixels.

DC

 

[DCurrent]

@Kratus,

After having a second look (I was on my phone before), I believe it's actually much LESS advanced than you think. I checked into Unity, and there's no great way to do clean resamples at runtime like you're seeing in that screenshot. It can be done, but there's a lot of complexities involved and still won't be perfect. It's far, FAR more likely the resampling was done to the assets themselves.

Same for SOR4. They aren't taking old Streets of Rage assets and resampling on the fly. Those were resampled during development.

Mugen can largely automate that process as @O Ilusionista mentioned, but as we've talked about before, you can do more in runtime with Mugen because generally you have a smaller computational load to worry about.

DC

 

[Kratus]

It's far, FAR more likely the resampling was done to the assets themselves

About the SOR2:NE, according to the developer's instructions and some modders tests, the assets are resampled on-the-fly, all they need to do is to configure the ratio using scripts in .json format. You can see an example made by the developer itself in the video below (after 01:55).

Same for SOR4. They aren't taking old Streets of Rage assets and resampling on the fly. Those were resampled during development.

Hmm I extracted all the classic character's sprites from SOR4 using the sor4explorer app and they have the same size as the original. There's some resample trick on the fly.


I made some searches about this topic in Unity docs and it seems that the engine has some native tools to do it.

How to choose the resolution of your 2D assets | Unity Blog | Unity Blog

Many Game creators often do not know what resolution their assets should be. The answer to this question depends on a variety of factors, read more to find out.

blog.unity.com

But this is not top priority anyway, but at least if we can make something closer in future updates would be a cool feature.

 

[DCurrent]

About the SOR2:NE, according to the developer's instructions and some modders tests, the assets are resampled on-the-fly, all they need to do is to configure the ratio using scripts in .json format. You can see an example made by the developer itself in the video below (after 01:55).

Hmm I extracted all the classic character's sprites from SOR4 using the sor4explorer app and they have the same size as the original. There's some resample trick on the fly.
View attachment 7103View attachment 7104

I made some searches about this topic in Unity docs and it seems that the engine has some native tools to do it.

How to choose the resolution of your 2D assets | Unity Blog | Unity Blog

Many Game creators often do not know what resolution their assets should be. The answer to this question depends on a variety of factors, read more to find out.

blog.unity.com

But this is not top priority anyway, but at least if we can make something closer in future updates would be a cool feature.

All my searches on Unity turned up the opposite. That while it can do resamples, it's a PITA.

To be honest, I'm impressed that SOR4 and SOR2:NE are live resampling, but from a programmer's standpoint, it's a parlor trick. There's no practical value in it, which is why I'm shocked to hear they would do it. Why spend CPU and video cycles on live resamples, deal with all those ratios, etc. when you could just resample in development once and its done? That's production time that could be used for anything else, and CPU time for cooler effects, scaling tricks, and such that actually do something.

Oh well. Doesn't change what I said earlier. You can do this in OpenBOR, but you'll need to plan carefully to get your ratios right.

DC

 

[DCurrent]

Moving the derailment here.

I'm actually considering this for my own project @Kratus. I wanted a bigger space than 480p to work with in my HUD and select screens but not main game-play. I just can't realistically produce HD assets. It never occurred to me mixing high/low res could be viable until you brought it up.

I'll die on the hill that runtime up-scaling to higher res is just silly unless there's a very specific edge use case. However, I might double my display resolution to 960p and upscale my assets *2 in production.

DC

 

[Kratus]

I wanted a bigger space than 480p to work with in my HUD and select screens but not main game-play

Yeah, I think it's a very useful trick for huds, it can be currently done easily in the engine without distortion if you are using proportions of 200% or 50%. In my case it's harder because I need a hud 30% smaller or the battlefield 30% bigger.

I could even double all my assets size and increase the resolution, but will still need to adjust it by 30%. In addition, I will need to remake the entire game, which can be a pain due to many "at" adjustments in the levels, scripts based on screen size, etc.

However, it sends me to a second idea that could be applied easier than the resample feature. Some time ago I was doing tests in the engine trying to separate the level resolution from the screen resolution, because currently both are always connected. As an example, if both works separately I could double the base resolution in the video.txt and instead of updating all my levels, I can simply define a openborvariant called "level resolution" to half of the main resolution but putting the entire battlefield in a new screen and double it using drawmethod.

May looks confuse in my description but I recorded a video showing it. In the video below I doubled the SORX resolution from 480p to 960p and put the battlefield into a screen with size x2 using drawmethod. However, you can see that some things like camera will act following the 960p behaviour, not 480p, and here is where the separated level resolution variant will act. This way we can easily avoid resizing at least all the level assets, only the ones related to menu, bgs, select screen, etc.

PS: The battlefield is around 30% bigger, you can see the difference compared with the second test in the same video.

Here's the script I'm using as a draft for experiments, in case you want to make some tests. Simply calling it in the updated.c and filtering using openborvariant("in_level") is enough to make it work. Use 0 to apply on hud or 1 to apply in the level.

void drawLevel()
{//Used to test different HUD sizes
    void vScreen    = openborvariant("vscreen");
    int draw        = 0; //0=HUD OR 1=LEVEL

    //CREATE A SCREEN AND SAVE IN A VARIABLE
    if(getglobalvar("allocScr_levelScreen") == NULL()){
        setglobalvar("allocScr_levelScreen", allocscreen(openborvariant("hResolution"), openborvariant("vResolution")));
    }
    if(getglobalvar("allocScr_hudScreen") == NULL()){
        setglobalvar("allocScr_hudScreen", allocscreen(480, 272));
    }

    //CLEAR ANY PREVIOUS SCREEN VALUE
    clearscreen(getglobalvar("allocScr_levelScreen"));
    clearscreen(getglobalvar("allocScr_hudScreen"));

    //DRAW EVERYTHING TO THE SCREEN (DRAW HUD)
    if(!openborvariant("pause")){

        //HUD
        if(draw == 0){
            drawspriteq(getglobalvar("allocScr_levelScreen"), 0, openborconstant("MIN_INT"), openborconstant("MAX_INT")/1000000, 0, 0);
            drawspriteq(getglobalvar("allocScr_hudScreen"), 0, openborconstant("MAX_INT")/1000000, openborconstant("MAX_INT")-10, 0, 0);
            changedrawmethod(NULL(),"reset", 1);
            drawscreen(getglobalvar("allocScr_levelScreen"), 0, 0, openborconstant("MAX_INT")-5);
            changedrawmethod(NULL(),"enabled", 1);
            changedrawmethod(NULL(),"transbg", 1);
            changedrawmethod(NULL(),"scalex", 210);
            changedrawmethod(NULL(),"scalex", 210);
            drawscreen(getglobalvar("allocScr_hudScreen"), 0, 0, openborconstant("MAX_INT")-4);
            drawspriteq(vScreen, 0, openborconstant("MAX_INT")-5, openborconstant("MAX_INT"), 0, 0);
            clearspriteq();
        }else

        //LEVEL
        {
            drawspriteq(getglobalvar("allocScr_levelScreen"), 0, openborconstant("MIN_INT"), openborconstant("MAX_INT")/1000000, 0, 0);
            changedrawmethod(NULL(),"reset", 1);
            changedrawmethod(NULL(),"enabled", 1);
            changedrawmethod(NULL(),"scalex", 512);
            changedrawmethod(NULL(),"scaley", 512);
            drawscreen(getglobalvar("allocScr_levelScreen"), 0, 0, openborconstant("MAX_INT")/1000000);
            drawspriteq(vScreen, 0, openborconstant("MAX_INT")/1000000-1, openborconstant("MAX_INT"), 0, 0);
            clearspriteq();
        }
    }
}