Wrong resolutions detected with multi monitor setup and scaling

[yaun]

Apologies if this is a stupid question, but is it possible to use OBS while having font scaling >100% active on any display? I could not get it working unless I set everything to 100% (even the displays that are not captured).

I am using a fresh install of Mint 22 Cinnamon with a 3 display setup of one 1920x1080 and two 2560x1440 displays.
For all 3 displays I configured scale of 125% in the gnome settings and I am trying to capture the 1920x1080 display.

$ xrandr |grep "+"
DisplayPort-2 connected 3072x1728+0+788 (normal left inverted right x axis y axis) 345mm x 195mm
1920x1080 60.00 + 60.00* 50.00 48.01 59.94
HDMI-A-4 connected primary 4096x2304+3072+0 (normal left inverted right x axis y axis) 597mm x 336mm
2560x1440 59.95*+ 74.97 72.00 60.02 50.01 48.00
HDMI-A-5 connected 4096x2304+7168+0 (normal left inverted right x axis y axis) 597mm x 336mm
2560x1440 59.95*+

I am not sure from where xrandr is getting these fantasy resolutions, but this is what OBS is using:

The odd effect of this is that since OBS thinks the configured canvas is smaller than the display, only part of the display is captured.

If I reset the scale for all screens to 100%.

$ xrandr |grep "+"
DisplayPort-2 connected 1920x1080+0+763 (normal left inverted right x axis y axis) 345mm x 195mm
1920x1080 60.00 + 60.00* 50.00 48.01 59.94
HDMI-A-4 connected primary 2560x1440+1920+0 (normal left inverted right x axis y axis) 597mm x 336mm
2560x1440 59.95*+ 74.97 72.00 60.02 50.01 48.00
HDMI-A-5 connected 2560x1440+4480+0 (normal left inverted right x axis y axis) 597mm x 336mm
2560x1440 59.95*+

And everything now works as expected.

I was now hoping that I can just set the 1920x1080 display which I want to capture too 100%, while leaving the others with 125%.

But I get exactly the same problem as I had when having everything at 125%

$ xrandr |grep "+"
DisplayPort-2 connected 3840x2160+0+571 (normal left inverted right x axis y axis) 345mm x 195mm
1920x1080 60.00 + 60.00* 50.00 48.01 59.94
HDMI-A-4 connected primary 4096x2304+3840+0 (normal left inverted right x axis y axis) 597mm x 336mm
2560x1440 59.95*+ 74.97 72.00 60.02 50.01 48.00
HDMI-A-5 connected 4096x2304+7936+0 (normal left inverted right x axis y axis) 597mm x 336mm
2560x1440 59.95*+

And OBS is showing

Is there any way to make this work?

 

[yaun]

I now better understand the problem: The reason is that Gnome applies fractional display scaling by first scaling the resolution to 200% across all displays and then fractionally downscaling (see also my question in the Mint forum: https://forums.linuxmint.com/viewtopic.php?p=2533945#p2533945)

There is a workaround for OBS: Set the canvas size to the 'virtual' resolution shown by xrandr / OBS and then apply downsampling back to your native display resolution, and you get the expected video.

However, the more reasonable solution would be to just not use any fractional scaling while using OBS under Gnome.