Proposed fix for Mantis issue 0000925 / github issue 4408: User Hotkeys trigger even when extra meta/modifier keys are held

[John Hartman]

I like hotkeys, and the inability to have X, Ctrl-X, and Alt-X (for example) operate as independent hotkeys inspired me to see what was going on.

The fix is actually pretty simple, depending on whether we want backward compatibility and/or UI. Details at "PROPOSED CHANGE" below, but the purpose of this post is get discussion about the scope of the fix. I see several possible paths:

1) Change hotkeys.c to ALWAYS operate so that X and Ctrl-X etc. are independent. This would break anyone who LIKES the current mode (where hotkey Ctrl-X also triggers hotkey X) but eliminates the need for UI or documentation of "how to change mode".

2) Change hotkeys.c to use the existing strict_modifiers flag to allow switching between the two modes of operation. In that case
? What should the default state of strict_modifiers be? Currently false. I would propose true
? How should strict_modifiers be set and cleared?
option a) Call the function obs_hotkey_enable_strict_modifiers from a Lua or Python script if you don't like the default behavior (and document this)
option b) Add a boolean value strict_hotkeys to the [general] section of global.ini, edited by hand if you don't like the default (and document this)
option c) Add a checkbox under Settings, Advanced.

Can anyone think of a plausible use-case for WANTING the current behavior?
That is, if I define hotkey actions for "1", "Ctrl-1", "Alt-1", and "Ctrl-Alt-1", then press "Ctrl-Alt-1", that I would want ALL FOUR ACTIONS to occur?
Or even to have "Ctrl-1" trigger both the "Ctrl-1" and "1" hotkey actions?

Bug reports
https://github.com/obsproject/obs-studio/issues/4408
https://obsproject.com/mantis/view.php?id=925
https://obsproject.com/mantis/view.php?id=1468
https://obsproject.com/mantis/view.php?id=1571 (dupe of 1468)

Forum posts complaining about the current behavior
https://ideas.obsproject.com/posts/...at-ctrl-alt-x-doesnt-trigger-ctrl-x-and-alt-x
https://obsproject.com/forum/thread...iably-for-scene-selection.146242/#post-535412
https://obsproject.com/forum/threads/hotkey-modifier-bug.79197/#post-333409
https://obsproject.com/forum/threads/please-fix-the-hotkeys.79150/#post-333256
https://obsproject.com/forum/threads/hotkey-modifier-issue.78002/#post-329251

Pull request from 2020 that attempts to fix this (This has a number of problems, both in functionality and in coding)
https://github.com/obsproject/obs-studio/pull/2785

PROPOSED CHANGE (using strict_modifiers to let the "old way" be selected if desired):
In obs_hotkey.c I noticed the "strict_modifiers" flag. This defaults to false, and there is no UI to set it, but the function
obs_hotkey_enable_strict_modifiers has Lua and Python wrappers (it isn't otherwise called from OBS code)

Alas, with the current code setting strict mode in a Lua script doesn't fix the problem: it is "strict" only when at least one modifier is used, due to a problem in modifiers_match:

static inline bool modifiers_match(obs_hotkey_binding_t *binding,
uint32_t modifiers_, bool strict_modifiers)
{
uint32_t modifiers = binding->key.modifiers;
return !modifiers ||
(!strict_modifiers && (modifiers & modifiers_) == modifiers) ||
(strict_modifiers && modifiers == modifiers_);
}

The "!modifiers" line bypasses "strictness" if there are no modifiers. Removing the !modifiers checks corrects the problem:

• If strict, the current modifiers (including NO modifiers) must exactly match the hotkey definition
• If not strict, require only the modifiers (including NO modifiers) requested by the hotkey definition so there is no change in current non-strict behavior.

So the function becomes

static inline bool modifiers_match(obs_hotkey_binding_t *binding,
uint32_t modifiers_, bool strict_modifiers)
{
uint32_t modifiers = binding->key.modifiers;
return (!strict_modifiers && (modifiers & modifiers_) == modifiers) ||
(strict_modifiers && modifiers == modifiers_);
}