Nope, removing --nogui changes nothing. It still refuses to use the hotkeys.
Would you mind checking to see if your PCSX2 setup in Launchbox maybe has an auto hotkey config? I ask because escape still works for me and it is the only thing setup in my auto hotkey tab. I doubt this has anything to do with it either but I figure it couldn't hurt to check.
UPDATE: Wow I feel kinda dumb. So what was happening was GeForce Experience was showing a notification when the emulator was launching the game causing the emulator to lose focus and thus break my hotkeys... Turned off those notifications and like magic hotkeys work again haha. Something so simple causing so many problems. Thank you JamesBond@ge and CTRL-ALT-DEFEAT.