It looks like the RetroArch core will support cheats from version 1.7.4 onwards.
https://www.libretro.com/index.php/upcoming-retroarch-1-7-4-cheat-code-searchingcreation-interface-with-rumble-features/
Yeah Demul emulates the Sammy Atomiswave, NAOMI, Hikaru, Gaelco, and Cave SH3 arcade boards. Some of these games got Dreamcast ports, but there is a lot of exclusive stuff worth checking out.
When you have a game loaded with the Stella core, go to the RetroArch Quick menu, select Controls, select Retropad w/ Analog as the User 1 Device Type, for User 1 Analog to Digital Type select Left Analog. Select Save Core Remap File. Now your analog stick will work aswell as the d-pad.
I've mentioned this to @Jason Carr in the past, I'd like to see all of those View options under one "View Options" menu item that expands when clicked. Perhaps he will look into this in a future update.
I would recommend Controller Companion, it's an app on Steam for €2.99. You can create unique profiles for any application and add your own hotkeys to exit, among other functions. I only got it recently and could not recommend it highly enough, it's a breeze to use.
Yeah I think you are having a conflict with controller automation and UI Cancel being set to the same buttons. Have you changed this in MAME and tested it? This resolved the issue for me.
Ok, you may have a conflict with your UI Cancel in MAME. I had this. Go into the MAME UI by pressing the Tab key and navigate to interface controls. Is it set to Esc? Change UI Cancel to something else, mine is Select + X.