I think you've got me on the right path. I was going directly into Mame and hitting tab, not hitting tab in Launchbox. I'm pretty sure both my keyboard and controllers are recognized (Twin USB Gamepad).
My coin and select buttons were mapped to buttons 4 and 5, not joystick 1 button 9 and joystick 1 button 10. I mapped them correctly and was able to play the game I pulled up. I then went in and mapped them under input (general) and they seem to be working on the other games I've pulled up too.
Now I'm looking for what I would consider the back button. I'm playing a game, I want to back out and look at all the games (previous screen). I see a "UI Back", but when I map it to buttons, it doesn't seem to work.
Only since I'm referring to button numbers, adding this photo for context.