With your chosen controller connected, load a core, set your controls, and choose to save core override. This should save that specific config for that core.
I use iBuffalo SNES pads for most of the 2D stuff and my Xbox 360 controller for PSX and N64 etc. RA will load the correct core override on loading that core.