What kind of controller are you using?
I use a PS4 controller with the DS4Windows software for xinput. I also enabled an option that hides the dinput device to avoid double inputs. The problem with that is, whenever my controller gets disconected, the software is unable to hide the dinput device, since RA captures it.
If I have maximum users set to 1, then I become unable to use my controller because the dinput device fills in port 1, and I have to restart RA. I also remember there being an option somewhere that limits menu control to the first player only, which would also give trouble in this situation. Otherwise, aside from the fact that I have 2 controllers detected in RA and my xinput device being in the 2nd port, I don't have any issues with not being able to reuse the controller after disconecting.