I've only recently started building a collection once I fell in love with what Launchbox and especially Big Box have to offer. I'm currently at around 18.500 games.
My "rules" are that I start with the US complete set (although I'm not sure that all my sets are 100% complete) and add all the European exclusives. I excluded Japanese games for now, but I might add translated versions of some of them down the road. There are so many crappy games in this collection, and some games flat out don't work. However, this isn't much of a problem since I just create playlists for the top 50 to 100 games of each system.
Whenever possible, I compressed my games (PS2 to .cso, PS1, Sega CD and Sega Saturn to .chd, Wii to .wbfs, and so on.) I made all my setup portable and saved all of it on a 10TB external hard drive. I made backups of all my files on various other harddrives, but this solution is not ideal AT ALL. I'd much rather have a NAS Setup, but I'm too inexperienced for now and I don't want to spend too much cash.
I use retroarch for most of the systems I emulate, except for PS2, Wii, GC, and WiiU. PCSX2 works like a charm and Dolphin standalone is still better for wii games, especially if you want to use a real Wii mote. Dolphin also has a cool widescreen hack for GC games that does not stretch the image. Desmume and citra perform better standalone and also run more games that way. However, the retroarch cores emulate the touch feature much more conveniently to the controller.
I have to say that I'm pretty happy with my setup. I don't know what else I could add to it. XBOX games maybe if an emulator eventually gets the job done. But I think PS3 and XBOX 360 games are too big to be stored on drives for now. That would quickly blow up in size. I'm currently trying to get Philips CDI games running through retroarch's Mame core, which I couldn't get to work yet. Other than that, I'm pretty much settled for now.
Someone wrote that they had more than 50.000 games. I honestly can't imagine how this is possible. This must include all the versions of games from different regions as well.