Windows, non-Big Box mode. ROM is not an archive, it's .nds.
I was able to confirm it's not a baffling compatibility issue, because I was able to load one of the offending ROMs through MelonDS's menu without incident. It just won't launch from LaunchBox correctly.
I did some searching and other people have had similar problems but the solutions were irrelevant to my case (I'm not using an SD card and I always extract my ROMs first)