Had to post this as I was having the same problem, unable to solve for an hour. Not sure why this would matter, but all I did to fix was go to the Dependency Files tab in the Emulator page for Retroarch. There's a drop-down for all platforms & their associated dependencies. I just chose Sega Dreamcast from the list, which doesn't do anything other than show me BIOS files for the associated core. I clicked OK from here and now my Dreamcast games launch fine with flycast_libretro in Retroarch. Maybe some kind of weird bug, but it worked.