Then it sounds like what @DOS76 said, that you have some overrides setup in which the filtering is on. Changing it with a core loaded and saving a core override will only change it for that core, doing it without a core loaded will turn it off globally for all cores, unless you have overrides for cores that are overriding that default setting.
Also do you have Retroarch set to to save a config on exit? If so turn that off, so that it only saves settings when you want it to.