wanna make sure so I can switch my gengar with no worries
No, it only breaks the Substitute, even if you're switching out.
Source: Logic and Experience