So all Pokémon in your party gain points with Exp. Share turned on AND your lead Pokémon gains 100% of the points it would normally gain, so why would I want to turn it off at all? is there an unknown down side?
Yeah you're Pokemon gain unwanted EVs from the Exp Share. Unless you count Lv.1's going all the way to Lv. 15-20 then that's the only downside.
The only true downsides to it are as follows: The Exp. Share can make the game a bit too easy. If you are trying to do something with a low level Pokemon without adding EV's and Exp. you should turn it off in case you get caught in trainer battle. I'm sure that there are a few other reasons that various people would turn it off for, but those are the most noticable and it really doesn't matter if you can or can't turn it off, but I suppose I can understand your curiosity.