If you want to evolve it and use Florges competitvely, you don't have to have perfect Speed IVs. Florges specializes in SpAtk and has monster SpDef, so it won't need that much Speed in battling, as it can send Pokemon packing with a Moonblast or a Petal Dance. Florges also naturally outspeeds some Pokemon, as it has a Base Speed stat of 75.
If you want it to be a fast special sweeper, this could be a problem though...
All and all, missing the Speed IV is fine.