Each Pokémon in the Safari Zone has a set catch rate and an escape rate.
Throwing Bait lowers the escape chance by 1 stage, but it also has a 90% chance of lowering the catch rate by 1 stage.
Throwing Mud, however, has the opposite effect; it makes the catch rate higher by 1 stage and the escape chance higher by 1 stage.
These stages are similar to Pokémon stat boosts. 0 is the starting stage and it can be raised to 6 stages or -6 stages.
According to Bulbapedia:
At the end of each turn, a random number between 0 and 255 is generated. If it is less than the modified stat stage, the Pokémon escapes.
This determines whether it escapes or not.
I hope this helped!
Source: Johto Safari Zone