If both players have chosen moves and one hits cancel, the server, which was processing the move choices, goes ahead with the battle as if nothing happened. If you cancel when the other player has not yet chosen, it properly works. Generally it is better to think about your move choice before doing anything, as using the cancel button while the battle is being calculated just causes lag.