Here's what happened.
My Drifloon in Alpha Sapphire had a 4-heart affection, with fullness and enjoyment maxed.
I deposit it into the PC, send it to Pokemon Bank and subsequently transfer it to Pokemon Y (back to its original trainer).
I start the game in Pokemon Y, not moving Drifloon around at all, do a few minor things like Loto-ID and buy a few things in the game.
I save the game, send it back to Pokemon Bank and back to back to Alpha Sapphire.
I withdraw Drifloon back into my party and open Pokemon-Amie.
I find that its affection now is reset to 0, and fullness has dropped by one.
What happened? I have a Crobat that was maxed affection (and fullness and enjoyment) who followed the exact same path as Drifloon, and its stats did not change. According to Bulbapedia, transferring Pokemon through Pokemon Bank shouldn't affect a Pokemon's Affection towards a given trainer.