I genuinely think that Pokemon Amie is flawed. I've seen people have all the criteria for Sylveon (2 hearts and the Fairy Move), but have got Espeon or Umbreon instead. It's highly possible that Eevee's happiness overrides its affection when evolving.
I'm posting this as a comment because it's not 100% confirmed, but just be aware that this could happen.