I think having an "ugly" or just not-as-cute villager gives a village character. I think if villages are too perfect, too clean, it's eerie and unsettling. I've also kind of felt bad about calling villagers ugly or being mean to them, even if I don't like one of mine.