Basically, when a villager moves out, your game remembers having them in your town at some point. In order to have them move back in, or for your game to forget that you ever had them in your town, you have to have 16 villagers move in/out
after that one villager moved out. This is known as villager cycling, as you are cycling through villagers in order to make your game forget them and give them to you again. When you've completed the cycle, that villager will stop appearing on your Main Street.
Does that make sense or did I explain it really badly?