I certainly get this message much, much, much more often than I actually ever retract my vote.
This seems like a bug.
In fact, I think I often find that my 'close' vote isn't there anymore even though I haven't retracted it. When I try to vote to close again, that's the message that I get.
I used to think that it was due to my making an edit to a question after voting to close it. But now I have an example of a question to which I have made no edits and yet my close vote is gone and I can no longer vote to close.
If this is the intended behavior, I don't know what the desired goal is.