I've been to all four of those cities.
Reykjavik is sort of out of place in your list. It's not really a major city. It's certainly worth visiting, but you can see most of what the city offers in a day, and the rest of the time you'll want to take some excursions to places like the Golden Triangle, the Snaefelsness peninsula, Thorsmork, and the Blue Lagoon.
The other three choices offer many attractions without leaving the city limits (though there are also plenty of things to see on the outskirts).
I would probably pick Reykjavik and one of the others. Iceland is so different from anywhere else in Europe, it's really an extraordinary place to see. The other three are really all equally worth visiting, so I can't pick one and say it would be my top over the other two.
I'd recommend you read up on the main things to see in each city and then decide what is most appealing to you.