Just to add some more opinions and experiences (and that's all this is - there is no "right" or "wrong" answer to your question):
A friend recently went with her family to Prague, Vienna, and Budapest; they flew into and out of Munich, but did not see the city itself. They really liked Prague and Vienna and really did not like Budapest.
I myself really liked Prague and Budapest (would gladly return to either), and found neither Vienna nor Munich to my liking (not sorry I saw them by any means, but no need to return). I did not find any of the four to be similar to the others - at ALL. And while Prague was more touristed than the others, I was prepared for this and did not find it to be a problem (in September 2007). I used the same tourist-avoiding tricks that one uses in Venice, New York, etc. - stay off the main tourist drags at the main times of day.
My sister loved Prague. She did not warm to Budapest because she wanted it to be like Prague, and it wasn't (object lesson there). And she loathed Vienna, where she had continued problems with everyone treating her rudely (I had no difficulties of this sort at all - another reminder that we all have different experiences, despite going to the same places).
So there you are - flip a coin. Or, more realistically, omit the place where you might go easily on a future trip (as pointed out, Prague and Munich go well with Berlin, which I REALLY loved). Remember - it's all good, and what you miss this time, you will simply have to return to see.
If you do go to Budapest, I loved my day trip to Vac (not listed in Rick's book; I found out about it from Frommers). It's practically unvisited, and actually faster to get to than the more famous Szentendre (which is nice, but as touristy as Prague, and it's too small for you to evade the hordes). In Vac, the mummies found under the main square (you read that right) and the delicious ham and smoked cheese pizza (also not a typo) are reasons enough for the trip, in addition to the town being very pretty.
For the other cities, I was too busy in them to consider day trips; they all have LOTS to see. I did do overnights to Cesky Krumlov and Olomouc from Prague, and to Pécs and Eger from Budapest; all of these are 2-3 hours each way, and while some people do them as day trips, that's too far for me. I found Eger a disappointment, but recommend the other three. My sister also liked her trip to the silver mine in Kutna Hora, but warns that it's not suitable for anyone with even mild claustrophobia.