In Prague, I second the suggestion to see the Municipal House (you can get behind the scenes tour tickets in the gift shop). It appears the tour covers this "when available" so ask your guide on the first day if you're going to go (then try to get tickets during free time). Frankly, I'd go twice, it was such a stunning building. I spent two full days in Prague ahead of the Best of Eastern Europe tour. On one day I took an organized tour to Kutna Hora. The tour guide was horrible, but I loved the small town and the train ride out and back. I also suggest getting up early to take pictures - I took some stunning shots of a basically empty Charles Bridge.
In Budapest, for sure go to the baths. Fantastic experience. It looks like you have pretty full evenings there, but if you decide to stay another day or two, I enjoyed sitting in the (very) cheap seats at the opera house. Our tour spent only 2 days there and I wished I had a full two days more.