On focusing on monuments and buildings....I can recommend tracking down several esoteric historical and war monuments, some of which I myself made it a point to get to. As an introduction I suggest Heroes Square where you see which ones are familiar to you. I found this place very enlightening, same goes for its counterpart in Vienna, the Heldenplatz, opposite of the Maria Theresien Platz.
There is one statue/monument common to both Budapest and Vienna, ie, that of Prinz Eugen, the Habsburg dynasty's victorious commander who never lost a battle, located on Castle Hill in front of the National Gallery. That building is worth seeing even from the outside since it was the Habsburg Royal Palace. Seeing Prinz Eugen there was a surprise to me historically, only in one way.
On buildings: see also the Operette House and the Parliament building, even if you only have time to see them from the outside, as well as the Mathias church. It was the location where the last Habsburg was crowned King of Hungary. Like in Vienna and Berlin, Budapest has its war monument, the Soviet War Memorial pertaining to WW2. A recent monument that generated controversy when it was put up is the German Invasion Memorial. With the three full days plan it carefully, you'll get to these buildings, view the inside too and still have time to see the monuments.