"Bilbao has two good art museums, not just one."
Actually it has three (that I found - maybe there are even more). The Guggenheim is the most well known, and then there's the Bellas Artes (one of the largest in Spain). A third one that I probably enjoyed even more than those two is the Museum of Reproductions. It has exact copies, made from casts, of all sorts of great sculptures from all over western Europe (Rome, Vatican City, Paris, Berlin, Madrid and more). The Venus de Milo, the Winged Victory, the Pergamon Altar, and many more great pieces are all in one place, in a former church, for €3. As a further bonus, while the Guggenheim was crowded and the Bellas Artes had other visitors (although not so many that I couldn't enjoy it), the Museum of Reproductions had only one other visitor while I was there.
"The inland capital of Vitoria-Gasteiz (with a really nice, large medieval district) has two smaller art museums."
Here, my absolute favorite was the Bellas Artes. I enjoyed it even more than the Bellas Artes Museum in Bilbao. Room after room of great stuff, by artists I had never heard of or seen before, and again, I was one of a very few visitors. And it's free!
There's also the Artium (modern art) which was €5 and worth it, and the Bibat (two museums in one - archeology and playing cards), which was free. If you go to the Bibat, get the free audioguide before going to exhibits; it only covers a few pieces in the archeology museum (and the labeling here is only in Basque and Castilian), but covers each room in the playing cards museum.
To get between Vitoria and Bilbao, I took the La Union bus; they run every half hour, take about an hour (they use the highway) and cost €6.30. To get between Vitoria's bus station and the city center, take a taxi (less than €8).
I'm still working on my trip report (I was just in Bayonne, San Sebastian, Vitoria, and Bilbao), and I'll post the link here when it's ready.