My original plans called for 3 days in Munich with a day trip down to Fussen to do the castles. Afterwards it's on to Salzburg. The more I read about the Fussen/Schwangau area, the more I think we should stay down there and make a day/overnight trip up to Munich. We are much more interested in seeing waterfalls, gorges, and castles than museums. (my personal goal in Munich is to have a beer at the Hofbrauhaus. After that, I'm good) We will be traveling mainly by train, but may rent a car in Salzburg to see some of the outlying waterfalls and mountain areas. So what would be a better home base for this leg of the trip? Schwangau? Fussen?
Schwangau, Hohenschwangau, and Füssen are all close to each other with bus connections between them. The train connection is in Füssen. I'd say where to stay would depend on what you can find for accommodations. There is a little more to do in Füssen; the other two towns would be smaller, more intimate. You could probably walk from Schwangau to the castles.
You should most DEFINITELY devote some days to the Schwangau area. This is also the area where you'll need a car. Our favorite place to stay is in the little town of Schwangau-Horn just below the castles with fabulous views to the castles at night. Day trips include Linderhoff, Ettal, Oberammergau, Mittenwald. Lots of trips to do from here and SUCH a beautiful area you'll be glad you did.