This itinerary will keep you quite busy, but it is doable. It also offers a nice variety, allowing you to see castles within the heart of Bavaria and also to explore two wonderful European cities. I would suggest filling the blank spot in your itinerary with returning to Munich, since you will not have even a full day there otherwise. This will give you almost two days in Munich in addition to the other sights you will be visiting.
Since you will be traveling by car, keep in mind that when you are heading to Neuschwanstein Castle, road signs in the region will refer to it as Königsschlösser and not as Neuschwanstein. Reservations are recommended during peak season and can be booked online www.ticket-center-hohenschwangau.de.
Since one of your fellow travelers has difficulty walking a lot, I've included a relevant excerpt from Rick's Germany guidebook below:
Getting Up to the Castles: From the ticket booth, Hohenschwangau is an easy 10-minute climb (just zigzag up to the big yellow castle, following the signs), while Neuschwanstein is a moderately steep 30-minute hike in the other direction (also well-signed--the most direct and least steep approach begins across the street from the ticket center).
To minimize hiking to Neuschwanstein, you can take a shuttle bus (leaves every few minutes from in front of Hotel Lisl, just above ticket office and to the left) or a horse-drawn carriage (in front of Hotel Müller, just above ticket office and to the right), but neither gets you to the castle doorstep. The shuttle bus drops you off near Mary’s Bridge (Marienbrücke), leaving you a steep, 10-minute downhill walk to the castle--so be sure to see the view from Mary’s Bridge before hiking down (€1.80 one-way, the €2.60 round-trip is not worth it since you have to hike uphill to the bus stop for your return trip). Horse-drawn carriages (€6 up, €3 down) are slower than walking and stop below Neuschwanstein, leaving you a five-minute uphill hike. Here’s the most economical and least strenuous plan: Ride the bus to Mary’s Bridge for the view, hike down to Neuschwanstein, and then catch the horse carriage from the castle back down to the parking lot (total round-trip cost: €4.80). Carriages also run to Hohenschwangau (€4 up, €2 down).
Warning: Both the shuttle bus and the carriage can have long lines at peak times--especially if it's raining. You might wait up to 45 minutes for the bus, making it slower than walking. If you're cutting it close to your appointed time, you may need to hoof it.
