"Are there any must-go places in Germany that I have missed out? Or any that are not worth going? What would be a good amount of time to cover these cities?"
About "Neuschwanstein Castle" and how much time it takes... it is a nearly 5-hour round trip from Munich by bus and train. It was built relatively recently. The tour of this late-19th structure takes 30 minutes. It's not a true castle, but the private home of Bavarian King Ludwig II (who lived within its unfinished walls for a total of 6 months before his death.) A theatrical set designer made the place LOOK like a castle, on the outside anyway. Since Ludwig died, it has had no purpose other than as a destination for tourists, a lot of whom probably think they're touring a castle because they've been to Disneyland and hey, N'stein looks just like Cinderalla's castle. Is it worth your time? Not IMO, that is, I don't recommend you go all the way to Füssen for that alone.
I assume Potsdam will include Sans Soucci Park, a UNESCO World Heritage Site vastly more complex and interesting than N'stein.
To see some real German Castles, look into Wartburg in Eisenach, not far from Leipzig and also on the UNESCO list (which is a very good starting point if you want to identify "landmarks" in Germany.)
UNESCO list for Germany
Notice the Middle Rhine Valley on the above list. There are 40 genuine castles and castle ruins in the 40-mile stretch of river - and a few that you can tour, including Rheinfels and Marksburg:
MARKSBURG
MARKSBURG INFORMATION
RHEINFELS
RHEINFELS INFORMATION
Two other UNESCO sites that are on your way to Munich would be BAMBERG and the Residenz Palace in Würzburg. Both are very worthwhile, IMHO.