Should I go East or West?

If you take the most direct route, which means less driving and more sightseeing, you cross

Lüneburg +
Lüneburg Heath +
Celle
Wolfsburg +
Goslar ++
Quedlinburg ++
Wernigerode +
Harz mountains +
Erfurt ++
Weimar ++
Wartburg castle ++
Würzburg +
Thuringian Forest
Schmalkalden
Coburg
Rothenburg ob der Tauber ++
Nördlingen +
Dinkelsbühl +
Augsburg

(+ = stopover, ++ = must see, bold = recommend stay in this area)

This is a "Best of Germany" route, which gives you a pretty complete view of the country. Hanseatic Brick Gothic in Northern Germany, timber framed towns in Central Germany, the Franconian towns of the RR, the biggest car manufacturer (VW), the most important castle (Wartburg, not Neuschwanstein), culture (Goethe, Schiller), history (Luther), and a bit of nature (+ steam trains).

I have seen Martin's bolded towns and--with the caveat that Rothenburg is very touristy--I say "go that way". Quedlinburg may be the prettiest town I've ever seen. Google for the background story on the cathedral treasure. I think the tourist office runs English-language walking tours. It definitely rents English audio guides.

Go west, young man.

Thank you! I actually prefer smaller towns and so this is great. I think it will be more enjoyable for 5 young boys too. any other ideas are welcome!

Can I ask what makes Erfurt so special please? I mean when I was there it's not so rememberable...maybe I haven't explored enough?

I recommend : go east and north. Spend the time visiting cities/towns such as Celle, Weimar Erfurt, Meissen, Halle, Lüneburg, Schwerin, Jena, Magdeburg, Eutin/Holstein, Naumburg an der Saale, Hameln, Minden an der Weser, Dresden, etc...some of my top priorities and gems in this list.

I though Erfurt had beautiful architecture (some interesting stuff that looked early 20th century as well as the medieval buildings) and a really lively street scene because of the students. I enjoyed the side-by-side historic churches and the merchant's bridge. I think there are also Martin Luther connections, but I'm not up on that. Finally, Erfurt works as a base for visiting Eisenach, Weimar and the Wartburg Castle.

Martin Luther lived in Erfurt for ten years from 1501 (age 18) to 1511. From 1501 to 1505 he was a student at Erfurt university studying the
"Septem artes liberales" Grammar, Rhetoric, Logic, Arithmetic, Geometry, Music and Astronomy. These were the usual subjects in the Middle Ages. In 1505 he studied law for only a few months. Then he entered the Augustinian Order as a monk and stayed there till 1511.

Can I ask what makes Erfurt so special please?

Two + were actually a mistake... one is enough. ;-)
But Erfurt is one of the few bigger German cities that wasn't destroyed in WW2. Sure, Erfurt doesn't offer real highlights like a world-class cathedral or a medieval town wall, but the preserved old town is actually one of Germanys biggest, with valuable buildings especially in the side streets (that many people miss) and some really tranquil corners.
And Erfurt, and many other Thuringian towns, still offer a glimpse of "Old Germany", which many foreign tourists sadly miss.... since a Thuringian Bratwurst isn't half as exciting as a Bavarian Pretzel... :D

I'll take a Thüringer over a Bavarian pretzel any day. Good to know that Erfurt came through the war intact.

Rick covered Erfurt in a video on "Hamburg and the Luther Trail" a few years ago. It's free to watch right on this website: https://www.ricksteves.com/watch-read-listen/video/tv-show/germany-hamburg

With 5 boys you may want to check out the VW Autostadt (Auto City) in Wolfsburg. It’s a combination of auto museum, activities - including kids (road course for kids) great cafeteria, and showcases for the various marques owned by VW - Bentley, SEAT. We were taken there by our German friends on our road trip of Germany and even I enjoyed it. He arranged for an English language tour of the site. It was a number of years ago but here is the website: https://www.autostadt.de/start and it’s available in English.