It is difficult to express to Americans just how good the German rail system is, including for short day trips. The point is that your options from any location are more numerous than you may imagine.
I personally consider Weimar to be a richer location for daytrips without a car than Frankfurt. I mean specifically that a lovely half-timbered town from Frankfurt is not as full a destination as, say, Erfurt from Weimar. It's much harder to do two towns in one day by train, but it is possible. (And a plus for Frankfurt is the extensive S-Bahn system.)
Note that your trip is in shoulder season. I'd check if seasonal locations like EGAPark in Erfurt or Saalburg (near Frankfurt) are open - I simply don't know. When do the middle-Rhine dayboats ( https://www.k-d.com/en/ ) start running? We were interested in the tracks of Bach and Martin Luther in Weimar. There are some Bauhaus buildings there. If you like art museums, Leipzig might be a better choice, where you also have the tracks of Goethe, Bach, and the fall of the DDR to follow. I was sorry to miss the Zeiss museum in Jena. And I failed to uncover the superb Schloss in Gotha before our trip to Weimar.
We stayed in a slick modern Weimar business hotel, Dorint Am Goethepark, because we had a car and it had parking. Doesn't apply to you. You give a good reason to stay in Weimar, but Heidelberg suggests a closer small city, like, say, Aschaffenberg for a two night stay. In your case, I would give Weimar or Leipzig at least three nights.
