You do not say if you are arriving and departing from Frankfurt on the indicted dates, or if you will be in Frankfurt for those dates for some event(s). If you want to just see Germany/Low Countries/Luxemberg /eastern France it can all be done easily by train. The RS Germany guidebook is best used an an outline of travel details and some hotels, but he doesn't go far off of the paths his tours take. Don't limit yourself to the main stops though.
https://www.bahn.com/en/view/index.shtml
There are many travel offers too that can get you around as a family at a discount on regional train routes (1-5 people) that are good all day, and many are good for both days on a weekend.
https://www.bahn.com/en/view/offers/regional/index.shtml?dbkanal_007=L04_S02_D002_KIN0060_ST-REGIONALE-ANGEBOTE_LZ01
Just be careful that you pick the proper spelling of the place names you want, and note that some places have more than one train station (guidebooks like RS will tell you the right one).
It will be cold, and probably wet and the days are short - not much competition from other tourists through - EXCEPT around Christmas markets.... Christmas Markets begin around December 1 and usually end by the 24th, so you can do those early if you want, and do some other traveling later. But, of course the Dec 23-Jan 4 will be school holidays. Note also that almost all shops in small- town Germany except tourist hotels/restaurants are closed on Sundays and after 8 pm which is a change for Americans.
Every town has a Christmas Market, including Frankfurt, but Nuremberg seems to be the one that all the guidebooks lead people to... You will be able to sample big ones like Nuremberg and smaller but really nice ones like Augsburg, Bamberg, etc.
Nuremberg has excellent museums too, including a toy museum that I particularly liked.
https://museums.nuernberg.de/nuremberg-municipal-museums/
The Marksburg castle on the Rhine is easily accessible by train from Frankfurt, and is an excellent intact example.
https://www.marksburg.de/en/visitor-information/
Bavaria of course has lots of castles and palaces to tour:
https://www.schloesser.bayern.de/englisch/palace/index.htm
If you are not tied down to Frankfurt doing a Rhine Loop - Braubach (Marksburg) - Koblenz-Cologne and return and/or a loop east and south Wurzburg, Bamberg, Nuremberg, Augsburg, Strasbourg, and back to Frankfurt are both easy enough.
You will have a great time. There are non-Alcoholic Gluhweins/Punches for the kids at the Christmas markets while you can go for the Feuerzangenbowle.
https://en.wikipedia.org/wiki/Feuerzangenbowle