One night (24 hours) layover in mid-November (family of 4)

I would wait to rent a car and see what the weather holds for you.

If you want to do a northern lights tour, I highly recommend Aurora Experts. He is excellent, and is not a "jack of all tours" as he only does northern lights. He has a small Mercedes van, so the groups are only about 12 or so people, and he does not follow the masses. He is just great overall. Note, only you know your kids and would know if they would be excited for this or not. So something to think about. If the lights are strong, you can see them right in Reykjavik itself.

As for a thermal lagoon etc, the hard part is the 8 yr old. Due to the age limit of Sky, which is right in Reykjavik, I would opt for a local pool instead. I personally like Sundhollin ass the rooftop hot tubs are great. There are others though, but I have only been to this one.

If you have a rental car the next day and weather is decent, head to the Golden Circle. Either Secret Lagoon or Fontana would be great, and the kids ages are fine. Secret has pool noodles, which the kids might enjoy.

Now if the weather is not great and you decide not to drive, you could opt for a private tour as the others are too long for you. You will need to leave Reykjavik by 1pm at the latest for a 5pm flight to the US.

With just 24 hours, the Golden Circle route is a good idea. You get a sample of Iceland’s geological features - waterfalls, a geyser, a volcanic crater, glaciers, etc. A stop at the Efstidalur dairy farm might be a highlight for the kids - and the parents, too. They make outstanding ice cream, and while you’re enjoying your treats inside, you can view the cows munching hay through a glass window to the barn adjacent to the ice cream parlor/resraurant.

The Golden Circle also includes Thingvallir, a national park where the North American and European tectonic plates meet - you can be on two continents at once. The excellent Fontana baths are nearby - soaking pools of different temperatures, plus saunas and a restaurant.

To possibly make things easy, Icelandair offers several tours. One that includes thermal baths and possible Northern Lights sighting is their “Warm Nights and Cool Lights” tour, and it’s available in November. We did it in April a few years ago, and they picked us up at our hotel in Reykjavik, took us to Fontana for dinner and a warm (hot) soak plus dinner. It was overcast that night, so no Northern Lights were viewable, and they took us back to Reykjavik early, but it was a worthwhile experience, and could be a nice evening if you did’t get to Fontana on your own during the Golden Circle drive. There’s also a Golden Circle tour offered, if you decided not to do the driving. Details: https://www.icelandair.com/vacations/tours/northern-lights-and-fontana-baths/