Overnight From Amsterdam

All of the towns you listed are great destinations but Bruges is a true gem. If it's not much of an inconvenience for you, then do an overnighter in Bruges . Antwerp is also a nice city. You will have a great time in whichever city you choose.

Brugge is - depending on the type of train - over 2,5 to 3,5 hrs from Amsterdam. Antwerpen also gives you a change of environment and country and is much closer (1h15 to 2 hrs).
For something quite different and still in the Netherlands, have a look at Zwolle, Deventer or Zutphen, in the east of the country. All very historical cities, hardly visited by foreigners, and a good base for visiting some castles (among which Vorden and Ruurlo).

Yes I would do a Belgian overnighter from there.

For a family, my opinion is that all those places you mentioned are a needed break from Amsterdam. Delft is relaxed and pretty, a good overnight. Haarlem, in my opinion, is a better place to stay than Amsterdam and only 15 minutes away by train. Bruges is a pretty, historic and a well kept town that an be very crowded during the day and a joy at night and morning. We spent the last three weeks in all of them.
Or try Enkhuizen if you have younger kids and do the Zuiderzee outdoor museum and stay in a quaint harbor town for the evening. It is a quick hour+ from Amsterdam by train.

The Zuidersee outdoor museum in Enkhuisen is wonderful. We have been twice: the first time a day trip.. not enough time. The second we found a nice BnB and enjoyed the outdoor museum at a more leisurely pace. There are lots of activities geared to kids....but we are seniors and loved it!!