The Solutre Rock https://en.wikipedia.org/wiki/Rock_of_Solutré is a well-known walk to do. There is a museum of prehistory halfway up, and an ancient breed of horses in fields next to the path. From what I’ve read here there is now a safety fence at the top.
The prehistoric caves of D’Arcy-sur-Cure are close by with a 25,000 year-old hand print of a five year-old. https://en.wikipedia.org/wiki/Caves_of_Arcy-sur-Cure
The Basilica in Vezelay kept my kids interested with the carvings, but there’s also space outside In the side yard to run around and play and a picnic area down the path behind.
The town of Noyers-sur-Serein is still walled and has a lot of half-timbered buildings.
Guedelon is your best stop (40 minutes off the autoroute), a medieval fortified castle being constructed using medieval methods, tools (except for two safety additions). Everyone is dressed in period dress, all crafts and trades are open for you to visit, for example: stone cutting, wool dying, farming, etc all with medieval tools and methods.
Two other chateaux in the Yonne region are luxury castles, not crenellated fortifications: Ancy-le-Franc and Tanlay.