The train from Paris to Provence stops in Lyon and Avignon, either one of which would easily fill a few days. We took out children to Europe when they were young - and yours are still quite young - and found that they enjoyed the simpler things (generally not museums) and that, if they were happy, we were, too. In our experience, one activity per day was generally enough with little ones - this is the time to stop and smell the flowers, have a crepe and slow down.
In Lyon -
take the funicular uo the hill and visit the lovely park at the top, wander the Roman ruins (an amazing theatre where the kids can run around), take a cruise on the river, visit pastry shops and decide which has the best tarte or other desert of choice (this can be done anyplace and was one of our favorite things to do), take the double decker bus tour and hop on and off whenever something strikes your fancy, go the Musee Miniature and Cimena (the kids will be fascinated by the miniatures though they probably have never heard of any of the movies), head to the Parc de la Tête d’Or , one of the largest parks in France, with lake, mini-train and small zoo, visit the Musee Confluence (the 8 year old might find it of interest) as I seem to remember dinosaur exhibits and ,lots of inter-active things for kids. Outside Lyon (you could take an Uber), there is a very nice Aquarium with fish from all over France and see a short kids movie,
In Avignon - for some reason, I'm having a harder time here, perhaps because we enjoyed Lyon so much -
The Palace of the Popes is the big draw though I doubt it would be best with little ones - I remember going up and down stairs and lots of crowds -
Let me leave Avignon to another reader - my vote is clearly Lyon.
Wherever you go - have a wonderful time!
