Paris

Are you renting a car ? Most small "villages " will not have trains going to them .

Here are some:
https://janettravels.wordpress.com/category/day-trips-from-paris/
Senlis might be a particularly good choice and we thought Moret sur Loing was lovely. Avoid any of these small tourist driven towns on Mondays as they are pretty much shut down.

There are dozens of lovely little towns near Paris and many are best reached by car or bicycle. https://www.frenchmystiquetours.com puts together small private bike tours and could take you to places you probably wouldn't discover on your own. It is run by an American ex pat who is married to a French woman and has lived and worked in France for many years.

Provins is a Medieval City and a good bet - but it's in the North East.

Provins is south east, and Senlis is north east.
Lovely blog post of Provins here.
http://www.coolstuffinparis.com/provins-france-medieval-village-day-trip.php