From Apr 1 - Nov 2 , Vaux le Vicomte now has daily shuttle buses running from the Verneuil l’Etang train station to the chateau at a cost of 10 euro roundtrip. To get to this station, take the direct TER train (Line P) in the direction of Provins.
More info here http://www.vaux-le-vicomte.com/en/useful-information/the-useful-informations/access/
If you decide to go to Fontainebleau on the same day, then it's best to take either the Line R from Gare de Lyon or the RER D to Melun, taxi to Vaux le Vicomte, taxi back to the Melun station, and then continue on the Line R to the Fontainebleau-Avon station. From there you can catch bus #1 for the 15 minute ride to the chateau. To to back to Paris, take the Line R. It's a 45 minute ride and you get off at Gare de Lyon. No need to prebook any of these tickets - just buy them at the station on the day you go. If you have a navigo card, these trips are covered (with the exception of the shuttle to the chateau)
This is a really good site to look up travel options and schedules: http://vianavigo.com/en/routes-district-maps/