transportation from Charles de Gaulle airport to Versailles

No train goes direct to Versailles from CDG. You would have to take two trains the RER B into Paris and get off at Saint Michel / Notre Dame and then transfer to the RER C train going in the direction of Versailles Rive Gauche chateau de Versailles.
I would rent the car and look up the route from CDG to Versailles on Google maps.

I don't blame you for wanting to bypass Paris traffic, but from the airport you should be able to get onto a main highway, such as the A3 and A4, to circle around the city without going through it. Personally I'd choose that no matter how many extra miles it is.

Another factor is the choice of cars available in Versailles and the hours of opening at the rental center -- both of which may be quite limited. We originally had your same idea (renting out of Versailles), but could not find a suitable car and even when we tried Chartres instead it looked dicey.

We ended up renting out of Gare Montparnasse and looking back with head-slap "What were we thinking?" because for another Euro or two on the taxi meter we could have rented out of Orly and avoided having to drive through Paris streets.

CDG and Versailles are on opposite sides of Paris. Traffic will affect you no matter what route you take. The shortest route would be A1 (from CDG), the périphérique (around Paris to the west), A13 and with minimal traffic it can be done in about 1 hour. An alternate route would be A1 to A86 to A13, taking about the same amount of time. Another longer option would be A1 to A3 to A13.

The big unknown will be the traffic and at what time you depart the airport. If it is raining or a particularly busy morning, the en route times can easily double (or more than double). Car rental desks have boards showing the local traffic and anticipated delays on routes all around the Ile de France. You really would not know the best route option until seeing this real time information.

As has been pointed out, there is no direct train from CDG to Versailles. You could take the RER B to St Michel, change to the RER C for Versailles but work along this route, near Pont de l´Alma, is in progress until August 2019. The en route time will be about 2 hours when this option is available.

There are other rail alternatives but routes will take longer than 2 hours and require several train/bus changes.

You will not go through Paris to get to Versailles from CDG which is not to say that you will avoid traffic because there are often traffic jams the highways in the region.
What is mid-range price for you? And when are you traveling? That could really affect the rates.

For driving around Paris I never use the Périphérique, but prefer the already mentioned A86.

You can follow the A86 clockwise around Paris (eastside) and need just a few km more as doing it counterclockwise (westside). The latter includes approaching Versailles a quite expensive toll tunnel (Duplex A86), costs between €7 to €10 or even €11 overday, depending time and day of the week. You can avoid that leaving the A86 just before the tunnel’s entrance near Rueil-Malmaison, but driving through urbain area this way, it involves from there some good navigation to Versailles as it is not so well signposted. In short is the clockwise drive the least complicated way.

It’s a good hint from Tocard to get informed about the traffic situation before hitting the road.

Thanks everyone