RailEurope is rather notorious for not showing all options. It sometimes seems to prefer to sell the more expensive tickets, but perhaps there is a different explanation (I'm trying to be kind here).
When traveling between regional cities in France you do frequently have to go through Paris; it happened to me three times this summer, even when both origin and destination were on the same side of Paris. When you have to change train stations in the big city, it is nail-biting time even when the first train is on schedule. At least in your case Paris is more or less directly in between your starting and ending points.
Looking at the 9:08 AM departure, I see that the Paris transfer (Auber to St.-Lazare) is by walking, so the two stations must be very close together. You wouldn't have to fool with the Metro to make that shift. On the other hand, you'd have a total of three changes to make, and in my recent experience it's not uncommon for French trains to be 10 minutes or more late. In one case I arrived in Paris only to be told that my connecting trains wasn't going to run at all that day. This is likely to be the most costly alternative (I haven't checked SNCF.com to be sure).
The 9:49 AM departure has a 9-minute transfer in Epernay (nervous-making) but nearly 3 hours for the cross-town transfer in Paris, so you should even have time to find a place for some sort of sit-down lunch near St-Lazare. If you can stomach that 9-minute initial transfer. The trip as a whole is very long, of course.
The 11:10 AM departure also takes much longer than the first option (it's all regional trains) but the shortest transfer time is 18 minutes and you don't have to go through Paris at all. And it will probably be the cheapest since it involves no fast trains.
Honestly, I don't know what I'd do in your shoes. I'd probably opt for the first departure, which has the virtue of being the fastest and getting you started the earliest, so if something does go wrong you have some recovery time.
Or I'd change my itinerary to avoid the whole problem. Are you also planning to see Epernay? If you stayed there instead of in Reims, you'd avoid the first transfer on the second departure, eliminating some risk. Unfortunately, you'd still have a trip lasting over 6 hours.