So after a month of Googling, emailing, calling and posting, I FINALLY have an answer for this. There is no pre-arranged transportation back to Paris from either Reims or Le Havre that I have found. I received a response from SCNF (the train company) that said there are no extended hours for trains back to Paris. I am EXTREMELY disappointed in France and FIFA for this oversight btw. Even in my podunk town (Sacramento), I can find a party bus to go to any game in the Bay Area (2 hours away). But apparently not in France, even though people keep telling me that the country's transit system is sooooo extensive. Apparently not when there are 9 PM matches (and wtf? 9 PM matches? Who's bright idea was that?).
Anyhow, my solution for the 7 of us going to these matches is that we rented a van in Paris and we will be driving to and from Reims and Le Havre on match days. It's ridiculous and I'm not happy about it, but it was the cheapest option by far (about $150 each match day for a 7 person van). There are private transport companies but they were quite expensive.
I wish I could offer you a seat to the match, but our van is full unfortunately. Good luck and let me know what you end up doing. It's probably cheapest to get an inexpensive room in Reims and take and early train back to Paris. That was our last resort, but we didn't want to do it because we are going on a side trip to Scotland after the match and we didn't want to rush home to Paris in the AM.