I think your plan is solid and it sounds like you know what you want to do and see.
In general I would agree with Adam about not wanting to park but AFAIK there are no left luggage/consignee facilities in Avignon TGV or Center so you have bags to contend with. That would be both getting everyone and their bags into Avignon Center and then you still have them with you. Since you're not staying in Avignon and can't drop them at a hotel I think the TGV rental is probably your best bet.
So picking up a rental car at the TGV is easy - it's directly outside the station - gives you a way into Avignon and a place to put your bags. Parking may be a bit of a hassle and keep in mind that all your bags will be in the car so choose where you park with some care. Since your rental is so close have you considered going directly there and unpacking and then maybe back into Avignon for dinner or evening exploration? Sunset isn't until 21:30/9:30 or so you can plan for a longer day.
If you want an easy place to eat with everyone you might check out Place des Corps Saints where there are tables in the center square from about nine different restaurants that surround the area. Much more lowkey and local than the scary tourist area restaurant row that you get below the palace.
I'd plan on an hour from Pont du Gard to Les Baux and probably the same from Les Baux back to Avignon area. Last castle entry is 19:00/7:00 in July but you can stay up to an hour after that. The Atelier des Lumières is pretty cool if you think everyone might enjoy the experience. Since you haven't mentioned St Remy I doubt your big van Gogh fans but it can still be worth seeing for the effect. It is the same parking as the castle and access to both can come with a combo ticket. If you want to do both I'd plan on getting to Les Baux no later than 4:30 to see the show and then make your way through town to the castle entrance before 19:00/7:00.
Also keep in mind driving into and parking on market day - especially in smaller towns - can be a hassle since everyone has the same idea. Take a sense of humor and just watch what the locals do. In St. Remy parking on sidewalks and islands on market day seemed de rigueur but I'm not recommending that.
But sanity checkwise I think your plan is solid, have a great trip!
=Tod