Hate to say it, but you would be better off touring Switzerland first. Then you could buy nonrefundable advance purchase tickets at a good discount to Paris and back to Geneva. This does not work very well when arriving at Geneva and going straight to Paris, because if you flight is delayed, the nonrefundable tickets become worthless and you will have to buy expensive full fare tickets to get to Paris.
Edit. I am making some assumptions for my prices, namely that all 4 children are under 12 and the infant is under 4.
Semi-Flex tickets on June 22 from Geneva airport to Paris is 433 EUR for a late afternoon departure. Returning on June 28 to Basel you can get Lyria Non-Flex price of 260 EUR. From Basel activate Swiss pass and proceed to Luzern, then Muerren, then Montreux, then to Geneva airport for the flight home.
Delaying the trip to Paris for after seeing Switzerland results in some savings. Land, activate pass then go directly to Montreux, then to Muerren, then to Luzern, then to Basel where you already have 260 EUR Lyria Non-Flex tickets to Paris. Then have 255 EUR Lyria Non-Flex tickets from Paris to Geneva the evening before your flight home. So that saves about 180 EUR.
I am using EUR because I used the French TGV site for pricing. And if some of the kid's ages are 12 or over, the cost of Lyria Non-Flex tickets goes up by 10 EUR per kid 12 and over. Assumption is that the infant will ride in someone's lap. These prices are based on buying now, as the cheap tickets sell out, only higher priced tickets are left.