We used Provence Pana for a day tour when we were staying in Avignon in 2012. I think anything that could go wrong did - but I'd give them another chance if we were visiting again without a car..
We did the one that went to Pont du Gard/Uzes and Arles. Well, the driver was about 15 min late and was a 'new' guy. The usual driver was sick. We had about 45 min at Pont, then about an hour to wander around Uzes on our own (which went very fast - he had some parking issues) then when we got to Arles, we arrived about 15 min too late for last entry at the arena - so we missed out on that. The entry times changed at the first of Oct and he didn;t realize they would be closing earlier. (You'd think that would be something they'd keep track of) I know we just had bad luck, but if I was to do another day trip with them, I'd look at a longer/full day one, or one that concentrated on only a few spots.
I'm not sure about the smaller towns - you'd be relying on buses mainly. Using rome2rio.com (a great site!) the bus from Avignon to Roussillon is 2 hrs. Then you're relying on bus schedules...you could look into taxis perhaps, but even then... I know when we were on the Amalfi Coast, we were able to hire a private driver for a day - I wonder if something like that is available in Provence so you could tailor it to what you wanted to see/how much time you wanted to spend in a certain spot. It probably wouldn't be cheap, but for 4 people, could be comparable to what you'd pay for 4 people with Provence Pana. (I just googled private Provence tours and looks to be some options - maybe someone will have a suggestion for you).
As for lavender - it'll most likely depend on the weather - I imagine if it holds out hot and dry, you may luck into them. We're returning the first week of July (before we went in early Oct) hoping to see the lavender, but we're going to have a car since we're there for 8 nights. And there could be fields in areas that bloom early - there is probably a website that tracks where it is blooming.