We went to Greece for 2 weeks with our adult aged children and my daughter's husband (21-31) last May. I planned the trip for the most part, taking into account people's varied interests and tolerances. But my older son really wanted to go to Meteora because he is really into photography. So he skipped out of part of the time we had in Athens, rented a car and drove by himself there. He spent the night and drove back, joining us for the last night in Athens (but half of our party had to leave early for work so he missed our official last dinner together).
We, of course, had two weeks not one but think about how to manage a trip with adult children. We also did not mandate that we all do everything together (there were seven of including my husband's brother) but except for my sons going off for a day and a half by themselves in Naxos (where they rented an ATV together), we stayed together. But it was entirely by choice. (BTW, we paid for the things we did together but anything they did separately was on their nickel, which I am sure encouraged family togetherness.) I did consider suggestions when we were planning--my older son really wanted to rent a car on Santorini and after some discussion with my husband, we decided it was better than taking the bus (and it was the best choice since we were in Oia which is not where the buses are headquartered). But this same son was also the one who really wanted us to go to Meteora and I said no--it is another trip, we are going to see Athens this trip. And I told him he could do it on his own if he preferred. And that is what he ended up doing.
We had a wonderful trip and I am sure you will too.