Avignon is great, and isn't absolutely crammed with people except during its marvelous Avignon Festival in July. Staying inside the old city walls makes for a great experience (I've done extended stays twice) . . . but mind your step, as the locals who take their dogs for walks don't necessarily clean up after them, and "deposits" on the sidewalks and cobblestones are frequent. Access to Orange, Arles, Nimes, etc. is easy. If you're in Avignon, be sure to have dinner at restaurant Numéro 75!
Avignon's a decent-sized city, though, so if you want a more rural setting, there's the island in the Rhone river just west of the city, the Ile de la Barthelasse, but getting around would require taxis, bicycles, or other arrangements to get into the city and its train and bus stations.
Another possibility might be nearby Saint-Rémy-de-Provence, a much smaller town with a great weekly market and a more rural feel. It's just down the road from the accessible asylum where Vincent Van Gogh stayed following his ear incident, and just beyond that are the fabulous Glanum Roman ruins. Bus connections are possible in and out of town but won't be nearly as frequent or convenient as if you stayed in Avignon.