Provence town for a week that's not "a base"

Have a look at l'Isle-sur-le-Sorgue - my favorite town in Provence - because it has WATER. Provence is very hot and dry. ISLS is called the Venice of Provence because the Sorgue river divides into an interior delta with many different rivulets flowing through the town. The surrounding countryside is very interesting, particularly the Fontaine de Vaucluse (the source of the river, a few km above ISLS, from where you can tube down the river to the town); and Mont Ventoux (a geographical anomaly, a huge white mountain, with no other mountains nearby, with views forever over Provence).

The one snag with L'isle sur la Sorgue is that, while it is very pretty, it's quite small and does get absolutely rammed with visitors.
I like Arles, and so did van Gogh. So you have art and Roman remains. And a river.
From there, you can do a day trip to Nimes with it's Roman amphitheatre, and also the Pont di Gard, spectacular Roman aquueduct.

A week with only one side trip seems fairly long in a smaller city, so it's obvious we don't travel the same way. However, I have spent about 2 weeks or longer in quite a few cities, and even small capitals tend to get at least a week from me.

I liked all of Aix, Avignon (my base), Arles and St. Remy but wouldn't want a week in any of them. I'd go with Nice, but I am a sight-driven traveler who loves art museums, of which Nice has at least six, with others a short train or bus trip away. I spent a couple of weeks in Nice in the first half of May 2017, taking some French lessons. The city didn't seem overrun to me, but I was staying near the train station (not most people's recommendation); it could have felt very different elsewhere. The mob scenes I experienced were in Menton and St-Paul-de-Vence.

One approach to this type of dilemma is to look for accommodation that appeals and let that guide you. Try https://www.gites-de-france.com/fr. You can search the broad area you're interested in and filter for "city," as well as your other requirements. Gites are traditionally country houses, so make sure the host provides sheets, towels, etc. Here's one I found in Sausset-les-Pins (on the coast) that looks very pleasant: https://www.gites-de-france.com/en/provence-alpes-cote-dazur/bouches-du-rhone/les-lauriers-13g202008?adults=2&children=0&infants=0
You really can't go wrong in the South of France, especially in the springtime. Have a great time!

I was especially drawn in by Arles. I spent 4 nights with one day trip. I could imagine a week with just one daytrip if I were better at just chilling, but for myself, I might worry about not enough to see and do without daytrips for a whole week.

I was really charmed by Uzes.. It is not technically in Provence, but adjacent. I personally would not spend a whole week in Uzes without more that one daytrip, but I did have a long conversation with someone who has spent a whole week there 6 times over the years and plans to keep returning. He and his wife usually rent a car and explore the area but they have also done things like cooking class.

I agree with acraven that Nice has the most museums and things to see in one town or city. I was there in October which may not be a good comparison for April. Nice is a bigger, busier city than any of the other places. you are considering but I did not ever feel it was too hectic from other tourists, just busy with people going about their lives. There are lots of easy daytrips possible without advanced planning if you changed your mind about not doing more than one.

Avignon is often selected for its many daytrip options for those without a car. Given your priorities, I think Avignon has much less to see and do than Nice, and lacks the charm and more laid back vibe of Arles or Uzes. I don't have a strong sense of Aix; I was there for a couple days some 20 years ago.

My assumption is that you don't plan to rent a car, since you primarily plan to spend time in the city where you are staying. Therefore, I should have mentioned that St-Remy doesn't have a train station. It is accessible via bus from Arles an Avignon, but if you for some reason changed your mind and wanted to take multiple daytrips, it would not be particularly convenient. I don't think it's large enough to have local companies offering bus tours of surrounding towns (though I have not researched this).

Don't spend a week in Aix, there's not enough to do.

Spouse and I have spent longer time periods in Arles and St. Remy - I prefer Arles because it's much larger, so more streets, stores, restaurants, to wander around. But we do take day trips, and our longest time period was five days. We've visited Avignon several times, but it's never really "clicked" with us, though I'm not being very helpful because I cannot provide a reason why. I feel the same way about Aix; it's a nice city, but just didn't click.

One thing, we booked hotels for our next trip around a month ago, and many hotels were booked when we were looking for a five-night stay. So, you might find that your decision might be made for you, in terms of hotel availability.

In terms of being crowded with tourists, Saint-Remy will be crowded on market days, but not so much the other days.

Thank you, all, your answers are helping me to clarify a few things.

Now, after reading some of the replies, I'm leaning toward Nice and Aix because, first, the first reply mentioned Provence's aridity, which I hadn't considered, and Aix has trees and fountains while Nice is obviously by the sea. Second, upon reflection, art museums do sound lovely, and it looks like both places have some good ones (though Nice obviously has far more).

I've checked accommodations and it looks like both still have options, though I should decide soon.

I know the two places are very very different, but what does everyone suggest - which city is more enchanting to simply be in? Also, would we better off traveling in the first half of April or the first week of May? Really want to avoid crowds.

And one last question: for those who have been to Nice, which area is the best to stay in? Hopefully somewhere with coffee shops and at least a few calmer spots to sit, talk, read, reflect. Will the historic center will be too chaotic? Beach access isn't very important to us.

I'm very grateful to everyone for sharing their thoughts, it's been so helpful.