We stayed 4 nights in Arles in May. We rented a car 2 days to travel to St. Remy, Les Baux-de-Provence, Pont du Gard, and Saintes-Maries-de-la-Mer. From SMdlM, we walked along the shore trail and viewed lots of wildlife.
I preferred visiting Camargue National Nature Reserve on a bicycle from Arles along the Petite Rhône, where we passed the farms of the famous cows and horses. On a bicycle we were higher than auto passengers and could see the flamingos on the other side of the dike that they couldn't see.
We preferred Arles, because it was more of a town feel. We loved being in the town where van Gogh paintings were staged at the spot they were painted. We found our favorite bakery for breakfast and good restaurants for dinner. You sound like you have more train travel to other sites, so Avignon may suit your needs better. It was a 20-minute train ride between the 2. We spent an overnight in Avignon, so we could take the TVG to Paris in the morning. Avignon definitely has better train connections.