Next month, we plan to take the train from Nice to Arles about 6pm and Avignon to Nice at a similar time, both on weekdays. Should we book online before we go or just arrive at the train station 30 minutes before and buy our tickets there?
Get tix and reservations as far in advance as possible. Some of the more popular train times get booked up and you may get stuck going at a time you don't want. I would not advise you to wait til the last minute. When you arrive in France, if it is a few days before you want to take the train, should be a good time to buy tix and get reservations. If you plan to take the train the same day you arrive in Europe (France), IMHO it would be best to book online. I've been stuck several times not getting the train I wanted, due to holidays or trying to travel at very popular times or waiting til the last minute to get my reservation. I would think that weekdays would be less crowded than other times, but that's only a guess.
Just buy your tickets at the station. If you want to avoid standing in line twice, buy your Avignon-Nice tickets at the same time you buy your Nice-Arles tickets.
No reservations are possible on regional trains and it's a regional train that leaves Nice Ville at 17:50 and goes directly to Arles.