Can you clarify what you mean by "the south of France?" There is train service running near the coast all the way from Italy to Spain, so I imagine you're interested in a different part of the south.
I spent a lot of time in southern France last year, traveling exclusively by train and bus. (Well, I took a ship to Porquerolles and a boat excursion to view the Calanques.) This is a list of my overnight stops, followed by side-trips I made. To the extent that I remember, I've marked the destinations to which I used buses for at least part of the trip.
You might like to take one or two bus tours if you plan to visit Provence. There are some places that are tough to reach by public transportation: Roussillon, Gordes, Les Baux, etc.
- Nice (2 weeks of language lessons): Biot/bus, Villefranche-sur-Mer, Entrevaux, Digne, Beaulieu-sur-Mer, Antibes, St.-Paul-de-Vence/bus, Vence/bus, Menton, Vallauris/bus
- Hyeres (2): Porquerolles/bus & boat
- Marseille (5): Cassis, Calanques/excursion boat
- Avignon (7): Arles, L'Isle-sur-la-Sorgue, Villeneuve-les-Avignon/bus, St. Remy/bus, Fontaine-de-Vaucluse/bus
- Aigues Mortes (1)
- Montpellier (4): Sete, Narbonne, Pezenas/bus
- Perpignan (4): Collioure/bus, Banyuls/bus, Ceret, Mont-Louis/bus. (You can also connect to the SNCF Yellow Train running all the way to Latour de Carol--a beautiful trip through the Pyrenees.)
- Toulouse (4): Albi
- Lyon (5)
And then I moved north. However, you can certainly travel through places like Pau to the Basque Country.