Places in western Provence that are easy to get to without a car include Marseille, Cassis Aix-en-Provence, Arles, Avignon, Uzes, Nimes (just over border of Provence now, but historically part of it), Pont de Gard, Orange, and a few others. However, most rural destinations are difficult or impossible without a car. So, when you say you want to see "countryside," you will probably need a day tour for this (there are companies that take you, and Rick's book lists taxi companies in the area you can hire for a day or half-day).
Of the three A's (Arles, Aix-en-Provence, and Avignon), my favorite was Arles; however, more destinations are available by public transit from Avignon. I stayed 3 nights in each and took day trips from both (saw Nimes from Arles, and Orange from Avignon). For me, Aix loses on both counts: I didn't care for it (although many like it), and it's on the eastern edge of part of Provence most people want see, making transit (particularly without a car) difficult. While Nimes is also on the edge of the area of interest, it was my favorite of the places I saw in the region.
Note that when I say "Rick's book," I mean Rick Steves Provence And The French Riviera. His general France book has MUCH less information about this region.
I really liked Monaco. In addition to the aquarium and the Jardin Exotique mentioned by Rick, my favorites were the Princess Grace Rose Garden and the Prince's car collection (they are emphatic that it's not a "museum"; by any name I really enjoyed it). The Riviera is the opposite of western Provence (note that technically it's part of Provence too); almost everything is easy to access from Nice by train or bus, although some destinations require more planning than others. From Nice to Monaco, there are 2 trains an hour and 4 buses an hour, so no advance planning required. If you don't like Monaco, it's easy enough to go to another town, or back to Nice. However, from Nice to La Turbie (with its great views down on Monaco), there are only 4 buses a day Monday to Saturday, and no buses on Sunday, so it's easy to get stranded it you're not careful. Again, Rick's book has all these details.
