My two cents - if you ARE going to see the tower try to do it early in the morning. I was happy to see it at 7AM after a night train from Paris. I was alone, the light was beautiful. As I was walking away from the grounds, tour bus after tour bus filed in. At the same time an endless row of souvenir stands began opening up selling Leaning Tower plates, towels, egg cups etc.
I'm not convinced it's worth the special trip when you could spend a little extra time in Tuscany. Montapulciano, for instance, is so close to Montalcino.
We drove out of Florence, btw, on our was to Tuscany a couple of years ago. We left early and our rental agency was a bit of a hike from the city center. That turned out to be a good thing because it meant it was an easy drive out of town.