It wasn't cheap at $260 for 3 hours, but renting a little 40 HP motorboat at Bellagio was one of the best parts of a memorable 2-week tour of Italy. Easy to pilot, and they really move at full throttle. The mountains are awe inspiring, and there was more than enough room for everyone even around Bellagio. The villas and towns are best seen from the water. You can't stop at ferry docks, but the operators will let you know of scenic restaurants -- we went to one by boat for lunch reservations, and guess who had the table next to us? Green Day!
On the other hand, at Capri your best bet is to get a traditional gozzo boat with the skipper provided. The water is choppier there, and part of the fun of Capri is stopping for a swim or to explore a cave like the Green Grotto. And of course the Blue Grotto -- don't even dream of trying to pilot your own boat there yourself, because it's a madhouse. Finally, you can shoot through the stone arch of Il Faraglione, kind of a must-see.