Rick says getting to Cadaques "is very tough without a car." Do you agree? Is it worth the trouble getting there by bus?
We went to Cadaques 9 years ago on a RS tour that no longer exists. My recollection is that the tour bus traveled a very curvy road over quite hilly terrain. There is not much there other than the small village and the Dali house, so one would have to be very determined to do it at all, car or bus. If the bus has difficult connections or is very expensive it'd probably only be worth it if you seriously want a quiet village on the Costa Brava.
Good luck.