Whether city or countryside, I try to determine my formula for the perfect visit: long enough that you aren't upset to be moving on, but short enough that you leave a little wistful to return.
Some places I got it right (for me): Venice (3 nights), London (9+7 nights over two trips) , Rome (7 nights), Barcelona (4 nights), Vienna (3+4 nights over two trips), Krakow (7 nights), Edinburgh (2+2 nights over two trips).
Some places I got it wrong (for me): Warsaw (9 nights, this decision was out of my control -- would have liked around 5), Prague (4 nights -- would have liked two, maybe three), Arles (5 nights - would have taken one away for somewhere in the Luberon or maybe added to my stay in Riviera), Glasgow (2 nights - would have preferred it in a day trip from Edinburgh), Kirkwall (3 nights -- 4 would have been better since there is so much to see).
Some places I haven't yet been to but expect to spend the better part of a week when I do get there: Madrid & Istanbul.