I'm in the Capri camp but only if willing to expend the shoe leather to explore some of the further reaches most day-trippers don't go to. As they said, THAT is the way to lose the worst of the masses that cluster in the town centers, easily reached overlooks and attractions
We stayed a couple of nights so had the advantage of hours the day crowds weren't around but hiking during the day was our escape from those. Some ideas on the Capri website (ignore the touts for tours: you don't need them):
https://www.capri.net/en/traveller-search?todo%5B%5D=nature&all=1
I'll recommend Villa Jovis and Pizzolungo as good ones for a day trip. If you can get up to Anacapri early in the morning, take the chairlift up Mt. Solaro before it gets busy.
https://www.capri.net/en/s/mount-solaro-cetrella
What I'd skip? The Blue Grotto. We bypassed that one due to a lot of negative reviews (time-consuming, expensive, long waits for just a few minutes in the thing, etc.) and didn't regret it a bit.