I've stayed in both Camogli and Rapallo (and day tripped to Santa Margherita Ligure). I don't think staying in any of them is a mistake - they are all nice. I think I liked Camlogi the best. I saw the most amazing sunsets into the ocean there, and it's a charming little town - very quiet in May especially compared to the Cinque Terre. The hotel where I stayed may still be closed for renovation but was perfect - close to the train station, easy walk.
I hiked from both Camogli - "walk" up a bunch of steps first to San Rocco then hiked from there to the abbey at San Fruttuoso. There are two different trails: the scenic route via Batterie, which I took - and was VERY scary in places, without much to hold on to, nothing like the numerous hikes I've done in the Cinque Terre - and a different (apparently less scary but less scenic hike) not quite along the ocean. I hiked alone and it was kind of dumb - didn't realize what I was getting myself into. This is a well-known trail but it was mostly deserted in May that year. I saw a few hikers occasionally, but I was alone for long stretches, and had I fallen or hurt myself in the difficult parts, I would have been in trouble. On the upside, it was glorious not having to hike with so many people around as you would in the Cinque Terre almost anywhere, and I managed the hike alone just fine despite my later misgivings.
From Rapallo you can walk to Santa Margherita Ligure (shrug, a few nice views) and then hike (very easy) on to Portofino. You can hike up above Portofino and from there on to San Fruttuoso or up to San Rocco I guess.
There are also ferry boat connecting a lot of these places. I'm not sure ferries are serving Camogli right now or not, but if hike from there to San Fruttuoso, you can catch the ferry back to Santa Margherita Ligure or maybe Rapallo and train back to Camogli from there. So it would make a fun day at least with a few stops here or there.
Because of the direction it faces, I'm not sure you'd see the sunset at Santa Margherita Ligure, if that matters to you, but it's very pleasant down, just not as quaint as say Camogli.