My preference is 2 smaller towns in the same area per day, so to me this is a bit rushed. I think you may have underestimated the time you'll spend going place to place in the car between the places you listed. Not only is there the driving time, and traffic, but figure on 20+ minutes at each stop just finding parking and walking to where you want to be.
I was in Dinan last summer and you could fill much of a day there if you wander around town, walk the ramparts, go down to the river. If you could add a day to Dinan you could see more of the town, and perhaps use it as a base in the afternoon to explore other places in the area. Dinard is nice (and across the river from St. Malo).
The scenic road to Cancale is nice. There didn't seem to be much in Cancale itself other than oysters and a stroll along the waterfront, so Cancale can take 1 hour to do get oysters at a stand and stroll, or 2+ hours if you sit down to eat.
I would not want to backtrack; staying across from the island to me was just as good as staying on the island since shuttles are frequent. But if you want to stay on the actual island then it is what it is.
After spending over a week in the Brittany/Normandy area last summer I want to go back and feel I just scratched the surface. So if you can add more time to this area I'd suggest that.