I can vouch for your original ideas, having been to all three places last year. Just don't make the mistake I did on the way to the Kroller-Muller Museum: Traveling from Arnhem, I had to do a bus transfer. When my bus reached the change point, there was another bus right there, so I hopped on it...without verifying its destination. It was going in the opposite direction from the park, so I wasted a good bit of time. It mattered, because the sculpture park outside the museum was larger than I had expected, and I didn't have time to see the whole thing.
I enjoyed all parts of the museum. It does get some tour groups, and they seemed to focus on the Van Gogh section. My advice would be to check out the situation there upon arrival. If it's crowded, try seeing the rest of the museum first, because the crowds in the Van Gogh section may thin out later. Of course, if the weather's iffy, you'll want to think about the best time to see the outdoor sculptures.
I don't mean to sell Leiden (or The Hague) short. Those are pretty cities with worthwhile sights, each easily worth more than a full day to me.
