Sunday at Kew Gardens

I can't speak to the crowds or the Great Pagoda(which we didn't visit) but if the weather is nice, I would choose the boat trip from Westminster Pier and an afternoon at Kew over a museum, as long as the museum you're missing isnt the British Museum. I think the British Museum is a must-see. I love museums and did not go to Kew until my 7th trip to London.
EDITED: I have not been to either place on a Sunday.

Extremely crowded is exactly what I'd expect the ground floor of the British Museum to be like on a Sunday, though conditions are usually a lot better upstairs.

I don't know about the pagoda at Kew, but it's a very large garden, so I don't see how conditions overall could be awful.

around lunchtime it will be crowded at the feeding areas, the Orangery (when it reopens) and the Family Kitchen especially.

Your second question - yes.