I'm not sure I can add anything useful, but I will try.
First off, the Tate Modern has both "Modern Art" and "Contemporary Art." There is a difference, and it's valuable to understand. Modern Art is generally understood to encompass works from approximately 1860 to 1970. There are a lot of -isms that belong under the umbrella term "Modern," such as Impressionism, Fauvism, Cubism, Surrealism, and Abstract Expressionism.
Contemporary Art is art of our time. It is understood to include work that pushes boundaries, for example beyond traditional media. Performance art, video, immersive installations, and a banana taped to the wall with duct tape are some examples. In addition, contemporary art often tackles social, political, environmental, and sexual issues.
The Tate Modern, as I said, has both. So you may find much to love in the Tate Modern. I was there last week, and I didn't linger with things I didn't particularly like, but I liked a lot of the art there and did linger for a while. (I had tickets to Shakespeare's Globe, which is practically next door, so it was a convenient stop for me.)
There are a lot of great art museums in London (ORDTraveler has a very good starting list). Don't push yourself to visit all of them. If I were going to pick one, without a doubt it would be the National Gallery.
I'll just add one other thing: I visited the V&A last week when it was in the mid 80s in London (before the heat wave). I stayed less than a half hour, because it was unbearably hot in the museum. I actually felt ill. As I was leaving I kept being drawn in to rooms and I wish I could have spent more time there.