I'm traveling in Czechia right now and don't gave access to my notes from last year's trip. I take long, slow trips, trying to find good day-trip destinations so I can avoid 1- and 2-night stops.
I think I was in Reims for 3 nights, but I didn't get to my hotel until about 5 PM on arrival day. That gave me more time than I needed since I didn't make any side-trips and had no interest in the champagne houses. I went to the Surrender Museum (perfectly OK, but not as interesting to me as other WWII museums I've seen), the cathedral two times (fabulous glass), the Foujita Chapel (attractive but certainly not a must), the Musee des Beaux-Arts (not one of my favorites), and the stained-glass museum (probably not of much interest to the typical tourist). I walked around the city a lot; it's frankly not as visually arresting as many other places in France.
I liked Rouen more from the standpoint of surface attractiveness. Lots of restored (or totally rebuilt) half-timbered buildings. I think Rouen was just a two-night stop. Again, I took no side-trips. I remember feeling somewhat rushed. There were three museums, all of which I enjoyed, but one of which I think was a temporary exhibition, then I went to the cathedral and walked pretty much the entire large historic area. I didn't go inside the Jeanne d'Arc church.