Before taking the Seine river cruise

I'm pretty sure that the only stop for Vedettes du Pont Neuf is the embarkattion/debarkation area right by the Pont Neuf. That doesn't give you much flexibility since the only place you mention that's nearby is the cathedral and it closes about six. That's not to say, however, that there aren't a million miles of interesting walks in the area. You could do the Arc since it's pretty much out of the way of everything else and then walk the Champs/Tuileries down toward the center of town. It's a couple or three mile hump which takes most people about an hour by the time they gawk a bit. To do it in thirty minutes you have to press right along. Alternatively, if you don't want to walk or poop out, Metro 1 is under your feet the whole way and you can hop it and get off at Louvre Rivoli and stroll across the northern hunk of Pont Neuf. If you want a bit more flexibility, take a look at Batobus which has maybe eight stops (but no narration) and is a hop on / hop off affair. Tickets are good all day of the date of purchase. You said 'late', however, and it shuts down at either seven or nine-thirty depending on the time of year - - and sunset at the summer solstice is around ten. There's also Bateaux Parisiens that starts right at the Eiffel Tower with the last departure around nine-thirty or ten. All the boats run exactly the same track.

The nearby Left Bank restaurant streets, Notre-Dame, the law courts and the two islands themselves make for great strolling. Across the Pont Neuf (no longer "new"; it's the oldest bridge) to the Right Bank you can easily reach the Louvre, stores on Rivoli, or the shops along the river, including several astonishingly expensive pet stores. Atop the bridge is a fine statue of King Henri IV on his mighty horse. It faces what looks like an alley directly across the road but opens into Place Dauphine, a pocket park with chestnut trees and a good wine bar right at the mouth. So yes, there is plenty to see casually, on foot.
Most posted opinions advise against buying the vedette meals (I don't know myself) and there are a wide range of restaurants if you want to eat before the voyage.

Thanks so much for all suggestions, very helpful

My favorite walk is around Ile St. Louis (little island east of Notre Dame), Ile de la Cite (where Notre Dame is), then along the Seine river to the Louvre - cross the bridge and walk back to Notre Dame. Absolutely beautiful and enchanting walk at night. At night, in front of Notre Dame and on the pedestrian bridge leading to Ile St. Louis are street performers... very enjoyable to watch. Sitting outside at a cafe (La Flore en l'Ile) at the western tip of Ile St. Louis at night is one of my favorite things to do.

Hope you're not going tommorrow. The Seine is REALLY high right now.

We will be there in July