I would like to just chime in and offer thoughts on our experiences.
We travel with my now 15-year-old son who has mobility challenges. He walks, but slowly, and is not well-coordinated and tires easily. So, a few stairs are OK, but a long, uphill climb is too much, and uneven surfaces are very challenging. We travel with a special needs stroller for him to cover longer distances.
I will say that traveling with him has probably dramatically improved our overall experiences. We are forced to slow down and enjoy where we are, not just what we might have been racing to see. There is a lot of stopping to smell the roses, so to speak, and they smell pretty good.
We do need to plan more carefully than we might otherwise (although that's not a problem -- I am a planner!). So, scoping out accessibility, terrain, distances, transportation, etc., etc., is important so that we know we can get from point A to point B and still enjoy the journey. Lodgings for us always must have an elevator, and because we tend to spend more time in our lodgings than many vacationers (because we need rest breaks), I prioritize location, comfort, views, and outdoor spaces like balconies and terraces. We take taxis where others might choose to walk, and we use a lot of public transportation, which, in our experience, is nearly always very easy to navigate for people who are mobility challenged.
I will also add that people are so very, very helpful and kind. We have had countless offers of help along the way and accepted them gratefully when needed.
More specifically to address your original question regarding destinations: our toughest place so far has been Venice. All those old walkways and bridges are tough to navigate if your legs don't listen to your brain very well. However, I would go back, with my son, in a heartbeat. We learned very quickly to plan our routes to minimize the ups and downs and when in doubt, to grab a vaporetto. (at least, I learned this quickly.... my husband maybe not so much. He had an epic "journey" with my son one morning on his own that wound up with several people helping to hoist the stroller over the Accademia Bridge. I would have taken the vaporetto :-) )
All of this is to mostly say.... keep on traveling, even when you may have to do it a little differently than you're doing now. Your experiences will still be rich and memorable, and slowing down can be a blessing.