As a native, I highly recommend family-friendly, bustling (with pedestrian traffic), walkable and lively Old Town Alexandria. Always changing, never dull. It's on the Potomac just a few Metro/subway stops (King St. Station) south of DC. From the station to the Potomac at the end of King St. is about a mile. Tons of eateries, pubs, ice cream & coffee shops, bakeries, pubs, small, locally-owned retail, abounding with 18th century history, Irish and Scottish influences, a couple of cobblestone streets, etc.
The Torpedo Factory on Union St. at the river is a former - well, torpedo factory - turned into a multitude of arts and crafts workshops. A real torpedo greets you at the door. The city also runs a small archaeological museum there. From the dock behind it you can take a ferry to Maryland to see the "Awakening", an unusual in-ground-above-ground sculpture that's graced the area for decades.
Carlyle House, the Apothecary (18th century pharmacy), Gadsby's Tavern (era-themed lunches and dinners served on pewter plates, outdoor seating and the occasional strolling troubadour - reservations a must). In the day, it was an inn frequented by many of the movers and shakers of the time. The visitor's center at the corner of King and Fairfax Sts. has all the details on what to see and do in Old Town.
If you're outdoors-oriented, then rent a bike and ride the paved Mt. Vernon trail (18 miles from Roosevelt Island across from Rosslyn & Georgetown to George & Martha's place. It goes right through Old Town, along Union St.). The plantation is nine miles south of Old Town.
Cycling or walking on the C&O Canal Towpath, starting in Georgetown, is very pleasant. I've been away a few years, but the mule-driven barge once used to haul goods may by now have been restored and made water-worthy for short trips on the canal. It and the park (C&O Canal National Park, that is) are run by the National Park Service. You can also rent a canoe from a concessionaire on the towpath.