Although this is Amtrak's most successful and profitable route (largely due to business travelers), it is still quite underwhelming, especially on non-commute days like weekends. For one thing, the constant delays mean unpredictability. Paired with that is the price. So, on one extreme, driving is by far the most expensive option, mostly due to all the tolls (and gas, parking, etc). On the other extreme, are the different long-distance buses that compete with each other and with Amtrak. I take one of them regularly over Amtrak - Bolt Bus - because it's the most cost efficient, even though it takes longer to get there. Amtrak is in the middle. I got tired of paying an Amtrak premium (on the regional line) only to arrive late. The Acela line is priced even worse because it's squarely aimed at business travelers - to me, the time savings are not worth the cost. So I've been mixing and matching one-way trips with Bolt, Megabus, even Vamoose (more expensive then the former two). You'd have to be both lucky and proactive to get one of those $1 one-way tickets right when the schedule gets posted for your travel date (which are really $4.25 or so with all the fees). But it's not difficult to get tickets for somewhere between $13-25 each way to NYC if you are flexible on dates.
The only places I would take Amtrak are: Harpers Ferry (beautiful place but not urban), Baltimore, Wilmington (if you want to see the Brandywine Valley), Princeton, Philly, and New York, and Richmond and Charlottesville. Basically between 2-3.5 hours or less. And I would compare all those points with bus service, and do the tradeoff (is the cheap fare worth taking a bit longer to get there?). Boston and other cities north (Maine, Toronto, Montreal, Quebec, etc) are well served by plane and take a fraction of the time to get there (and the fares could be very good, especially to Boston). It's about 7 hours or so by Amtrak to Boston (if you're lucky and not late), so that's a "no go" for me.