We day-tripped to Esslingen twice and found little to do other than the market. We considered staying in Esslingen but there were few hotel choices.
You missed out on the J. F. Schriber Paper Model museum? For shame! ;)
It's true that most of Esslingen's attractions are just sort of wandering around, but there are a handful of small sites worth seeing, mostly in the form of tours. I will show my hand here and admit that I do tours of Esslingen, but to avoid a forum conflict of interest and just for the sake of honesty I will suggest a tour with Tom Hale, an American who has made his home there for decades and is as passionate about the history of Esslingen as anyone I know. He's also licensed by the city. If you do take a tour with him please tell him I sent you! There are also special tours the city does (which Tom also sometimes runs) that take you into hidden underground places such as the crypt underneath the city church that dates back past 800 - truly remarkable stuff but requires advance booking which you can find through Esslingen's website. If you don't see any tours in English, please contact their TI.
That said, I think an afternoon/evening is sufficient to enjoy the Esslingen market for most people.
Stuttgart had the most choices for hotels and restaurants. We stayed twice at the City Hotel and enjoyed it. The market was large and varied. I hope they still have the Swedish salmon teepee. Stuttgart has good connections to the other towns for day trips. Sarah is right about the area around the old and new palace being very attractive.
The food at the market in Stuttgart is so good that I don't know how much anyone visiting would want to eat at restaurants, but I will share the locals' tip that the best traditional restaurants (or honestly, any good restaurant) is found just outside the city center, either in the city's original "suburb" (from a 14th century perspective) called the Bohnenviertel, which has several great "Weinstuben", traditional Swabian restaurants The slightly further districts of West and Sued also have great traditional restaurants. Feel free to DM me for more info!
BTW the Finnish market (easily confused for Swedish!) is at Karlsplatz at it's a highlight. We've had it every year from 2011-19 and I and many others are really hoping it will be back this year. A lot of people don't even notice it as it's set apart from the rest of the action. The best gluhwein at the market is at the stall in front of the "Alte Kanzlerei" restaurant on Schillerplatz. It's homemade, not from a bottle and you can tell the difference!