4 minutes is the standard connection there. I wouldn't necessarily suggest zooming away from Augsburg that morning, however. You might be overlooking some very interesting, less-touristy places.
Augsburg, another Romantic Road town, is pretty interesting itself. And its Christmas market will already be underway when you arrive.
https://augsburgerchristkindlesmarkt.com/der-christkindlesmarkt/?lang=en
Noerdlingen, another RR town surrounded by an old town wall like Rothenburg's, opens its Christmas market on the 28th and can be visited easily on a day trip by train from Augsburg. Noerdlingen isn't just a tourist-town like Rothenburg; it's populated by locals who aren't just innkeepers and service personel - people from all walks of life live and work there, just one of the reasons I find it an interesting place.
https://www.noerdlingen.de/fileadmin/noerdlingen/prospekte/pdf/Ortsprospekt_Englisch-Franzoesisch_2019.pdf
https://www.noerdlingen.de/tourismus/veranstaltungen/weihnachtsmarkt/
Donauwoerth is another small RR town worth visiting, only about 25 train minutes from Augsburg.
https://www.deutsche-donau.com/data/thumbs/img/4987519_donauwoerth-titelbild-luftbild.jpg
So you might do well using Augsburg as a base town for a couple nights before moving on. The Hotel Ost am Kö was a good choice for my visit there last year.