I've visited all three of these cities/towns. I loved them for very different reasons. We visited Olomouc when we were using public transportation between Eastern CZ and Prague. It's a picturesque college town and the castle there was interesting. But, in my opinion, I really loved the southern region of Czech Republic and enjoyed the little villages, the wine country, and touring the castles in this area. I rented a car during this trip and it did make it really easy and our rental car was only about $24 per day.We started out in Brno and stayed two nights with friends and they took us all around Lednice, Mikulov (LOVED this little picturesque town), all around Palava and Klentnice - beautiful coffee shops (Cafe Ferra), churches, countryside and wineries. Then back to Brno and we toured St. Peter's Cathedral and ate at a little restaurant below the cathedral -L' Eau Vive. Walked around the town square, took in a farmer's market and then went and walked around Spilberk Park & Castle grounds.
Znojmo - stayed in Znojmo at a pension with a vineyard on the river (Kaplanka) - beautiful town with scenic views, incredible wines, and cute shops plus the Znojmo Castle and the Rotunda of St. Catherine was very, very interesting & beautiful with it's original frescos. Pro tip - don't park in public areas marked for residents. Got a ticket & boot on my rental car but the Czech police were very nice and the fine wasn't too hefty. We drove to Branisovice and ate at a cute restaurant (U Cisarske Cesty), and drove through picturesque Narodni Park, stopping in little towns and seeing the sites and churches.
So, it wasn't public transport, but I'd recommend Mikulov or Znojmo over Olomouc. Feel free to reach out with any questions!