We stayed 2 nights in Toledo. I thought it was lovely at night. Some areas were busy than others. Honestly, if it were me and I wasn't able to stay overnight, I'd try to get there around 9 and leave in the evening. If you find yourself tired or you have depleted your agenda, to a break for wine or coffee.
BTW. We took a taxi from city center down the hill, and across the river to get that beautiful view you see so often in photos. It cost me about 6 euros is my recollection. I looked into the buses and the taxi was cheaper and then we could control the timing. Or if you take a taxi from the train station, ask to go see the view before you go up into the town.
Toledo is amazing!