Day trip to Toledo from Madrid

The historic center of Toledo is a mile or so from the train station. We walked it and rather than walking up the hill we used the escalator. To take the escalator and coming from the train station you’ll pass the Bisagra gate, then the Torreón del Cardenal. The escalator will be on the left (Escalera mecánica del Parque Recaredo). I’m sure there are inexpensive buses to the historic center

How much time you need will depend on what you want to see. To me, a half day isn’t enough. In October we spent a half day just walking around, not going into any sites, and we’ve been there are least six times. When we visit Toledo, generally after a flight to get over jet lag, we overnight there.

If you want to take a cab, my recollection is that it was about 5 euro (about 5 years ago). There is a scenic viewpoint of Toledo on the other side of the river. We later realized we could have had the cab driver take us to that spot to see the view and get "the" picture on the way to our hotel. Note, that Toledo has a lot of pedestrian streets. We walked a fair distance to our hotel from where the cab dropped us off.

EDITED TO ADD:
I didn't pick up on the "half day". No, a half day is likely not enough. We stayed 3 nights and wished we had stay 4. Toledo was amazing and peaceful at night. However, if you want to see the cathedral, you may want to see if you can purchase tickets in advance or get to the ticket office before or soon after it opens in the morning.

There are multiple options to travel from the Toledo train station to city center - as you have identified. The above post explains walking. We had planned to take the frequent bus service to city center, but the taxi rank just outside the station was too inviting. We took a taxi to city center. Again, returning to the station, we passed a taxi rank in city center and were seduced by the ease. In short, multiple options, personal preference.

Half a day would not have been enough for me, personally. Indeed, one full day was not enough - Toledo is on my return list. But, my preferences for visits may not be yours. Will you visit the Cathedral? You might want to check if advance tickets are recommended - I don't know the current situation. Delightful city center to wander and explore!

I didn't visit at night.

Is the city center of Toledo easy to access by walking from the train station or would a bus be better and are there buses to take easily from the station?

In hindsight, I would've just spent the $5 and taken a taxi. There were always a few sitting outside the train station. I would get the earliest train you can, take a taxi to/from the train station, and enjoy the daytrip.

Toledo is one of my favorite cities in the world, and I could spend months there.

I have been to Toledo on a few occasions when I had unlimited time to take the trains and walk around independently, but on our most recent visit in August we went with a day trip tour company, because we had a tighter schedule. This was a luxury bus with an all-day tour guide accompanying us. They organized and arranged everything, included a city walking tour, as well as the cathedral entrance fees.
Lunch was on our own (we ate in an outdoor restaurant and enjoyed a typical dish), and of course, there was shopping time included, so I could browse all the Damasquinado, and purchase a lovely bracelet. The bus also drove around the city so we could get some different vantage points for great photos.

All in all, it was a wonderful experience and definitely worth the fairly insignificant cost. My motto these days is not to do my travel in the cheapest fashion, but to do it in the most enjoyable and productive way.