Toledo tour?

Why not decide week 2?...once you get a feel for what you like and how things go. I was in the area early Nov 2017. it isn't so crowded or over booked that you can't decide closer to arrival date.I get the tired by end of week 3 thing. I am a solo traveler who pretty much never takes day tours other than a group walking tour, so being lead around from start to finish has appeal. You have alternatives of Viator tours, or checking out a private guide (consider the website Tours By Locals) while on the ground.
I found Toledo to be more physically strenuous than mentally taxing, but I don't read out every bit of information and feel compelled to educate 3 people at the same time. Read up the night before, maybe grab the audioguide for a couple of places, and take the walking tour. Other than the Alhambra, all my entries and tickets were bought either the night before or walk ups at the site. The only thing I didn't get was a time or entry to the Royal Apartments part of the Real Alcazar. I din't lose any sleep over it or feel my exposure to Spain was stunted in anyway of missing it. Instead I went to Casa des las Duenas. But then I am a more visual and visceral tourist. Sure, it is important that I read the history about it for some context, but what I will take away is that something is circa 1492, but more importantly, the vision of pomegranates on the trees that are a little past expiry date (brown and bursting out of their skins), the depth of the carving of the plaster work in the Nasrid Palace....that sort of thing. If you are the literal kind, then maybe my suggestions won't suffice.

Allow me to mention a third option, which I employed in 2016. I assume it still exists. I purchased from the tourist office a plasticized-paper bracelet that allowed entry to 6 or 7 secondary (but beautiful if you love Moorish/Mudejar architecture as I do) sites in Toledo. At most of those a brief, bilingual tour was offered at least twice a day. My visit was in late May or early June, so things might be different for you. RS mentions the bracelet deal (he's not a big fan) in his guide book, or at last he did in the edition I used. These are additional sites beyond the major ones (cathedral, El Greco Museum, Santa Cruz Museum). Among the places covered by the bracelet in 2016 were the El Transito and Santa Maria La Blanca synagogues.

I'm sure you wouldn't have time to get to all the bracelet sites on a day-trip--and the timing of the tours would also make that impossible--but I think even seeing some of them, the added value of the tours would justify the modest cost of the bracelet. Whether you'd have time for any of the bracelet sites at all would depend on how much time you spend at the Cathedral and the two museums I mentioned above.

I'm certain the Cathedral has an audio guide you could use there.

In 2017, I used a card for multiple entries I honestly can't remember whether it was a card or bracelet, but I bought mine at the Monastario de San Juan de los Reyes. I looked out for this because of acraven's experience the year before. I do know that the museums on the card have a small entry fee, and though I think I saw them all over 2 days, the savings was minimal. However, even if I didn't use them all, I just wanted to support the cost. 2Euro either way wasn't going to break me.
I was disappointed to arrive at the El Greco museum to find they were closed 2 days for maintenance. However, given the amount of el Grecos in town and in the various churches in Seville and other Spanish cities, my disappointment was short lived. Of course, there are also many in el Prado in Madrid, but I realized that they are at their best when actually in situ in the location they were meant for....and nothing beats the church of Santo Tome in Toledo for 'the Burial'. Very moving.

I think your idea of the walking tour would be good, not only so your daughter can opt out, but you will have the flexibility to return back to Madrid at the time that you decide when you’re there, I.e. you could stay for dinner or leave early afternoon.

We stayed overnight two nights in Toledo as our first two nights of a 3-week trip in 2017. We didn’t join any tours and enjoyed just exploring Toledo ourselves. The cathedral is gorgeous, and Plaza Zocodover was the busiest with the tourists. (This was late September.). If you happen to decide to see Toledo without any guide, take the silly tourist train. It gives you wonderful scenic photo op views from across the river; sit on the right side of the train.