The Bodleian Library is another must-see (for me) but needs to be booked far in advance.
Not only that, but there are very limited tickets available, and the odds of getting something last minute are slim to none.
I would vote with just walking around and looking at the gorgeous architecture and the colleges. I don't even know that I would go inside. I just don't think you'll have enough time for that. I loved the Ashmolean Museum, but you really don't have time for there unless you exclude the rest of Oxford and just focus on the museum. I do think it is worth it, but obviously you want to see Oxford and I don't blame you. It's really a lovely city and has such a great atmosphere.
I would definitely check out the parking in advance. I know there are some car parks on the outskirts of town that you can park at, but I wouldn't try and take a car into the center.