Both are very pretty towns. Each with a completely different feel.
Annecy has a beautiful old center, plus more to explore (ideally by bike in nice weather) around the lake.
Chamonix is small with an Alpine village feel. Chamonix's sights are the gondolas up to Mt. Blanc then another over to Italy. It's pricey (about 100 euro when I went) but, for me, was a memorable one-time splurge. There is also an ice cave and train ride in the immediate area, that were nice if you had the time in the area.
I ended up getting a rental car out of Lyon because the two aren't convenient, to each other, by train (it was possible, but took hours as opposed to about an hour by car). Using public transport out of Lyon to one or the other is fine.
If I had to choose one. I'd look at both and decide based on which interests you the most.