Yes, the reason to stay in Caen is primarily to have good access to buses traveling east to places like Honfleur, Cabourg and Deauville/Trouville. Caen was obliterated during the war and has little old architecture left. It does have a few historic sights and the very large Peace Museum, but I wouldn't stay there unless I had multiple days of activities planned that could be more efficiently accomplished from Caen.
You do not need to rent a car to get to Bayeux. There's rail service from Caen. If you plan to take a small-group tour of the invasion sites (which I would recommend)--most of which depart from Bayeux--you can spend your second day enjoying the sights in Bayeux: the excellent invasion museum, the cathedral, the Bayeux tapestry, the picturesque historic center. A car would not be necessary if those were your planned sightseeing targets.