Do you plan to have a rental car in Normandy, or will you be using public transportation? Without a car I'd choose Bayeux. First, it has several worthwhile sights (the tapestry, the cathedral, a good Invasion Museum, the British Cemetery, the historic center). Second, it's the origin point for most of the D-Day tours. Unless one of you is a WWII historian, I really recommend either hiring a guide for at least one day or taking one of the small-group tours. Just driving around in your own car will not be nearly as informative, nor will it be efficient. Beyond that one day with a professional guide, there are places you can navigate to and see by yourselves: the American Cemetery and visitor center, the Airborne Museum in St-Mere-Eglise, the Bayeux sights previously mentioned, etc. You might also want to spend a bit of time at non-D-Day spots. The small coastal town of Honfleur is cute but very popular. The larger town/city of Rouen was beautifully restored/rebuilt after massive wartime destruction. Its historic(-looking) area is gorgeous. Honfleur is accessible from Bayeux by taking a train to Caen and a bus from there. Rouen is linked to Bayeux by rail., changing in Caen.
It's perhaps worth mentioning that Caen is a major port city that was nearly flattened during the war. There's little historic architecture remaining. It does have the very large Peace Museum, which requires a bus or taxi ride from the train station if you don't arrive by car. I liked it a lot, but its scope is broad, covering the lead-up period to the war and the post-war (Cold War) period in addition to the war itself. It's quite possible to spend the full day there, it isn't cheap, and it can be quite crowded. Quite a lot of folks find the Peace Museum overwhelmingly large. Many prefer one (or more) of the smaller D-Day-related museums. For general purposes, the Bayeux Museum may be best. I also found the new museum in Falaise very interesting. It focuses on the civilian experience during the war, including Resistance activities.