My favorite area is Ile St Louis (4th arrondissement), the little island next to the bigger island where Nôtre Dame is. You can't get more central and in the heart of Paris, easily walkable to many places you'll want to see. It's charming, has amazing views b/c you're surrounded by the river, and is like a little village unto itself. Great restaurants, cafés, stores, ATM, pharmacy, small grocery store, produce store w/ the best fresh squeezed OJ we've ever had, bakeries, Amorino gelato shop (amazing), and the famous Berthillon ice cream. There's a nice park at the eastern end of the island and another one behind Nôtre Dame just across the pedestrian bridge from ISL. There's also great street entertainers day & night on the pedestrian bridge in the summer months.
There are four hotels on ISL, all very good: Hotel de Lutèce, Hotel des Deux Iles, Hotel St Louis en l'Isle and Hotel Jeu de Paume.