Bakeries and some other food shopping are generally open before lunch on Sundays. In some neighborhoods, this includes an outdoor market, such as the Marche de la Bastille.
This is the Sunday reminder from the intro to Rick's Paris guidebook:
Sunday: Many sights are free on the first Sunday of the month, including the Orsay, Rodin, Cluny, Pompidou, Picasso, Quai Branly, and Delacroix museums. Several sights are free on the first Sunday, but only during winter, including the Louvre and Arc de Triomphe (both Oct-March) and all the sights at Versailles (Nov-March). These free days at popular sights attract hordes of visitors. Versailles is more crowded than usual on Sunday in any season, and the garden’s fountains run (April-Oct).
Look for organ concerts at St. Sulpice and other churches. The American Church often hosts a free concert (generally Sept-June at 17:00--but not every week and not in Dec). Luxembourg Garden has puppet shows today.
Most of Paris’ stores are closed on Sunday, but shoppers will find relief along the Champs-Elysées, at flea markets, and in the Marais neighborhood’s lively Jewish Quarter, where many boutiques are open. Many recommended restaurants in the Rue Cler neighborhood are closed for dinner.