There are Sunday markets throughout Paris, so where are you going afterwards? It might make sense to stay close to the train station from where you'll leave, if you take a train.
Also, do note that the vast majority of markets start to wind down by 1-1.30pm.
The first area that comes to mind is the part of town between Bastille and Gare de Lyon. Marché d'Aligre is in the area, and that's really one of the most popular markets in Paris, with both a street market and an indoor market. It is also very convenient if you happen to have to take a train at Gare de Lyon afterwards!
There is also a large market under the metro near La Motte Piquet in the 15th arrondissement, which is not very central but close to the Eiffel Tower and to Montparnasse station.
Also of note, a market of organic produce on Boulevard Raspail in the 6th.
The list is far from exhaustive! Have a look at the city's website using a translation tool as needed.