Remember, they'll know you're a tourist by your mannerisms and especially once you say something. Being polite is the best way to be viewed in a favorable light. That said...
In early May, the "look" more or less for men in Paris seemed to be nice fitting dark jeans, either dress shoes or white Stan Smith tennis shoes (often without socks, or the kind that don't show), a long sleeve dress shirt, sportscoat, black peacoat or black rain coat, no tie but often a scarf and/or sweater (a black rib-knit sweater with a looser crew collar seems like a good choice). I'd say if you want to look less like a tourist, make sure everything fits well. No baggy jeans or coat.
Last fall wasn't much different in Paris except the tennis shoes were black or dark brown leather. These aren't running shoes, they're leather tennis shoes, specifically Stan Smith or identical style tennis shoes. White was the predominate color in May. Women, at least many women, wore the same but with some trim feature like metallic gold where men normally have green or black trim.
Don't forget the beard. Last fall I was debating whether to shave during my trip or not. When I got to Paris, I'd guess more than 90 percent of men have something between a three day beard and close-trimmed beard. Big beards, clean shaven, mustache alone, beard alone, and/or painstakingly trimmed beards were really rare. Most seemed to let the hair grow on their cheeks and shaved their necks every couple of days.
I think that also works elsewhere. My trip included Burgundy and Alsace and Reims this trip and you wouldn't have looked out of place with the same clothes. The sportscoat might be more urban than you need outside of the bigger cities.
I haven't been south yet, that's for my next trip in September.
sweater (and beard):