I think it's reasonable to want to not stand out looking like a stereotype of an American tourist. I think many Americans know how we're viewed abroad and want to avoid that as a first impression, even though when we open our mouths it's obvious that we're visitors/tourists/foreign.
I will also disagree with the common consensus and say that there ARE differences between how the average American (or Australian, or Canadian, etc) looks and dresses compared to Europeans (and there are differences between countries, too!) This is what makes playing "Guess the country" at a popular square in a touristed city so much fun, after all.
BUT the differences are so much more nuanced than "Europeans don't wear shorts" (a lie) and stuff like that. A lot depends on age, social class, etc. If I see a guy in his 30s with a baseball cap that's curved/bent wearing a graphic tee, cargo pants, Oakleys, and Tevas, that guy is probably an American, so I could say, "Baseball caps and sports jerseys give you away as an American." But if I see a man in his early 20s wearing a flat-brimmed baseball cap, a long sports tee, and shorts, he's probably German. A lot of the basic elements are the same, it's the way they're combined and the age of the person that gives it away.
It also depends on what the activity is. Germans dress a certain way for hiking that's not dissimilar to how Americans might dress, but they're likely to be wearing Jack Wolfskin branded clothing that sets them apart even in Europe.
So there's no one "way" to fit in when traveling in Europe. For Americans over 30, I'd recommend dressing a little bit less casually than you might at home - think nicer sandals instead of flip-flops, maybe - especially in the evening, as Europeans seem to differentiate a little bit more between daytime and nighttime casual especially in sit-down restaurants . My husband was going to wear a t-shirt to dinner recently - I suggested something else. When we arrived at the restaurant, sure enough, there wasn't a t-shirt in sight, but polo shirts. Still casual, but a little bit less so than my husband's instinct.
For people under 30, it matters less, as youth fashion is incredibly globalized now. What's trendy in the US is trendy in Europe, at least for casual dress.
And despite this there are always exceptions. During the last heat wave I saw a young woman wearing a t-shirt and bikini bottoms in the center of downtown Stuttgart. THAT was unusual, but the rules do go out a bit when the heat gets intense.