I've spent 24 months bouncing around Europe since 2015, traveling solely by train and bus. I've seen a total of one person smoking on a train or bus. I think it happened in Bulgaria in 2015. I didn't hit every country in Europe on those trips, but I got to 19 of them. I'm pretty sure smoking is forbidden on trains throughout Europe, and it seems we're past the point of people flouting the rules. I'll be curious to see what the more knowledgeable train folks have to say about the "fumar" markings you've seen online. Maybe the photos were really old?
Edited to add: I should note that I've had olfactory evidence of people smoking inside hotels on occasion, which I think is a violation of the law/regulations. I remember encountering it in Italy in 2015 and in Spain in 2016; I'm not sure it has happened since then. In both cases it seemed to be employees who were smoking.
There definitely is considerable variation in the incidence of smoking from country to country. You'll see a lot of people smoking in France (including retail employees standing in or just outside doorways); I observed far fewer in Norway, Sweden and Finland. This is something I tend to notice, because exposure to smoke in a closed-in area will give me a headache in as little as 5 or 10 minutes. I'm happy to report that it's not a problem I worry about these days except when considering whether to eat outdoors (often the designated smoking area) or indoors.