My take is that it really depends on your personal comfort and the country you are in. If I were in a third world country, or one that I found intimidating for some reason then I would carry my passport everywhere. In the EU countries, here are my personal guidelines:
-if I'm going on a day trip outside the city where I'm staying, I take my passport with me
-if I'm in transit and don't have a hotel, I take my passport with me
-If I'm staying at a hotel without a safe in the room, I take my passport with me
-If I'm staying at a hotel with an in-room safe, I test the safe to make sure it works then use it for my passport.
Bottom line is that it can be stolen anywhere, but I feel good about the safes in the hotels I've been in. Yes, they can be cracked, but it's less likely there's a thief in my room than when I'm out and about at tourist destinations. As far as being stopped...I have a copy on me, as well as my US ID and I think that if I'm in the same city I'll be able to go to the hotel to retrieve it. This seems very unlikely to be an issue to me. But opinions vary wildly on this topic, so do what you feel is best.
I will note that I keep it in a money belt when it's on me. But I hate moneybelts and only wear them when necessary. I often need to carry 2 passports (mine and my child's), and that makes the moneybelt huge and conspicuous, too.