Only you can answer the following questions, which will determine whether you should take the camera.
How much of your trip will be spent worrying about the camera? If that's going to be a preoccupation, leave it at home.
How much of your trip will be spent regretting that you didn't take the camera? If you are looking for high quality photos, will you be upset (during the trip or later) that you only had a lower quality camera (whether a point-and-shoot, or the one on your phone?)
Of course, it's perfectly possible for both of these to be big issues. In that case, I'd prioritize the trip itself over the pictures. In other words, if worry about the camera's whereabouts is going to ruin your trip, leave it at home, even if you look at your pictures later and regret the lower quality.
The posts above are also correct. A "pickpocket" tries to work as unobtrusively as possible, and wants cash. So you don't have to worry about that, but other kinds of theft. (Most hostels have lockers - be sure to use them). And never bring something on a trip that would be devastating for you to lose.