The law (art. 6 c.3 of the Uniform Immigration Law) requires it.
The alien who, at the request of the public security officers and agents, fails to comply, without justified reason, with the order to show the passport or other identification document and residence permit or other document certifying the regular presence in the territory of the State is punished with arrest up to one year and with a fine of up to € 2,000.
Some people prefer to leave it in the hotel safe and take their chance rather than to take the risk of losing it or having it stolen by a pickpocket, but that is a personal choice. There are circumstances when you will need identification, besides when requested by police (which is very rare, unless you look like a middle eastern or African immigrant, who get stopped and questioned by their immigration status by police all the time for any reason). In any case, if you prefer to take that chance and leave it in the hotel safe, carry a photocopy of the passport and be sure you are never too far from your hotel. So if you stay in Florence, I wouldn't go on a day trip to Siena without a passport.
Also keep your passport secure at all times in your body and separate from your wallet. It is true that thieves are not interested in stealing your passport, but if the passport is in the same purse where the wallet is and the purse is snatched by a thief, the passport goes with it. Replacing the passport at the US consulate is probably easy and relatively fast, but not every city has a US Consulate. The US Consulate of Florence covers a large territory in Italy, including Emilia Romagna. So if you are in Montepulciano and you lose your passport, you need to travel all the way to Florence. If you lose it in Umbria, you need to go all the way to Rome. That's a hassle that is going to eat away precious vacation time if it happens.