You can bring back up to one liter per adult without paying duty. After that it gets a little tricky.
If you bring back more than one liter per adult, technically it's subject to duty. Since figuring the duty is a major chore, however, many customs officers won't charge you anything for relatively small amounts above your duty-free limit.
Be sure to declare whatever you have. If the officer want's to, they can definitely cut you a break - honesty is always the best policy.
On the other hand, if an officer doesn't want to cut you a break, they can make it difficult. The worst possible case is an officer judging your wine to be for resale (they have authority to do so), in which case you have to get an importer's license and pay duty - usually at a much higher cost than the wine is worth. Ports of entry destroy cases and cases of wine that's abandoned rather than claimed because of the cost. This usually happens to people who don't declare their wine (or other item), then deny it again when asked directly.
It's never a crime to bring too much home (though it may result in duties), it's a crime to try to smuggle things in (fail to declare them).
The state you're heading to matters as well. The customs officer has to enforce not only federal rules but also your destination state. Wine producing states are generally more restrictive on bringing in wine.