North Americans seem to confuse customs and immigration. They are two very different things.
- Customs is the check on goods. Checking the contents of your luggage to see if it contains any forbidden items or items for which customs duties (taxes) must be paid.
- Immigration (aka passport control) is the check on people. Checking whether you are allowed into the country and if so for how long. This is where your passport is stamped if necessary.
You (or rather your luggage) will not go through customs in Vienna. Your luggage will be labelled through to your end destination . You will not see it at Vienna, and customs will be when you pick up the luggage (Athens or New York).
Customs in European countries is not big issue. Most people are not stopped, they just stop and check a small random sample.
Both Austria and Greece are part of the Schengen Area. You will go through immigration in Vienna, and if necessary have your passport stamped. There will be no passport check after you land at Athens.
On the way back, there will be no passport check at Athens (but the airline will want to see your passport as an ID check). You will go through immigration in Vienna as you are exiting the Schengen Area, and again if necessary have your passport stamped.
You will go through US immigration when you land in New York.
Vienna is a small airport, your times (2h and 1½h) should be enough provided you don't waste time by shopping or going to a restaurant. The airline wouldn't have accepted the booking if it didn't think it was possible. If you miss the connection the airline will be very unhappy as they will have to unload your bags.