Amsterdam 6-hour Layover

You will arrive in the non-Schengen concourse and pass through Schengen immigration between the two sections of the concourse. Go through immigration before you store your carry-ons, since when you return, you will be flying from the Schengen section of the terminal. You go through customs (luggage inspection) in Vienna, assuming you will be on one ticket with luggage checked through to Vienna. In fact, they generally do not inspect luggage and you just walk out the "Nothing to Declare" door.

6 hours is the bare minimum of time to go into Amsterdam on the train and get back. Head back to the airport 2 hours before flight time from Centraal station. Since you will have a boarding pass, you just have to get through security and walk to your gate.

Are you landing in AMS from the US?
Forget Customs, you need to go through Immigration/ Passport Control. I was there last year and the lines were quite long, about an hour to get through. Later , for Customs, walk through door marked as Nothing to Declare.

To do this, you will need to clear Schengen entry passport control in Amsterdam (which you would have had to do anyway for your connection to Vienna, even if you had stayed in the airport, so it is not an extra step) when you land and head to the train. When you take off again for Vienna, there is no passport control, because the Netherlands and Austria are both in the Schengen area; it's effectively like flying domestically within the US or Canada.

It looks like there is baggage storage at Schiphol located "landside," i.e. after you clear passport control and exit into the public area of the airport.

There may also be lockers elsewhere, but in your itinerary you should not use any lockers in the gate area and you should clear passport control with all your carry-on bags. The issue is that if you are arriving into the non-Schengen area, but you are departing from the Schengen area.

These two areas are separated because non-Schengen flights require clearing passport control (in both directions), while Schengen flights do not. If you leave your bag in an airside non-Schengen locker, you will run into problems getting that bag when you depart because you won't be able to return to the non-Schengen area without a non-Schengen air ticket.

Much obliged - this was just the information I was thinking about - especially grateful for the Customs vs. Immigration explanation. I plan to play it by ear, and if the timetable just looks too risky, I'll stay in the airport. Am I right that if I stay in the airport, immigration isn't necessary?

Buy your train tickets in advance at the NS site to save time standing in a queue at the ticket window or puzzling at an unfamiliar ticket machine. A return ticket costs EUR 9 and is valid on all trains from Schiphol to Amsterdam Centraal.