varenna to st moritz via bernina express

If you go one way the ticket will be one way. If you go and come back you will pay the travel out and then pay for the travel back, usually at the same time. Much like any other bus or train.

This is a regular train route with a train every hour and intermediate stops at the towns and villages along the way. It connects the Swiss and Italian rail networks by going over the Alps.
You can travel the whole route (Tirano to Chur), part of the route (for example as far as St Moritz) and you can stay or come back, totally up to you.
The Bernina Express is an extra tourist train. But you see the same mountains whether you travel on this train or the regular hourly trains.

For more info, and a map of the line, go to the "Rhätische Bahn" website (the railway company running the line), here: https://www.rhb.ch/en/panoramic-trains/bernina-express

Do you mean to go up and back the same day, from Varenna? You could go one way on the named Bernina Express with the special cars, and the other on a regular regional train ( no reservations needed for that).

Here is the schedule for the Bernina Express. Note that until May 11 they are on winter schedule.

https://www.rhb.ch/fileadmin/user_upload/redaktion/Home_Marketing/Panoramazuege/Bernina_Express/Fahrplaene/BEX_Europa-Zentral_DE-EN-IT-FR.pdf

You will need to get to Tirano from Varenna by Italian regional train, a journey taking 1 hour 28 minutes. Check Trenitalia.com for the daily schedule. And buy that ticket online or from the travel agent outside the station in Varenna; tickets are not sold elsewhere in the station.

There are not roundtrip ticket discounts in these areas, if that was part of your question.