The classic Norway in a Nutshell route is the classic for a reason. You can make changes, but they will either add time (and you don't have a lot) or omit one of the scenic segments in the classic Nutshell. If you plan to start your trip in Oslo, the logical thing to do is take the classic Nutshell route between Oslo and Bergen. If you prefer, you can run it in reverse, but it seemed to me when I was planning my own trip last year that the connections looked just a bit better in the Oslo-to-Bergen direction.
Regular train from Oslo to Myrdal (this train goes all the way to Bergen, but you get off before Bergen).
Flamsbana train from Myrdal to Flam.
You can and should buy a ticket from Oslo all the way to Flam from the Norwegian rail web site, www.vy.no . It will cover your trips on the regular train to Myrdal and the Flamsbana from Myrdal to Flam. The ticket may be cheaper if bought earlier, but pay attention to rules about refunds and exchanges. Enter accurate information about your ages to see whether you qualify for a senior discount on the regular-train segment. Note that the trains can sell out, though I don't know how early that would be likely to happen.
Naeroyfjord ferry from Flam to Gudvangen. You can buy that ticket from www.norwaysbest.com, which I think is the same company selling Nutshell packages at a mark-up.
Bus from Gudvangen to Voss. As recently as last June, you didn't need to buy the bus ticket in advance; you just paid the driver.
Regular train from Voss to Bergen. This ticket you buy from www.vy.no .
You can of course pay more and have someone else put the package together for you, but buying the tickets from home is easy enough, and it saves money.
The Nutshell route makes for a long day. Some people like to break the trip and spend a night in fjord country. If you have only one night to spare, you really need to choose one of the towns I've named, or perhaps Aurland would work (I don't know about bus service). Balestrand is by all accounts a more attractive town than Flam, but it's on the other side of the Sognefjord, and there's just one ferry a day between Flam and Balestrand. The schedule is such that if you stay in Balestrand for just one night, I think you'll have to take the express boat straight from Balestrand to Bergen, missing the more scenic Naeroyfjord ferry and the nice bus ride from Gudvangen to Voss.
Unfortunately, the demand for lodging far exceeds the supply in Flam, and it may be too late to find a room for your travel dates. If you don't want to do the Nutshell in one day, check booking.com for a room immediately. Make a cancellable reservation at any remotely acceptable lodging while you continue to make plans. You can of course book directly with the hotel rather than through booking.com if you prefer. (It might be a bit cheaper, and there might be more rooms available that way.) The reviews on booking.com are usually pretty reliable.
If you have no luck on booking.com, go to Google Maps and zoom way in on each of the towns you'll be passing through. You may find hotels or rentals not available through booking.com. Contact them directly to see whether they have space for you.
After completing the Nutshell route, you can spend some time in Bergen.
For your onward travel, there are flights from Bergen not only to Oslo and other Norwegian cities but also to Stockholm and Copenhagen and a few other cities in Denmark and Sweden. Alternatively, if you have more time, you can hop on one of the overnight coastal ferries to Alesund (12 hours, or 24 hours with side trip into the scenic Geirangerfjord) or Trondheim (36 hours including the Geirangerfjord). Cabins on the coastal ferries can sell out. From Trondheim you could return to Oslo by air or rail; there may be flights to Denmark and Sweden as well; I haven't checked. From Alesund you'd need to fly out or take buses or a bus/train combo.