You'd definitely want to take the train rather than the bus. Or another idea, rent a car.
When we did Norway in a Nutshell, we stayed two nights in Balestrand rather than Flam. We booked NIN on our own, just as you are planning to do, instead of buying a package of tickets. By booking very early, we got a great rate on the Oslo to Myrdal leg. We wanted to break it into two segments to enjoy some time in the fjords. In doing the planning, Flam looked and sounded a little too touristy for us. We wanted to stay in Balestrand.
Our first thought was that we'd take NIN only to Flam, then take an express boat up to Balestrand, then return to Flam 2 days later by express boat and then complete the NIN journey to Bergen. We knew we wanted cars in Balestrand, and you CAN rent them there, and that gave us the idea of driving to Bergen. But the Balestrand car rental place doesn't allow one-way rentals. Our plan, which worked very well, ended up being a departure from Oslo on the early NIN train (6:30 a.m.'ish departure--train was very uncrowded!) and Norway in a Nutshell all the way to Voss. There is a Hertz car rental facility in Voss. When you board the bus from Gudvangen to Voss, tell the drive you would like him to let you off at the bus stop in Skulestandmo. It's just before you get to Voss. The driver had no problem with this request. We were then a 5 minute walk from the rental car place.
We drove up to Balestrand--loved it!!--then after our two nights there, we drove to Bergen. Instead of re-tracing our driving route from two days earlier and seeing the same scenery again, the owner of the Balestrand Hotel helped us choose a different scenic driving route to Bergen that allowed us to get out at the Gaularfjellet viewing platform for magnificent panoramic views of the high mountains and deep valleys. We returned the rental cars on arrival in Bergen. You wouldn't want to have a car in that city.
The one-way car rental fee is steep but we had two couples splitting the cost. If you don't want to pay that fee, you could return the car to Voss and take the train to Bergen from there.
The proprietor of the Voss car rental place is super nice to work with. Email or call him to reserve your car. He speaks perfect English, as does almost everyone in Norway.
We enjoyed the opportunity to have cars for a couple of days on our Norway trip when we were in the fjords. Balestrand was our favorite destination of a 12 night trip. And having cars allowed us to be in control of our schedule on the day we were going to Bergen, rather than having to rely on the departure times of the express boats that go between Flam and Balestrand.