Setting aside the lodging-availability situation (more later), if you spend the night of September 8 in Flam, you will already have taken the Flamsbana; that's how you cover the last bit of the trip to Flam (from Myrdal). So on September 9 you'd begin the day with the boat from Flam to Gudvangen, then take the bus from Gudvangen to Voss and finally the regular train from Voss to Bergen.
Honestly, I didn't even know there was bus service from Voss to Bergen, but the Nor-Way Bussekspress website shows buses at 4:20 PM and 5:20 PM, so taking a bus would be possible if you prefer it to a train. You'd need to change in Voss.
Lodging in the fjord area is very tight, and it's probably worst in Flam, if only because Flam is such a convenient place to stay without a car. Flam-the-town isn't especially attractive, but the setting is great, albeit often marred by the presence of a monster cruise ship. Unless you book more than six months in advance, it's common in Flam to have to settle for what's available rather than being able to get a room at a place you read about and would prefer. That's just the way it is. Booking.com shows several rooms available in Flam, at various properties. I'd suggest grabbing one immediately--being sure there's a very good cancelation policy--and then thinking about whether you want to do the entire Nutshell route in one day. Such rooms as exist could disappear at any moment. You can also go to the individual hotel's websites to see whether you can beat the rates shown on booking.com.
I spent a couple of nights in Flam (in a private room with bath at the hostel, which was very plain but inexpensive and clean), so I can't tell you how exhausting the Nutshell route would be if done in a single day. I know Rick recommends the very early (6:20 AM? something like that) departure from Oslo in that case, and there's no way I was willing to subject myself to that on vacation if I didn't absolutely have to. The Nutshell transportation schedules fit together very nicely, and I haven't read of any issues along the way, but the Flam-Gudvangen boats don't run frequently, and it's my nature to be cautious about such things.
I didn't want to risk not making it to Bergen the night I was due to arrive. Hotels in Norway are too expensive to pay for a room you're not using.
Based on what I've read, I wouldn't recommend the express boat from Flam to Bergen. It doesn't go into the Naeroyfjord, which is much narrower than the main Sognefjord and therefore more beautiful. Travel literature indicates the latter part of the express-boat trip finds the fjord so wide it's more like traveling on a river.