My post in this thread: https://community.ricksteves.com/travel-forum/norway/bergen-sognefjord-stave-church-oslo-how-to-details discusses how to do it.
All the official NiN booking does is look after the details of booking the different methods of transportation for you and putting them together in a nice little folder. In other words, this functions more like a travel agent than a tour, per se. There is no tour guide. Now, that can be useful, as it can be daunting to make sure everything lines up, but it is definitely doable and can save money.
I booked most of my tickets in advance and took advantage of the minipris on the Myrdal-Oslo leg. The only ticket I haven't booked in advance is the bus between Bergen and Gudvangen, because it wasn't possible (and supposedly isn't necessary) to do so.
Even if you are going a different route than I booked, most of the website links and phone numbers I gave should apply to your bookings, also.
Start by going to the Fjord Tours site and choosing the Norway in a Nutshell route that appeals to you. Go through the steps of booking it for your dates without completing the booking. You will be able to see an itinerary with times. Use that to figure out what to book and when. (Optionally, if you are not trying to do the route all in one day and have the leisure to choose alternate times, you may be able to use the itinerary to avoid the Nutshell times in hopes of avoiding the crowds, since you will be travelling at a busy time.)