In theory it should be doable, but there may be tight connections that the package folks aren't comfortable with selling you. Or perhaps some of the trains are sold out, making it impossible to complete the trip and get back to Oslo the same day. The fjords are much farther from Oslo than from Bergen, so trying to make a round-trip from Oslo rather than continuing on to Bergen makes it a lot more difficult.
You can explore the train situation here: https://www.vy.no/en. Your departure point would be Oslo S. Flam is listed as Flam Line. The first departure is before 6:30 AM and arrives in Flam at about 12:30 PM. The second departure, after 8 AM, reaches Flam around 3 PM. There are some day-to-day differences in the train schedule, so it's important to research the schedules for the day you intend to travel.
Once in Flam you have a choice (assuming tickets remain available) of just reversing your morning trip and taking the Flamsbana back up to Myrdal and then getting the train to Oslo (over 6 hours), taking buses back (also over 6 hours), or continuing with the Nutshell route (ferry to Gudvangen and then bus to Voss) before getting the train back to Oslo. How long that would take depends on how the various transportation legs link up--i.e., how tight the connections are.
I think the afternoon ferries depart from Flam at 1:45 PM, 3:20 PM and 4 PM. Running time is 2 hours. Tickets for the middle departure are sold by NorwaysBest.com. Tickets for the first and last afternoon departures are sold by the company operating the boats: https://en.lustrabaatane.no/flmgudvangen.
Once in Gudvangen, there's a bus connection up to Voss. When I checked the schedule a few weeks ago, I found the scenic Bus 950 departed Gudvangen at 2:50 PM and 5:45 PM (there are also a couple of morning departures). Those buses depart from down near the ferry dock. There are other buses taking a less-scenic route. They depart at 3:18 PM, 4:18 PM and 4:43 PM. They depart farther from the dock, supposedly about a 7-minute walk away (again, I'd guess uphill). Skyss operates the buses. Travel time is 55 to 75 minutes.
I observed some gaps in the rail schedule as I researched this answer. I suspect they indicate sold-out trains. Just judging from the number of posts about Norway on this forum in the last two months, it appears that a lot of people have decided to travel there this summer.
For the random date I checked, I found trains departing Voss for Bergen at 5 PM (arriving 10:54 PM) and 6:58 PM (arriving 12:54 AM).
It adds up to a very, very long day.