Mainz is a nice city but if you're looking for a small old-world-town experience in great scenery you will want to base yourselves instead in St. Goar (or possibly Boppard.)
You "are taking a train from Munich to Mainz on June 6th..." Not clear whether that's a tentative plan or you already have that ticket or a rail pass. You might want to continue past Mainz to Bingen Rhein Stadt by train instead, which is just a short distance further. From there, walk to the KD boat dock and cruise your way to St. Goar (or Boppard) on either the 14:30 or the 16:30 boat. This is the most dramatic way to view the Middle Rhine castles, vineyards and towns for the first time. (If you need a ticket, the price to Bingen is likely to be the same as the price to Mainz.) The cruise fare is €16 each after your 20% discount (show your train ticket at the Bingen kiosk.)
I'd book June 6-8 in St. Goar. It's easy and cheap to get around from there to Braubach, Moselkern (the Burg Eltz train stop,) and Boppard.
Use a VRM mini-group ticket (€22.10) on one day for the train to Moselkern and back. Buy from a ticket machine and use after 9 am; the same ticket will get you to Cochem as well, if you wish.
The same ticket will work for you on the other day for the ferry crossing to St. Goarshausen; the train from St. G'hausen to Braubach; the train from Braubach to Boppard (change in Koblenz); and the final leg back to St. Goar. You can do the reverse order as well if that works for you better.
On June 9: You can visit Bacharach and/or Oberwesel on this day, nice towns 5-10 minutes south of St. Goar. After that, you might visit Rheinfels Castle in St, Goar, or head to Mainz, possibly with a little sightseeing and shopping there (leave bags in station locker.) The Rheinland-Pfalz ticket day pass is €29/2 and will cover St. Goar - Bacharach/O'wesel - St. Goar (for bags) - Mainz (leave St. Goar after 9 am with this ticket.)
From Mainz to FRA takes 25 minutes by direct train. The R-P does not cover this trip but the cost is only about €5 each for a normal RMV ticket. There are plenty of trains.