The reason you are getting suggestions like Bingen, Bacharach, and St. Goar is that these are small wine towns that are separated by only a few miles and situated in one of Europe's most scenic river valleys... AND because they are quite close to FRA airport AND on the route to Amsterdam. It's a place where Germans, the Belgians and the Dutch often go to relax, enjoy the outdoors, and try the "mighty fine wine" from some of Germany's most popular wine regions. The entire stretch of river between Bingen and Koblenz is a scenic playground dotted with medieval castles, half-timbered old-world buildings, vineyards, biking/hiking opportunities and more. So it's kind of a complete no-brainer based on your interests.
Area map
Scenery near St Goar
Cruise boat in Oberwesel w/ Castle-Hotel Schönburg
Marksburg Castle in Braubach
Old-world streets and buildings in Braubach
Weingarten in Boppard
Aerial view of Boppard with cliffside vineyards
Gedeonseck terrace with view, near the top of the chairlift in Boppard
A car is not needed to visit. Probably not a good idea for wine-sippers anyway. Wineries are in town. Trains, ferry crossings, and cruise boats have you covered here on both riverbanks, and you can leave the area easily for Amsterdam from any train station:
http://www.loreley-info.com/eng/rhein-rhine/walking-hiking.php