Are there sight seeing boat excursions on the Rhine River in February? We'll be in Cologne.
None that I am aware of. But you can sight see by train.
The scenic part of the Rhine is well to the south, south of Koblenz. In winter the green is all brown but it's still dramatic with all the castles in this area. Rheinstein Castle near Bingen:
http://www.michaelleukel.photography/images/gallery/ML-11048-Winter-3_1458122215.jpg
You might catch a train to Bingen, near the end of the scenic part, and stop in one or more of the attractive towns nearby on your way back - Bacharach, Oberwesel, St. Goar, Boppard.
Alternatively, catch a train to Rüdesheim, on the opposite river bank, and return along that side with a stop in Braubach to visit Marksburg Castle, an authentic medieval castle open year-round to the public: