I would book a Champagne cellar tour ahead. Lots of them fill up and it is not necessarily easy to just walk in. We also found the tourist office to have hideous lines for the small amount of information they had to offer. We do our homework and rarely use the tourist offices.
There are trams and buses easily used in Reims if you don't want to walk to the Champagne houses, but the walk is manageable. The Cathedral is not far from the train station and the major Champagne houses are about that far again. Don't miss Basilica St. Remy which is near the Taittinger House -- wonderful. There is also a WWII museum as well as the stunning Cathedral. You can get champagne everywhere.
If you book the TGV early, you can get very cheap tickets for the 45 minute ride to or from Paris.