Public transport to Hadrian's Wall

During the last week of September I shall be leaving Stratford-on-Avon to spend 2 days exploring Hadrian's Wall. Is Carlisle the best place to stay to access the more interesting parts of the wall using public transportation? Or, should I spend one night at a Carlisle hotel and then take the train (have RailPass) to Newcastle for a night to see the more eastern parts? I see that there are nice, smaller towns between these two larger cities but do not see any reference to public transport to the wall. I do not want to drive.

Thank you.