We stayed at and can recommend the Ardenlee Guest House in New Town, walking distance to the train station and Royal Mile.
http://www.ardenlee.co.uk/rates.html
Rate includes a Scottish breakfast
January - March : £30 - £40 per person (excluding New Year & rugby
internationals) April - June : £40 - £50 per person July - August: £50
- £60 per person September - December : £30 - £40 per person (excluding New Year & rugby internationals)
New Town is pretty gentile and good for sleeping. It is down hill from Prince Street but only about a km to the Royal Mile. Interesting that "New Town" in Edinburgh is older than most North American communities, built between 1767 and 1850.