I agree that your itinerary looks fine. I think it's better to have too many days than too little in Portree, and there's a whole lot to see and do on that island. I was there for four nights and found plenty.
You might like to make a day trip to the Isle of Raasay, which is a 10-minute ferry ride from Skye (out of Sconser). It's a charming small island and is usually very secluded, which makes it a great place to get away from the crowds you will find on Skye. It has a lovely little country store there, a distillery where you can sample some wonderful whiskey, and a very nice inn where you can get lunch or a snack in the afternoon. The scenery is lovely and you can get in some light hiking if you like really enjoyed visiting there for the day. If you do go there, make sure you read about Calum's Road, and if possible, come and visit it. It's really an incredible story.
Don't worry about driving on the left. After the first few minutes, it will seem as natural as eating pancakes. It is a little bit more difficult driving down the single-track roads, but if you just apply common courtesy and check the etiquette rules before you go, you'll be fine. Here is a video that can get you started. Driving on Single Track Roads. Keep in mind that while there are plenty of single-track roads on Skye, there are also plenty of roads with two lanes, including the road from Armadale up to Portree.
I really enjoyed Glasgow and would certainly suggest a visit there if you have the time. You could go there on a day trip from Edinburgh if you have the time. That said, I would also leave room to make a day trip to Stirling Castle and the town of Stirling, which is pretty wonderful. I loved my visit there, and it's such an incredible castle with a beautiful view of the countryside. The town of Stirling is also very charming, with a lot of history there.
And your idea of heading to Glasgow and returning the car there would also work. That way you wouldn't have to take any time away from Edinburgh, which is always a plus. Inverness is nice, but most people really just use it as a base to see a lot of the Highlands. You might have to pay a small extra fee for dropping it off at a different location, but it shouldn't be much. I did that when I was there a few years ago, and I think it was only around £25.
One thing I did want to mention is that while you are on Portree, make sure you make reservations at restaurants for both dinner and lunch. Lunch you probably only have to make the reservations a couple of days ahead of time, but if there are any particular restaurants you really want to eat at, make sure you make reservations a couple of weeks ahead of time.
For what it's worth, it would help to know what time of year you were going. I'm assuming it's late spring or summer since you're anxious about the time frame. Don't worry about finding a place in Glasgow if you decide to stay there. Both of the cities should have a decent amount of hotel rooms available. It's the islands and the Highlands that will be booked up.