Oh my gosh yes, change things around as that festival is crazy! It starts on Thursday, and we arrived the evening before. By the time we left on Saturday morning, the amount of people walking off that ferry seemed to never end.
If you go 2 days before the start, you will have a nice time and the crowds will not have arrived just yet. I would start in Selfoss and return to Reykjavik. The reason is that it would allow you to get on an earlier ferry with being closer to it, and that would give you a bit more time.
If you want a really cool dinner place for the way back, check out Ingolfsskali Viking Restuarant in Selfoss. So good!!! Or if you are cutting through the GC by chance, Mika is great. You need reservations at either one.
The Reykjanes Peninsula is nice, and we have wandered around on it on several trips, but it just doesn't have quite the punch that other areas have. It is good if you want a very low key slow paced day.
Between Sky and Hvammsvik, Hvammsvik hands down. It is our favorite commercial hot spring in all of our trips. I tried so hard to fit it in for our trip next month, and just couldn't, so we are headed to Sky. Sky is nice (been to both multiple times), but there is just something special about Hvammsvik.