Gamla Stan can be done in a 1/2 day stroll. Keep your camera at the ready the whole time.
Djurgarden area will definitely take an entire day, longer if you want to linger in the folk museum. Both the Nordic Museum and the Vasa Museum are great and it's a great area to stroll.
One evening as it gets dark, stroll out to Skeppsholmen island with a snack and a drink and plant yourself on the western edge, looking across the water toward Gamla Stan. All lit up at night with the reflection off the water - it will be magical...
So I woud say yes, you have enough time to see most of the highlights in 4 days. If the rest of your plan includes the night Silja/Viking ship to Helsinki, that offers a wonderful view of the archipelago for the first 2-3 hours. Then you go to sleep & wake up in Helsinki. It's an amazingly worthwhile trip, even if you come right back the next night.