First, don't dismiss the "standard" things in St. Petersburg. The Russia Museum and the Church on the Spilt Blood are absolutely amazing, and well worth doing even if they are "typical" things for tourists to see. In the Russian Museum, the painting called The Wave by Aivasovsky is absolutely breathtaking, and by itself justifies a visit (the more famous The Ninth Wave by the same artist is impressive too, but not in the same league)
The Hermitage is great too, but also overwhelming. As with other mega-museums like the Louvre, if you can find a way to make multiple shorter visits rather than a single visit, you'll enjoy it more. With three days, you should be able to make this work.
I really enjoyed seeing the Yusupov Palace. I didn't get to the basement (at the time it was a separate tour), but I found the main part of the Palace to be not only beautiful, but surprisingly livable (unlike many which seem too vast and cold to want to live in). The theater in the palace is lovely. Sometimes there are performances in the theater, which would be a great nighttime activity.
My favorite thing that is not part of the standard tourist itinerary, but still right in the center, is the Sheremyetov Palace, now the Museum of Musical Instruments. Again, the house is amazing, particularly the Etruscan Room. And don't miss the videos of the miniature violins being played. As a bonus, the museum was practically empty.