Too cold?
Too cold depends on where in Norway and your adaptability to temperatures. Southern locations near the coast are relatively balmy due to the Gulf Stream effect in comparison to northern inland locations. And "too cold" depends on your adaptability with respect to clothing. https://www.visitnorway.com/plan-your-trip/seasons-climate/winter/
Winters in Norway can be bitterly cold, even if they aren’t always.
How to dress for outdoor activities thus depends on what you are doing
and where you are doing it.
it looks like a lot of things are closed in the winter?
You should check sights and activities for winter hours. For example, Oslo's outdoor folk museum is open but with shorter hours than the summer months: https://norskfolkemuseum.no/en/
Museums like the Holmenkollen Ski Museum are open year round, but again shorter hours during the winter: https://www.skiforeningen.no/en/holmenkollen/skimuseet/apningstider-og-priser/
What is the sunlight like? I REALLY want to see the aurora and I know that's a gamble....
As previously noted, winter sun will be weaker (lower in the horizon) with obviously shorter daylight hours. That said, if you want to see the northern lights, no sun (with clear sky) is better than sun and clouds. https://www.visitnorway.com/plan-your-trip/seasons-climate/winter/
October, February and March are the best months for seeing the
northern lights.
The northern lights are most commonly seen in the north, but may on
rare occasions be seen in all of Norway – even at the country’s
southernmost point.
https://www.visitnorway.com/places-to-go/northern-norway/the-lofoten-islands/northern-lights/
