You have to have a guide/tour for ice caves and glacier hikes.
Weather can be anything this time of year, so be very cautious with driving and keep an eye on the road and weather conditions multiple times each day. The wind can and does blow vehicles off the road. There are basically no shoulders on the roads, nor guardrails, so you really need to pay attention. Do you have winter driving experience? If you don't, Iceland is NOT the place to attempt it.
You may want to reconsider the Blue Lagoon right now as the new volcanic activity is basically right under it. Some tour operators are ceasing their stops there due to the risk. This new volcanic activity is different than the last 3 eruptions and the experts are saying they expect it to be much worse, and of course the risk to infrastructure etc. We hiked the volcano in 2021 and this summer and lucked out both times, but I would be very leery of this new one right now. Even if it erupts any day, it will take time to decide if it is safe to let people hike there, etc.
The northern lights are a bonus if you see them. You can do a tour, or just look up. We had the most amazing show of lights last October right in Reykjavik up at the church. I would not go driving myself to look for them as driving. on unfamiliar roads in the dark and not knowing where I can safely pull over is not something I am comfortable with.
The other things you mention are easy enough to do on your own with a vehicle, weather permitting. Definitely plan to be back in Keflavik or Reykjavik the night before your flight so you don't get stuck on a closed road if weather is bad.
Have you just decided on this trip, or have you done any research?