Another one here who really liked Madrid (and not just for the museums, but the urban vibe). As for Granada vs. Seville, two big determiners:
1) Can you get Alhambra tickets for your dates? Apparently they are selling out quite quickly now, so depending on when you are traveling, you may have missed them. I would not go to Granada if you can't see the Alhambra.
2) Logistics. Seville not only has a fast train from Madrid, but it has a larger airport with more options. Granada takes longer to get to because they're working on the fast train, so you take a train as far as Antequera and then a dedicated bus from there to Granada. The bus is run by the train company, coordinated with the trains, and goes right from the train stations in each city (rather than the bus stations), so it's not hard, but it does take longer and require a transfer. Granada's airport is small with limited flight options. All of these factors mean that Seville is logistically easier than Granada. However, with proper planning, both can work fine (for instance, fly US to Barcelona, see Barcelona, fly to Granada, bus/train combo to Madrid, fly from Madrid back to US).
And finally, it's not budget that should make you pick destinations in Spain, but time. Barcelona is definitely more expensive than Madrid, Seville, or Granada, but none are Norway or Switzerland. Don't skip Granada or Seville to save money; if they both interest you, see both and cut time in Madrid or Barcelona!