Both yes and no because it is more the landscape around that places :-)
Tromsø (the "ø" is at the end) is an Arctic town - means small and very functional but a town. I would suggest to additionally spend a look from Fjellheisen when you are there and drink a coffee at Kaffebonna.
Svolvær is a little village - even the main village on Lofoten islands - also very functional as well as fishing and tourist oriented. Oil and gas upcoming. The attraction on Lofoten is the nature and landscape you can see when driving down the scenic route Lofoten to Å i Lofoten.
At both places you can enjoy midnight sun as long as you have free view to Northern horizon (if it is shining).
3 full days are not much time for the journey to and from Lofoten.
1) Rent a car and explore area north of trom Tromsø (driving up the E6)
2) Rent a car and explore island of Senja instead of far away Lofoten which is much closer. Summer ferry Botnhamn - Brensholmen will shorten your way. No need to drive via Finnsnes.
3) An idea could be to enjoy Tromsø on arrival day and to take the souuthbound Hurtigruten ship to Svolvær (sailing plan, swith direction to southbound). it drives through a very nice landscape and in late afternoon of following day into very narrow Trollfjord. You can try to get a cabin to get a bed and a bath or you try to sleep in the public rooms, e.g. panorama deck.
Have a good journey.