What you are proposing is exactly like this:
We will be in southern California, and we are interested in visiting Mexico, a large, diverse and fascinating country right next door. We will walk across the border into Tijuana and spend a few hours there. If people approach me, how do I know who is a good one and who is a bad one? How late should we stay?
Do you think your 3-hour visit to Tijuana would give you much insight into Mexico, and a good experience?
Tangier is the Tijuana of Morocco. Where you will find a crass, fake, grotesque, manufactured-for-tourists experience that only bears the slightest resemblance to actually visiting the wonderful country of Morocco. You are guaranteed to experience the absolute worst possible that the country could show you.
Morocco is a complex, fascinating, beautiful and yes, different (some would say "exotic") country, filled with great things for the visitor. It's as worthwhile a destination as any country in Europe. It's also inexpensive, safe, and (dare I say it) quite easy to visit - tens of thousands of people (mostly Europeans) flock there every year. There are crazy-cheap flights to points acrooss Morocco from all over Europe - dirt cheap. Flying there is quick, cheap, easy, and makes sooooo much more sense than wasting a whole day of your trip to Europe on the ferry crossings (which may or may not go, due to conditions) - but most of all, the ferry takes you to the dregs of the country, while the flights (which may actually work out to cost less than the ferry) takes you to the stuff you really want to see (which is far away from the awful day-trip, ferry port/bordertown "Tijuana" places).
Morocco is great, and is worth a few days. Do yourself a big favor. Skip the miserable day-trip/ferry experience. Fly to Marrakech or Fes, and spend a couple days there, get out and explore deeper into the country if you have time.