Istria for 5 Days - Worth It for Slow Paced Travelers?

My advice would be two days in Pula and three in Rovinj. They're both beautiful and you can't go wrong either way!

I would also divide my time since you won’t have a car.
2 nights Pula and 3 Rovinj looks good, as mentioned above.

A couple of side-trip options along the coast for you: Vrsar (tiny) and Porec (colorful). Bus service available from Rovinj; I'd guess also from Pula but I don't know for sure.

When I was in Istria back in 2015, there was a bus tour to inland Istria running on Wednesdays. I caught the bus in Rovinj (not in the center of town), but we were joined by groups from other origins, which I imagine included Pula and Porec. The tour made brief stops at several places, including Groznjan, Motovun and a museum in Pazin. There was also a wine-tasting at a vineyard.

I'm always dependent on public transportation, so it's a real treat for me to be able to see several small places in one day; it's hard to do that solely by bus.

Do yourself a favor and don't spend nights in Pula. It is not really a pretty place compared other places you can stay in Istria. By far, Rovinj is the most beautiful and I would plan on spending every night there. Rovinj and Pula are really close together - only 35 minutes apart by bus, so it really doesn't make sense to me to split up your stay. Rovinj is more central.

The question is should I stay in Pula for the entire time or Rovinj?
Or divide my time in half?

That depends on what you want to see in Istria.

One highlight is a hike to Cape Kamenjak and the Safari Bar. That would mean taking the bus from Pula to Premantura. From there it's about 1-hour walk or you can hire bikes in Premantura.
If you want to do that, two nights in Pula would be just right.

I was in Pula 2 years ago at the end of May. We stayed in a tourist hotel on the Verudela peninsula. From there you can reach the Pula amphitheater in 15 minutes by local bus. The water was still cold but a short swim was possible and watching the dolphins swim past on the cliff at sunset was magical.

Pula is a city and industrial port with 50,000 inhabitants. Yes, Rovinj and Porec are much nicer. But it's an interesting city with a long history, with the amphitheater, the castle with underground passages, the area around the forum with the Temple of Augustus and the illuminated shipyard cranes at night. I liked it very much.