Istria or Split?

For option 1, investigate the cost of renting a car in Austria and returning it in Slovenia. If it is allowed, there is typically a very high fee to drop off in another country.

You will probably find better flight options from Zagreb than from either Split or Ljubljana. I’d investigate what your flight options home are to see if one of your two options works better.

We'll be in the Austrian Alps and Vienna

One other thought …
Could you start in Vienna and then go to the Austrian Alps? If so, it is a 4 hour train ride from Salzburg or a 2 hour train ride from Villach to Ljubljana. You could pick up your rental car in Ljubljana and continue on to the Istrian peninsula; make sure the rentalcompany knows you will be driving in Croatia

We can't switch directions, I’ve got a work thing to schedule around in Austria. But good point about the car rental - maybe we just train from Vienna to Ljubljana or nearby and rent a car there, if we go to Istria? Or skip the car rental and go to Split?

I think the Istria trip with car would be more efficient and mesh more with your stated likes on past trips. There are parts of the Istrian coast that reminded me a bit of cinque terre (this is a loose comparison, but there are similarities in appearance, not in the sense that there are villages you can hike between).
Check flights to Rijeka, Zagreb, Pula.

Depends on the time of the year, the length of the trip and if you guys 3. 4 or 5 star travelers.

We are 3-star travelers - budget-mindful but also just easy going and don't need much in order to have a great time. We're going end of July (due to overlap with work in Vienna). We'll have about five days for the Croatia leg of our trip.

Thanks to suggestions above, I'm now thinking that if we go to Istria we'll train to Ljubljana and rent a car there, so as to avoid the 500 euro fee for returning a car in another country.

Also, so far flights home from Ljubljana are about $400 less each than Split (and less than Zagreb and Pula). $1200 is consequential, but I don't want plane tickets prices to be the deciding factor in Split vs Istria.

Oh nice, with the airfare that makes even more sense. I’ll go ahead and plug Piran and Rovinj. Rovinj is pretty resorty, but superb beaches and swimming. Enjoy Piran while it lasts, I feel like this town could blow up alongside Slovenia’s skyrocketing popularity.

It’s also possible to visit some islands while you are in Istria, as depending on where you stay, you are not far from the ferry port to Cres/Losinj, or the bridge to Krk.
Airfare is as good a factor as any because the north is just as beautiful as central Croatia, I can assure you!