Croatia uses tolls on their highways. Paying the tolls is easy - cash or even credit card worked for me.
Instead of tolls, Slovenia uses a vignette sticker (driving pass). If you rent a car in Croatia, you MUST buy a vignette driving sticker at the border or risk a steep fine if caught without one!
You may not realize it until you look at a map, but driving from Dubrovnik north to Split and toward Zagreb, you briefly drive through a small section of Bosnia at Neum (look at a Google map). So you cross borders into Bosnia then back into Croatia (passports checked) not long after, and technically you are leaving the EU. This probably affects the cost of the car, too, because you are driving the car out of the EU (yes, barely). Check with the car company to make sure this is allowed. Some companies make exceptions for the Neum Corridor or charge you only a little more. You'd think the car company would assume anyone driving north from Dubrovnik would cross out of Croatia, but it's best to check and confirm this border crossing is OK and not be surprised when you pick up the car. (You could avoid leaving Croatia by taking a ferry instead of driving through Bosnia.)
Be careful crossing the border between Croatia and Slovenia that you use the larger border crossings and not take obscure border crossing perhaps following your GPS or your phone mapping, because some smaller border crossings allow EU residents only to cross.
Having a smart phone for GPS is a great idea. You don't even technically need a cell phone connection (data) to use a smart phone as a GPS - Google Maps for example has an "offline" mode where you can download the maps ahead of time and use your phone as a GPS that way. Try it, it works! SIM cards in Slovenia and Croatia aren't expensive, though, if you have an unlocked phone, and you can use a SIM from one country in another (EU roaming fees have mostly been eliminated.)
Look at the total cost of car rental from Dubrovnik to Ljubljana when comparing cars, including the drop fee. Have you priced a car with Sixt? (Check their website.) You would save some money (on the total rental cost) no doubt if you dropped the car in Zagreb or Rijeka, took the train to Ljubljana, and rented another car (if need be). But for four people, maybe the extra drop fee or rental cost is worth the convenience of not needing to change cars and take a train.