Over the last few years on 3 occassions I have tracked prices from Texas to Central Europe. Between 200 and 300 days out I looked for the best cheap flight (one change max and 2 to 4 hour layover) and then had Google Flights send me updates on the prices The prices bounced all over the place, but they all were at their cheapest between 4 and 6 months out. But what was more interesting is that flights that cost more when I started my search often became the lowest cost flight. Fortunately Google send you updates on those too. All I learned was to wait till 4 to 6 months out and use the cost matrix where you can see if moving things one day at one end of the trip or the other end of the trip will produce a savings. Google's date/cost matrix always sets my travel days.
If the direct flights are such that I had to cut a day out of my vacation to pay for them, well, the first day with the Venice connection is also a day out of my vacation, so it breaks even.
So flying to a place that would require an entire lost day and require a hotel in a town I wasnt interested in plus the communte from the airport and to the train the next day is more underutalized time: I wouldnt do it. But this always comes down to personal values. There is no wrong.
But, if flying to Budapest or Vienna or Krakow all cost about the same, then maybe an open jaw would also cost about the same. Nothing more efficient than heading in just one direction. And there is always the economy option. Dont know what sort of tickets you are looking at, but one or 2 with a checked bag and the others with carryon only might assist with the price. Its how I took my two kids on their first trip to Europe. We played the carryon and personal item to the hilt (but within the rules).
I also dont drive on holidays. Well, I did it once and it wasnt pretty. I am on holiday. And the cost of a car can multiply fast, the rental fee is only half the cost (okay 65%). Then to avoid the drop off charge I would end up back tracking. Okay if it were a tight region to be seen and a loop really works without detracting from the overall enjoyment, but when crossing a lot of real estate I dont want to go back. First I would see if I would get as much personal value out of trains mixed with a few private tours. If its close, equal or only slightly more thats the way I would go.
I guess what i am saying is that the equation is greater than the ticket cost and only you know the parameters for your best and most enjoyable holiday. What works and worked for me just wont work for everyone so all i can offer is things to think about, but no answer.