Those are mostly Regionale trains with no reservations. Our experience last September, traveling with a group of 8 family members, was that the Regionale trains can be extremely crowded. On one, we had to stand on the steps between the upper and lower seating areas the whole time. On another, we found seats on the upper level, but the aisles filled up with standing passengers and it was very hot and crowded, and we had trouble getting off at our stop.
However, looking at the schedule, I see one Frecciarossa train (FR 9797) departing Venezia Mestre at 10:11, arriving in Trieste at 12:05. These Frecce trains come with seat reservations. If I were doing this trip, I would book that train now. There are group discounts available on Trenitalia, but I don’t know how to go about that. Or you could just book the lowest Freccia days price that is offered for September, at 10,90 euro ( but limited and there may not be 19 tickets available at that price).
You will have to get yourselves from Venice Santa Lucia station to Venice Mestre on the mainland in time for that train, but that should not be difficult.