Some may have online booking. Others may have only a telephone number and an email address.
I found the establishments below, but I know that in Monterosso there are at least 10 or so just in the Fegina side and maybe another 2 or 3 on the old town side . The first one (Stella Marina) has online booking capabilities. I tested for August 15, which, being a holiday, is probably the busiest day of the year, and they have plenty of spots available. There are also a couple of free beaches (not occupied by establishments) on the Fegina side and one on the old town side, if you don't mind planting your own umbrella in the sand.
Generally Italian families book umbrella spots at some bagni for weeks at the time in August, but I'm not sure if at the Cinque Terre the establishments do so or if Monterosso is a popular place for Italian families that spend weeks in the same spot. Most Italian families avoid the Cinque Terre for that purpose given the crowds nowadays, and therefore most will probably show up for the day if they happen to visit. It's been a couple of decades since I've gone there and rented an umbrella in July (and even longer ago in August, maybe 40 years ago), but I bet if you wing it and show up early enough in the morning you'll find places that are not pre-booked. Nobody goes to the Cinque Terre anymore, it's too crowded (Yogi Berra).
http://www.stellamarinamonterosso.it/
https://www.maravelacinqueterre.com/bagni-alga/
http://bagnifegina.net/