Spent three weeks in Latvia, Lithuania and Estonia during 2018. In the big cities, small towns, and some pretty remote areas.
While I don't often drink "tap water" in places where I have concerns about water quality, I am generally very careful to not allow a single drop in my mouth, nose or other orifices - so in those places, I always brush my teeth with bottled water and take careful steps to keep the odd drop of water out while washing, showering, etc. On other trips I have gotten sick pretty easily from less-than-perfect water & questionable sanitation overseas, so I tend to be pretty obsessive, bordering on complete paranoia, about "water discipline" eschewing ice in drinks, regarding uncooked produce with suspicion, etc.
In the Baltic countries, I took no such measures. I didn't drink a lot of tapwater, but I brushed my teeth with it every day, enjoyed ice in drinks, crispy salads, rinsed off fresh produce in sinks (wonderful berries in mid-summer!) - all things I would never dream of doing in Mexico or most of the developing world.
I had no stomach issues at all in the Baltics.
Unless you were staying/eating in a place that looked visibly funky, I would not be terribly concerned. Maybe skip the water fountains in the Museum of The Occupation and other old Soviet gulag charmers...