I went to Giverny last month on a Monday mid-morning. You are wise to buy your train tickets in advance. I nearly missed the train because of the line (mostly tourists buying tickets to Vernon). Also plan to board the train a few minutes early. The train was chock full. In addition to the many tourists going to Giverny, there were a goodly number of locals going farther, with luggage that often took up an extra seat. There were people sitting on the steps and standing all the way. In Vernon, you go down a flight of stairs to cross under the tracks to the station. There were 3-4 buses lined up for Giverny. I was in line for the first one but the rear one left first! The tickets are purchased on the bus and it took a long time until we were ready to go. The tickets had to be processed in a machine by the conductor and it was slow. Added to that, he only spoke French so it took time to explain each time whether to buy one-way or return tickets and whether the younger kids needed tickets or not (age-dependent). Then there's a few minutes' walk from the bus to the house. I was lucky that I was near the front of the ticket line - but again, it went slowly. There are choices with combined tickets that had to be explained and then the choices had to be considered.
For next time, I would definitely take an earlier train to get there when they open and get tickets for the train and the sight in advance, and maybe rent a bike in Vernon to get to Giverny or take a taxi (but return by bus). I think most people spend between 2-3 hours at the sight, then there's a small museum (I didn't go) and Vernon looked to be a nice place to spend a little time wandering around. Note that the buses back to Vernon are timed for the trains. If you plan to take the bus, you could have a long wait mid-day when there's a "gap" in the train schedule.