True and not true, JHK. Joigny is in northern Burgundy so one would take the TER. It’s a couple of hours by freeway or TER from Joigny to Dijon. It would make sense to go to Dijon via TGV if the destination were in southern Burgundy. In fact, there are two separate sets of tracks: the TER that goes through towns, and the TGV that by-passes towns via the countryside. I’ve taken this exact TER for decades to visit family south of Joigny.
OP—the trains won’t have many people on a Sunday going south. The later in the day that you wait on Monday, the more people will be returning to Paris if it’s a 3-day weekend. Between Bercy and Joigny there’s only one stop, in Sens. The trains from Gare de Lyon are local commuter trains that make 13 stops before ending in Joigny. Joigny has a station; no change of train needed.
You may qualify for a ticket discount even on a TER depending on: the time of day and your age. Buy at the ticket window to see if you get a discount, rather than a machine. Don’t forget to have your ticket punched in the yellow machine. The basic ticket price on that route won’t change between today and your travel day.