Purchasing/reserving single train tickets

Paris-Chartres is a TER. Price never changes and it never sells out as there are no seat reservations. Stand if you must..
Paris-Reims is TGV, so you would benefit by buying in advance. Brussels-Amsterdam is a Thalys and would also be cheaper in advance.

Advance rates for the TGV and Thalys are best if locked in about 3 months in advance, rather than several days. The lowest rates sell out quickly, although the whole train probably won't on most departures.

Buy those tickets Paris-Brussels and Brussels-Amsterdam as soon as you know your travelling dates. Chances are that the cheapest tickets are not available anymore if you wait until a week or so before travelling.

Prices do not vary like our airline tickets. The number of discount ticket available will vary depending on time of day (more at 6am, fewer at 11), per train schedule, etc., but the numbers are fixed so when they are sold there will be no more. Also it is rare for any train to be completely sold out so tickets will almost always be available so you buy in advance to take advantage of the discount prices. However, these tickets will come with restrictions as to change and refund.

If the advance tickets on the Thalys between Brussels and Amsterdam are sold out, you can buy a ticket for a slower but cheaper Inter-City train on the day. The Inter-City tickets are more expensive than Thalys advance fares but cheaper than the Thalys normal fare.