Where can book the bus tickets from rennes to foret de paimport?

I am not sure the bus goes all the way to Foret de Paimpont. I found a schedule (via Rome2Rio.com's link) for the 1a bus to Paimpont. That bus has a number of stops in Rennes. Although I know nothing about this particular bus, my experience in France was that if you were boarding at a bus station (Gare Routiere), you were expected to buy a ticket in the station before boarding the bus. If you were boarding at a later stop, you just paid the driver. I can't guarantee that the Paimpont bus works that way.

It's not usually necessary to buy bus tickets in advance, but if you're concerned about getting on a particular bus, it would be best to go to the starting point, the bus station.

There's also this schedule for a similarly numbered bus.

I don't read French, so the details are lost on me.

There seem to be no 1a buses on Sundays.

As previously precised, you can buy bus tickets at the "gare routière" right next to the train station in Rennes.
There is no other stop in the forest than in the center of the town of Paimpont. Most of interesting places in the forest are not close to Paimpont (forest called "forêt de Paimpont" or "forêt de Brocéliande"). So if you are considering to go to the forest from Rennes over the day with the bus, it would be difficult to make the most of it.