Train Palma de Mallorca to Soller

A Sunday in August will be very popular. I would recommend taking the earliest train that you can. Normally I would advocate picking up the train at Bunyola as that is the point where the scenery becomes more interesting however as you will already be in Palma it would make sense to start there plus you're more likely to get a seat. You buy the ticket at the train station and take the next available train, there's a timetable online.

There is no bus that takes the same route and even if there was it just wouldn't be the same. The joy of the journey is the train itself plus the winding route around the mountains. Make sure you try an ice cream from one of the many sellers in Soller, I particularly recommend the one made with local oranges.

It's same-day ticket purchase only...ie no advance reservations. You usually need to show up about 45 minutes before your proposed departure time to ensure that you get a seat. Yes, it'll be crowded on a Sunday.
Not sure whether you can reserve one of the return times when you purchase your outbound ticket to Soller. Check their website for details : http://trendesoller.com/en/train/
Don't know about a bus. Depending upon how many will be in your group it might be cost-effective to just spring for a taxi to Soller and get tickets there for the short ride on the narrow gauge railroad down to the port, which is the scenic highlight of the trip anyway.

I'll be in Mallorca the week after you and from what I have read, there is a bus between Palma and Soller that takes about half the time and goes along the main highway. I think I will opt to take the train out to Soller and the bus back. I also hear the trains coming back to Palma get super packed and people end up waiting a really long time to get a seat on the way back.