We'd like to tour the gardens and castle but we will have a car at this point in our trip so we don't want to take the train from Rome. We also have a specific date and the only day for individuals to book is Saturday. They take groups of 15 or more on other days. Are there any tour companies that build groups and meet right at Castle Gandolfo?
I don't think you have any other option. An established tour company like Walks of Italy is just tagging along with you on the official Vatican-led tour and they assume everyone's starting from Rome.
The gardens (Gardens of the Pontifical Villa, a.k.a. Villa Barberini) are at Albano Laziale and the "Castle" is the Apostolic Palace in Castel Gandolfo. The Apostolic Palace in Castel Gandolfo (right in the center of the town on teh main square) has tickets available 6 days a week. You self-tour.
The Gardens of the Pontifical Villa are also open daily, and just looking at September, there is spotty availability but definitely they are open for tours on days other than Saturdays. You take a little train through the grounds and there is a decent audio guide.
If one does not have a car, seeing both the same day requires a taxi or it is a long walk between the sites.
Are these the sites you want to see? Where are you seeing only Saturday availability? This is the Vatican ticket site.