In case you haven't already, I recommend you thoroughly explore this Aiken website and contact the Tourism Division listed at the bottom of the page. They can probably answer all your questions and then some.
When I was 19, I made a cross-country trip on a Greyhound bus from San Antonio to Hampton, NH. I had to take the bus back home, too. I don't recommend it!
If you absolutely don't want to fly, you might consider Amtrak. There are a couple of routes that might get you close enough that a bus to Aiken wouldn't be so challenging. The town seems to be closer to Augusta, GA than anywhere else. But you would have to use Google maps in combination with the Amtrak (or flight) schedules and the Tourism Division to see what would be your best options.
I know many residents of NYC do not drive. Unfortunately, driving is a skill that is almost mandatory if you want to visit anywhere except a big city with lots of public transportation. That goes for any country.
I live in an area where there are lots of horses and I see people riding them frequently. If those folks want to transport them, driving a truck and pulling a horse trailer is required. I also have a friend in WA state who owns horses and regularly takes them places to camp and ride. In fact, she chose her CA college(s) because of the horse programs nearby.
Good luck with your trip. Where there's a will, there's always a way. It just may not be an easy way.
