I think you pretty much have to drive. There is no train station in Gordes. There is no bus from Arles. I know there are tours that go from Avignon or Marseille, but I doubt there are any from Arles.
To avoid driving, you would need to do the following. Take the train from Arles to Cavaillon (which requires a train change in Miramas). Then take the bus from Cavaillon to Gordes (only a few times per day). This would take significantly longer than driving.
If you go, you will love it. Gordes is one of the most beautiful villages in France. If you go, I hope you will also go to the Senanque Abbey - it's beautiful and impressive.