Is it possible to do this by plane cheaply? Train is nicer but more expensive.
Check flights for your dates at skyscanner.net
AirFrance has the route, no idea of the price. The Bilbao airport is up on the plateau to the east of the city, but I think there are buses into town (or maybe you have to ride the funicular down to a bus line). The flight should be less than two hours. Trains into Bilbao get tricky time-wise at the border.