I don't know where you're staying in southern France. Two years ago my friend and I stayed in the small village of Cucugnan (recommended by Rick as a true back door experience). I'd love to return someday. Stayed at L' Auberge du Vigneron which has a great restaurant (spectacular casoulette). The next morning, after visiting the bakery at the operating windmill, we drove an hour and a half to Carcasonne - definitely worth the visit but didn't stay overnight. Very convenient parking right outside the main city gate.
The drive to Albi is about 90 minutes and it's worth an overnight stop. We stayed for two nights, touring the city itself and followed an abbreviated version of Rick's tour of the Bastide towns. One of the towns we visited was Puycelsi. Arrived close to 2PM and went to the bistro at the entrance to the town. It was in November and off-season so they were only serving drinks, no food. They recommended eating at L' Ancienne Auberge - about the only open restaurant in town. The hotel's restaurant serves meals from Le Terroir. Because the kitchen was about to close, the only thing they would serve us was the special of the day which was rabbit. The meal had to be one of the best that we had on our travels and with beer, wine and a digestiv, the bill for two was 32€. If in the area again,I would even consider staying overnight at the hotel.
Instead of driving straight to Sarlat -about 2 1/2 hours - consider driving first to Les Eyzies-de-Tayac. If you can, reserve a tour of the Grotte de Font-de-Gaume cave immediately arriving there or possibly on-line beforehand. These are original cave paintings, not copies and admittance is extremely limited and there are English tours. From there you might visit the Musee National de Prehistoire. Then drive to the Baynac Feudal Castle on your way to Sarlat.
This is a beautiful area in France.