The Cambrian Woolen Mill is located at Llanwrtyd Wells - which is deep in the heart of rural Wales. If you intend to visit this place, note that it is on the Heart of Wales Railway = a slow single track line that has infrequent trains running between Swansea (with fast frequent connections from Cardiff) and Shrewsbury. The HoW is very scenic. http://www.heart-of-wales.co.uk
The Woolen Mill is about 1 mile north of Llanwrtyd Wells on the A483. If you do intend to visit, I would definately request that they send someone to pick you up a the station - and return you to the town or station. (Bear in mind that the trains are usually 4 hours apart and that this would massively eat into your limited time in Wales).
If you did use the Heart of Wales Railway, it would be silly to return to your starting point - far better to keep going and use it as your link between south & north Wales via Shrewsbury. The quickest way to get from Cardiff to Shrewsbury is via the Marches Line which follows the Wales / England border (roughly). Cardiff to Shrewsbury on a fast train is 1 hour 45 minutes.
You can find train times and fares here:> www.nationalrail.co.uk (On longer journeys, book specific trains in advance will get you cheaper fares than turn up & go fares). You should also check the fares against Passes :>http://www.arrivatrainswales.co.uk/ExploreWalesPass/
Buses can be found here:>http://www.traveline.cymru
When in Cardiff, I would definately visit this place - which is 5 miles west of the city centre. (Bus 32 or 320).
Also visit the waterfront in Cardiff:> http://www.visitwales.com/explore/south-wales/cardiff/must-do-cardiff-bay