You have indeed made a very good choice of where to stay = a quiet valley in the Black Mountains. The road up to Llanthony is quite good but take any bends slowly. When you go beyond Llanthony, the road becomes narrower and sometimes you have to reverse to a passing place. If you venture up this road, you will come to the tiny Chapel at Capel-Y-Fin = well worth going inside and signing the visitors book. Continuing northward, the road climbed out of the valley and over the Gospel Pass from where you get sweeping views northward. Eventually, it descends to Hay-On-Wye. So, this route is well worth driving - providing you can drive backwards if need be to passing places. You could return south to Llanthony via another route.
Now to answer your query. You can indeed take the train to Cardiff and to visit the middle of Cardiff plus Cardiff Castle/ Museum and the Bay area, the train would be more convenient than using a car. You could also buy a ticket from Abergavenny to Caerphilly which is £14.80 (pay on the day return) - Cardiff is £14.30 pay on the day return. If you buy the ticket through to Caerphilly, change at Cardiff Central & your Caerphily train should go from platform 6 - roughly every 20 minutes - less on Sundays. When you return to Cardiff, I would get off at Queen Street Station. Now, because you have a through ticket to Abergavenny, it will not open the barrier - so go to a staff member & say you have a Caerphilly to Abergavenny ticket but wish to now see Cardiff on your way back. They will then open the barrier without using the ticket. You will need to do the same thing when going through the gates at Cardiff Central. However, you may wish to also consider making a separate trip to Cardiff by car from your base. If doing this, you could then cover these places around Cardiff > Caerphilly Castle > Castell Coch (on northern edge of Cardiff) > join M4 at J32 and off at J33 to A4232 for just over 1 mile - left (hard) turn into St.Fagans Castle & Museum (which takes at least 2 hours to see). You might even be able to add Dyffryn Gardens - which is about a 12 minute drive from St.Fagans Museum. If driving to these places, I would take A465 to Abergavenny and then A465 towards Merthyr & then follow signs for Cardiff via A470. At the approaches to Cardiff, you will see the sign for Caerphilly. When you leave to return to Llanthony, head east on the M4 to J24 & follow A449 to Raglan & then A40 to Abergavenny for A465 back to base. (The A4042 route from Newport to Abergavenny is shorter but has many roundabouts).
At Abergavenny rail station, there is a shortage of parking on the town side. Look at this satellite view of the station:> https://email@example.com,-3.0089734,371m/data=!3m1!1e3!5m1!1e1
You will see the A465 coming down from Llanfihangel Crucorney - which is at the entrance to the Llanthony valley. Can you see cars parked in a lay-by on this road? Most of these people have left their cars here in order to catch the trains, If you wished to use free parking on this road, I would drive down to the roundabout at Abergavenny and loop back northwards and use the lay-by that is on the station side of the main road. (Vehicles are travelling at up to 60mph on this road). You will find a rough track (be careful) leading to a footbridge at Abergavenny station. Go into the station to purchase your tickets. Then back over the bridge to the southbound platform.
I would also consider going the other way on the train - to Ludlow. Pay on the day off peak (10am+ Mon>Fri) return is £19.90. On the way back, consider stopping at Hereford - though it is almost a mile from the station to the centre/cathedral. (Same thing at the ticket barrier as advised earlier). http://www.ludlow.org.uk
Also consider going down a coal mine (free as part of museum service).https://museum.wales/bigpit/