Writing Retreat!

One suggestion would be to consider one of the Aran Islands - specifically Inishmore which would be fairly easy to get to via public transportation: upon arrival at Dublin airport hop on the express bus direct to Galway, thence by ferry from Rossaveale (via shuttle bus to the ferry landing from the town) or by flying with Aer Aran.
For inspiration you might read the John Synge classic "The Aran Islands" for a first-hand account of life there.
Lots of cute little B&B's and guest houses to choose from, and the proprietors would be a great source of local knowledge re places to walk to, places to eat or for a pint, etc.
The place is gorgeous and might be just what you're looking for.

Ferries to Inis Mor do now also sail from Galway City between April and September.

You don't mention when you're going, which doesn't necessarily make a difference.
A couple of other places to consider are the ends of three other train lines from Dublin:
Westport is a busy town with a fair amount of tourism. Staying down by the Quay generally provides great views over Clew Bay and some nice walks, and there are shops, restaurants and pubs on hand. It's a bit of a schlep from the train station, but there's a car service you can arrange to meet you and take you there very affordably (I think we paid 10euro in a shared van).
Sligo is a very quaint town that sees mainly Yeats-related tourism. Nearby sights include several beaches and Carrowmore Megalithic Cemetery (on the bus line).
The other is Ballina, which I'm not very familiar with. It's definitely not a big tourist destination. When I googled it, the Belleek Wood came up. Looked pretty.
Of course, there are also the usual heavily touristed destinations in the SW, but you don't need my help for those.

I know there was an artist retreat in Ballinskelligs at the Skelligs Ring near the Ring of Kerry

Very remote spot quiet and wonderful views

Alternative: This B+B on Valentia Island
They offer Shuttle Service at add cost and their neighbour Kennedy has a Taxi Service

Doolin. Walkable. Lots of small B&Bs (try the Fisherman's Rest). On the coast. Ferries to the Aran Islands. Three pubs with lively trad music in the evenings.

I salute you for creating your own writing retreat. I'm a writer, too.

Cloghane, Co. Dingle, might make a good fit. It's on the northern, Atlantic-ocean-facing coast of Dingle, where few tourists go; most of them go to the southern coast facing Dingle Bay. The town is incredibly charming in the shadow of Mount Brandon and surrounded by probably thousands of Celtic antiquities. Given your interest in mythology and sacred sites, a town on Dingle would work.

Either Ballina or Killala, Co. Mayo, might also work. In 1997, I visited the Ceide Fields, an archeological site few Americans ever see: https://heritageireland.ie/places-to-visit/ceide-fields-neolithic-site-visitor-centre/. I remember thinking north Mayo would be a great place for a writer.

A third would be Aughrim, Co. Wicklow. (Do not confuse it with the Aughrim in Co. Galway.) Aughrim, Wicklow, is located in the beautiful Wicklow National Park in the hills and mountains. And you'd be near the famed Glendalough monastic site (but not too close to the touristy place): https://glendalough.ie/