Kerry Ireland & Islands without a car

You'll probably need to look into train and bus schedules if driving is not an option. Portmagee is off the beaten path, and it's a long way from Dublin down there. And yes, I'd tell you to spend the night there--or close by. Southwest Ireland has a number of great B&B's in the area.

Getting to Killarney from Dublin is pretty straightforward. There's frequent train service and the journey takes about 3.5 hrs.
Getting from Killarney to Portmagee without a car will be a challenge. I'm not aware of any Skellig tours that offer pickup service from Killarney, so you'll need to rely on the bus to get out there. That journey alone probably takes about 3 hrs., so you'd need to go out the afternoon/evening before and stay over one night in order to be in place for a Skellig cruise the next morning. Believe that there's a return bus to Killarney that departs around 3:15 PM.
There are two types of Skellig cruises and both are weather and sea-state dependent. The one that lands on the island for the hike up to the beehive huts needs to be booked months in advance and is more prone to cancellation than the Eco cruise that just circles the islands. That one takes about 2.5 hours and so would probably be your better choice - less chance of being cancelled and you'll be finished in time for the return bus.
The Skellig Center just across the bridge from Portmagee is worth a stop just to view the story boards and historical displays. They run their own version of the Eco tour from there, though I think there are one or two local boats that depart from the pier right in Portmagee.
Per David, staying over one night is recommended. It's a pretty little village with a few convivial pubs and some very pretty coastline walks.

I’ve had two trips to Ireland. Second one, in 2018, was with rental car, and went north. First one, 2011, was by touring bicycle, but we had 3 weeks, and were flying home from Shannon, not Dublin. But we used the national bus line, Bus Éireann, when we weren’t pedaling. The buses have an Irish Setter on the side, instead of a Greyhound.

So how all this fits into Portmagee or not: partly to avoid extra time and pedaling as evening was approaching, I stayed overnight in Cahersiveen, a few miles before Portree. The morning of my hopeful Skellig Michael boat trip (they decide that morning whether the seas are too rough for the crossing, or if they’ll be able to go), I hopped on the bike, and rode the short distance to the very short ferry from Reenard Point to Knightstown at the western tip of Valentia Island. My boat captain, who I believe has now retired, lived up there, and gave me a place to stash my rental bike, then a ride to the boat docks to await the decision whether to sail or not. We went, and the Skelligs are awesome!

But going to the Bus Éireann Website just now, https://www.buseireann.ie , it seems they don’t go to Portmagee, but they do go to Cahersiveen . That would get you in the area, but a taxi or other arrangement might be needed -even a bicycle rental in Cahersiveen for the day?

After connecting Killarney-Cahersiveen by bus #279A, the combined transport site www.transportforireland.ie shows a van service option to reach Portmagee: This is a partially accessible door-to-door service operated by Kerry Community Transport Ltd and requires advance booking. To book, please call 066 7147002.