I would respectfully disagree with Paul, who may be working on the basis of one trip to Greece ... and perhaps he is thinking about local busses in Athens that may get hung up in traffic jams, not the "KTEL" intercity busses, which are modern, air-conditioned, reserved seats and -- during many usages in my 12 trips -- virtually always departed on time.
Your concern needs to be with Day 2 -- you say you need to be back in Athens and "get on a ferry" by mid-afternoon. Working backwards: if you need to catch a 3 PM ferry you need to be in line on your PIraeus Pier by 2:30- 2:40, ticket in hand. If you come from Delphi OR from Nafplio, to either KTEL bus station, there is NO direct bus from either to a ferry pier -- a taxi will take you 25-30 minutes & cost you €20 or so. Thus, you'll have to arrive back at an Athens bus station by 1:45=2pm to allow time to get luggage, stand in taxi line etc. Here are the bus timetables. Delphi only one bus back to Athens would suit you. It's a shorter trp from Nafplio, 2.5 hours, and busses are hourly.
DELPHI: http://www.athensguide.com/practicalinfo/busschedulesB.htm The 11 AM bus would arrive back in Athens 2pm; the schedule says 2.5 hours, but in my experience this one is more like 3
NAFPLIO: http://www.athensguide.com/practicalinfo/busschedulesA.htm Busses back to Athens go 9:30am 11, 12:30.
Having done busses to both, I'd tend to vote for Nafplio. My thoughts on both:
NAFPLIO - THe shorter trip, more frequent busses would give you the entire day to enjoy this beautiful Old Town. Get the 9:30 bus and you'd be there by noon. This outbound journey can be enjoyable for the view (ask for seats on LH side outbound to see Saronic bay). In town, check out its small superb Arch. museum on square, Go up to the top, Acronafplia (there's an elevator built right into a cliffside), enjoy what I consider the BEST sunset in Greece from the waterside promenade ... and have a choice of many tavernas with traditional cuisine. If you wanted some ancient ruins without having to trek to Mycenae, there's Tiryns just outside town. See if you can get a sea-view balcony room at hotel Agamemnon, one of the only Nafplio hotels right on the seafront.
DELPHI - If u choose this, outbound there are 2 busses, 10:30 and 15:00 (3pm) ... I would suggest earlier one, because you'd want to do y our main site-seeing on day #1, having to catch 11 AM bus day #2. The tour-group throngs, fortunately for you, will be clearing out of the site by about 2 pm, so it would not be as crowded. Hotel PAN has great views from its BACK rooms looking down the mountain (a hawk circled BELOW my balcony!). Yes, not much of a dining destinations, but there are some nice outdoor terraces, looking down the mountain at sunset, pleasant for a spaghetti dinner. DO NOT MISS the Delphi WATER! It comes from the top of Mt. Parnassus, and is Nectar of the Gods.
A WISTFUL ALAS -- if only you had 2 days, I could recommend the PERFECT Ruins-plus-Beautfiul Town Trip -- to Nafplio, Mycenae and Epidaurus. http://www.chatours.gr/tours.php?exc=8&exccat=1 Runs Tues-Wed or Wed-Thurs thruout season... The great advantage to this trip is, FREE TIME It takes you to Mycenae then on to NAFPLIO where you have entire afternoon evening & next AM on Your own, to enjoy its beautiful place. Then It goes to Epidaurus en route back to Athens.