We arrive in Southampton early on a Friday of a May bank holiday weekend and plan on spending the day in Portsmouth before traveling public transit to sleep Friday, Satuday and Sunday nights in Lyme Regis. If we spend Saturday looking for fossils is there anything to do on the second (Sunday) day? Would public transport work on the holiday Monday to let us spend some time checking out Salisbury en route to Bath for our next two or three nights, or are buses and trains on very limited schedules? We end our week in London before flying home on Sunday.
Look up the nationaltrustuk.org site for lots of good info on Lyme Regis area. There is a huge estate called Golden Cap which is evidently quite the nature preserve. Open every day from dawn to dusk. Has fabulous views of ocean and cliffs. Be sure to see the ship Mary Rose when you are in Portsmouth.
Since no-one with personal knowledge replied I did some searching for you. Buses from Lyme Regis to the nearest railway station at Axminster are operated by First Dorset and South Somerset - for timetables go to http://www.firstgroup.com/ukbus/dorset/ and search for route 31. Buses are two-hourly on Sundays. Usually the same services operate on public holidays, or sometimes it may be more frequent for a tourist-heavy seaside town like Lyme Regis in the spring/summer.
Trains from Axminster to Salisbury are operated by South West Trains (http://www.southwesttrains.co.uk/uploads/pocket20.pdf) and run every hour on Sundays. There are fairly frequent trains from Salisbury to Bath for the next leg.
Be warned, though, that there are sometimes railway engineering works at bank holidays that add time to your journey, and it's too early now to find out if there will be works on that route in May.
You might find it prudent to check with your hotel. The seafront took a real bashing this week, and more coming again tonight. I don't know if there was any damage to the Mole but saw seriously big crashing waves on it.
Many of the English and Welsh seafronts facing south or southwest have been damaged this winter, some pretty badly; the train line west into Cornwall has been washed out.