Hi,
If you want to visit chalk cliffs, Stevns Klint or Møns Klint are your two options.
I don't think Stevns Klint is difficult to reach by public transport. See an example of a journey plan here for the "Stevns Klint Experience Center" or here for just regular access to the coast and cliff. Nb. "Vinkestrækning" means there is no bus stop, you have to wave to the driver to get on and notify him/her if you need to get off.
Møns Klint is further away, so requires more dedication. The chalk cliff here is bigger than at Stevns, so if you're interested I think it can justify the longer journey. The bigger issue is that the public transport bus only goes all the way out to Møns Klint in the peak summer months. Example of how that looks here, during the weeks that it does run.
There are day trips from Copenhagen to Camp Adventure and Møns Klint by coach. Maybe that could be of interest to you? Read more and book here.
I agree with the recommendations above. Worth noting is that if you travel off season the more rural areas (such as Ærø) can be less interesting due to the seasonality of the tourism sector in such places. If you're into contemporary art there is also Arken. And if you visit the kind neighbors in Sweden, I would like to highlight Lund as good choice.