Most small towns and villages will keep you occupied for half a day maximum.
Rye is good all year round, but be prepared for inclement weather at that time of year. In addition to the town centre, you could walk down the hill to Rye Harbour and walk the loop round the nature trail to the sea and back.
Nearby Hythe is worth visiting for a non-touristy small town. Good walks along the seafront, a heritage steam railway and explore the Military Canal, built to keep Napoleon at bay, but never needed. Winchelsea is also worth a visit.
The Cotswolds have been ruined by the numbers of tourists and the resultant traffic log jams. You could visit Rutland instead, which has similar chocolate box villages but without the crowds. Take a ride on the pleasure boat on Rutland Water or hire bikes and cycle round. Nearby towns worth visiting include Stamford, Oakham and Uppingham. Oakham Castle is interesting. Many gardens to explore if that's of interest plus Leicester is about an hour away (Richard III etc).
Another option would be Yorkshire - Harrogate, Thirsk, Whitby and smaller towns such as Pickering. Surrounded by great countryside.