We did a similar trip in 2022 and thoroughly enjoyed it. We started with Athens then flew to Santorini. This is beautiful, but does not require a lot of days. From Santorini we took a fast hydrofoil ferry at sunset to Heraklion, Crete where we rented a car. We spent time on the southern coast and then went to Chania. From there we flew back to Athens for another day before returning home.
Crete is definitely too large to completely cover in that timeframe, but it is very pleasant and you can get a good taste. The Venetian port city of Chania is a charming town, and I agree that staying within its walls is worth it.