how difficult is it to get around Crete without a car?

It's not difficult, just time-consuming to use the bus system. It takes a minimum of a week with a rental car just to barely scratch the surface of what Crete has to offer. I can't imagine what it would be like by bus. Crete's roads are fine. There's nothing to worry about. If you live in a country where drivers use the left side of the road it might be a bit of a challenge at first, but after a while you'll be fine. If you do decide to rent a car I strongly recommend you reserve one with an automatic transmission so there's one less challenge you'd have to deal with using a manual transmission.

Why does the thought of driving in Crete scare you?

If you are used to driving regularly, is there something specific and unique about the roads in Crete that would prevent you from driving, or have you already had a bad experience driving in Crete?

In this case you can use public buses.
In Greece public buses are managed by the Ktel companies.
The main website for Crete is this:

https://www.e-ktel.com/en/

The specific website for Heraklion is

https://www.ktelherlas.gr/en/

The bus network is good but be aware that you will be completely dependent on bus schedules and routes, so you will probably have to remove some "remote places" from your plans.

That being said, Crete sees around 3 million tourists every year. If there was any particular danger in driving, it would be known.

What frightens you about driving on Crete?? Are you renting cars in other areas of Greece? We spent two weeks there in Chania and covered a good portion of western Crete including the southern coast.
Touring by bus on Crete would not be easy or productive. Rent a car and enjoy Crete.

Our group of 4 rented a car for two consecutive days of our 12 days on Crete earlier this month. The highways and roads are well maintained, and the Greeks seemed to be very good drivers. There is a driving custom on Crete (apparently not on the mainland) that, if safe to do so, drivers stay to the right - including driving on the paved shoulder - to allow others to pass. That was easy to adapt to. Where you don't want to do much driving is in places like the old town sections of Heraklion and Chania, with narrow, sometimes winding little streets that are often one way. And we drove through a few small towns in the mountains where slowing down and allowing oncoming cars to pass just made sense. But in just the two days we had the car (with an automatic transmission) we visited far more places than we could have done relying on the bus system, and we felt safe the whole time.

You will see half as much and spend twice the time getting anywhere. You won’t be able to reach any famous beach or hiking spot.

Just rent a frikin car. We drove all over the place. The roads are good. It isn’t scary.

Any time you do something new regardless of what it is there is some anxiety involved.

Driving in another country is definitely anxiety causing.

Just because you drive with no problems where you live doesn't mean it will be the same in Greece or another country.

However I've driven on many islands and the Peloponnese and for the most part there were no problems. The biggest challenge is using a paper map which may be hard to follow especially on secondary and back roads.

I use Google Maps for navigation and for the most part works well with some "hiccups" here and there. Eventually I got to where I wanted to.

Crete is the largest of the Greek Islands and you could spend weeks and still not cover everything.

With 10 days I would select one part of the island rather than driving to all parts of it. There's more than enough to experience in each area to keep you occupied.

While there is decent bus service it doesn't go everywhere and the schedule can be difficult for planning.

Rent a car, you'll see far more and experience more out of the way areas of Crete and driving will not be any more difficult than back home other than not always knowing the route you are taking to get somewhere.

We rented a car for half the time we were in Crete. You can do a lot without a car. We flew into Chania, stayed three nights, and then took a bus to Hora Sfakion on the southwest coast. We traveled by ferry (there are no roads here)nand then took a bus back from Sougia to Chania where we rented a car. But we could have easily taken another bus and gone to Rethymnon and Heraklion. You can take a bus to Knossos.

The buses are modern and pleasant. You can go a lot of places easily but of course a car gives you more flexibility.

My family and I spent two weeks in Crete this summer and due to medication I was taking I was unable to do the driving (I always do the driving usually) so my wife undertook the task. She hates driving in a foreign country, even more so if they drive on the right yet even she quickly became comfortable with driving there.

I couldn't imagine trying to explore the island by public transport, it would be far too time consuming and restrictive.

In October 2018, on Crete I took the bus from Heraklion to Chania. The bus ride worked out ok. I bought the ticket at the bus station about one-and-a-half hours in advance. The station might have been semi-fancy with a cafe, but just before getting on the bus I was waiting under a covered semi-outdoor area with other passengers. Since my trip to Greece, I have rented a car twice, in Southern California, and in Florida. I have not rented a car in Greece or in any other country (yet). Those were my only two times renting a car. KTEL is the long distance bus company in Greece. The company is divided up into subdivisions that are each operated semi-independently in different regions. Thats why there might be more than one KTEL website. You don't need to rent a car on Crete. I was traveling solo. Don't waste your money on renting a car that you don't want to drive anyway. Don't rent a car just to avoid feeling stupid for going against the advice from other repliers (who do not have experience taking the bus). I assure you that you will be fine with the buses. Here is what might be accurate schedules. Or the schedules might change before your trip. Check again just before your trip.

https://e-ktel.com/en/

Also I took a local bus in Heraklion to and from Knossos, and from near my hostel in Chania to the place my overnight ferry departed.

In Heraklion just after I got off my ferry when I arrived, I saw waiting taxis. I got a ride in a taxi to my hotel.