Seat reservations on ICE trains

How come certain seats are always greyed out on the seat map and not bookable? Are those first come first serve or are they reserved for something else?

Partly reserved for people with handicap, partly reserved for travelers with high loyalty status.

If you don't have a seat reserved, and some else reserves that seat, you will have to move.

If you want to be certain, you need a reservation. Otherwise you can enter and look at the displays if there is a reservation (provided they work - they sometimes fail). In case you find a seat not reserved, you are safe from there.
If you only find reserved seats, the display should tell you from where on it is reserved - if someone with a reservation shows up, they will ask you to move (and it is their right to do so).

Not all seats are usually in the pool for reservations. Some are for loyalty members, some for disabled passengers, and a few are actually still held as a reserve for people who did not reserve (afaik, might have changed) and work as first come first serve.

Generally speaking, at least in my experience, the indicator is roughly reliable. Note however that it is partly just based on reservations made for this specific train and partly experience/statistics. If you extrapolate from a different day, be aware that a special event like a business fair or soccer game can increase demand significantly.

Thank you

Partly reserved for people with handicap, partly reserved for travelers with high loyalty status.

Are they free for all if it is not occupied when the train leaves?

Are trains usually full on the first station of the route?

When you board the train, each seat will display if the seat is reserved and from where to where. It's too variable to answer when and where trains are most full along routes. Commuting times and Friday PM and Sunday PMs are obviously busier times. What month it is and school schedules will also affect this.
You could keep checking how many seats are booked on your train each week leading up to your trip. If you are worried about it, it may just be worth making a seat reservation now. You'll have the seat you want and will not have to think about it again!

thanks @mel

I believe it's possible to reserve a seat very close to departure.
Decades ago, my DH and I boarded an ICE and found great window seats in a compartment which were not showing as reserved. Just as the train pulled out of the station, a man and his dog appeared with reservation in hand. We had to move.

The seats all have an electronic board above them, that will let you know if it is reserved or not or from where to where. Let's say you just want to travel from Frankfurt to Köln, but your seat says reserved from Köln to ...... You can sit there then.