We were there Apr this year (9th - 16th, then on to Belgium). It was our first time there (stayed in Ams proper) so I really don't have any historical context on crowds.
We didn't have any issues with the Rijks - we bought a combo ticket with our Canal Cruise, so we were able to bypass the ticket line - which was not very long on a mid-afternoon in April. Not sure if it was considered crowded inside compared to summer months, but I will say that we were there until closing and the line up for people waiting to get their coats/checked bags was very VERY long. (We hadn't checked anything). We stayed 2 hrs and saw about 2/3rds I think.
With 3 weeks and a car, you can see lots. I'd love to go back for a week or two to explore more of the NL. From Ams, we visited Haarlem, Den Haag (for Madurodam - so fun) and Keukenhof. Then we moved on to Dordrecht for a few nights and stopped in Rotterdam on the way there, went to Kinderdijk and also enjoyed a visit to Gouda.
Now, you have a car so that will make getting to Keukenhof easier. I have a 'def not a morning person' husband, so getting there early to avoid crowds was not in the cards. BUT...I think we stumbled on an even better idea. We arrived around 1pm-ish (took the bus from Schipol). The parking lot was crazy busy, so we grabbed some fries from a truck in the parking lot, then rented a bike (Rent A Bike Van Dam) and spent a few hours tooling around the bulb fields - it was one of the most glorious things I've done. And I'm not the best on a bike, but it was pretty easy going. We got back to Keu around 3:30-4ish and the parking lot had really cleared out. We had the pavilions inside almost to ourselves (not really, but compared to how some people described them as being elbow to elbow, they weren't very full at all) and the gardens were not crowded. So if you can't get there early morning, go early afternoon, rent a bike then head into the gardens mid afternoon. Maybe we were just lucky that day, but it worked out perfectly, since I hate crowds.