Jill both times we rode bikes at Kinderdijk we were bicycle touring, so had bicycles already and rode through with our bags too another overnight stay destination.
Generally though you have a set of options I would think for cycling Kinderdijk. It appears that you can rent bicycles from a couple of outlets near the entrance to Kinderdijk. So you could travel there by whatever ordinary means and then just use a bicycle to tour around the windmills.
https://www.kinderdijk.com/plan/directions-address/by-bike/
Or you could travel to Rotterdam and then rent a bike from one of the many rental shops in Rotterdam. I'd look at reviews and reserve ahead of time. The rental experience can vary from delightful to crusty grouch. I recall once renting from a bike shop on the same street as the Rotterdam train station a couple of blocks down the way to the viewers left as you face the station. It was a nice shop and the guys working there were pleasant.
Then ride down to the water taxi. You have to cross the river anyway. It's a big ferry boat and if I recall the bikes just roll on free and easy. Decide which stop you want to get off at depending on how long you want your ride to be. Then ride out to see the windmills, stop somewhere on the way or back for lunch. Decide how far you want to ride back.
I've also rented in Dordrecht and used the ferries from there. If it was me I think I would choose Dordrecht as my start and finish point, smaller city less busy easier to deal with. If I recall correctly the water taxis go there as well, or at least you can ferry across the river.
A last option would be to rent bikes in Leiden, take them on the train to wherever you think is a convenient/interesting stop to ride to kinderdike from, have your day out on the bikes, and return the same way.
"bikes can be taken on all trains except in peak hours (06:30-09:00 & 16:00-18:30 Monday-Friday, no restriction on national holidays or in July & August). You need to buy a one-day bike ticket for around €7.50."
Upside here is you can get the bike squared away in a leisurely way and go where you want. Downside, outside of the no bikes on the train restriction times noted above, is that it can be a pain in the butt dragging the bike up on the train depending on the type of train it is. And how many bikes you have. And how heavy they are. And how many other bikes are already on the train. And how annoyed the Dutch people whose way you are getting in might or might not become. You can look at rail websites and generally know which type of train. The local commuter style sprinter trains are usually easy to roll on and off with a lot of open space.
Anyway, many ways to skin the bicycling cat :-) have a great time