We've traveled to Europe with our kids (now three of them aged 1, 4 and 6) at least once a year since our oldest was born.
We have always taken our car seats from home. I have never felt comfortable renting one, first because you can't be sure they have never been in an accident and second because I have heard stories of people arriving to pick up their car but there was no car seat, despite it having been reserved. If you search Amazon, you can find car seat carry bags with backpack straps. On our first trip we didn't put our car seat in a bag and it got pretty beat up, so I recommend a bag - the backpack style bag made it easy to have hands free to hold kids or extra bags when moving through train stations or airports.
I suggest you rent apartments as much as possible - look for ones with outdoor spaces or playgrounds. Plan to eat out for lunch and then visit a local market and have your evening meal at the apartment. Your evenings will be much more relaxed and enjoyable if the kids can move around don't have to worry about sitting still and quiet in a restaurant.
Everywhere you go, ask where the local playgrounds are and go there. The playgrounds web have found in Europe are amazing and really help buy us some kid goodwill for more 'boring' (adult) activities.
I always pick up a jar of peanut butter and some bread. Our kids eat pretty much everything when we travel but I make them each sandwich in the morning and there are times (usually standing in line somewhere) when a familiar, comforting snack makes all the difference to the day running smoothly. I always carry a bag of other snacks as well.
If you are planning to have your kids sleep in regular beds and are worried they might fall out, see if you can find some inflatable pool noodles. I picked two up at Walmart a few years ago. Tuck them under the fitted bed sheet along the edge of the bed and you have a DIY bed rail.
Take a decent double stroller, if possible. We have an older version of the Phil and Ted's Dot. It has lasted us 6 years and at least 9 European holidays and is still going strong. If your 4-year old is like mine he/she will want to walk but having the extra seat in the stroller will be a life-saver to keep them safe in crowded spaces or if he/she gets tired. It also makes a great place to store your bags and coats when they aren't riding. I highly recommend the jogging-style stroller. I can't imagine using anything else on all those cobblestones.
I personally haven't liked to visit museums with our kids just yet. It isn't worth the stress of keeping them happy and quiet. We more than make up for this by viewing Europe from a more human/local perspective - playgrounds, fountains, ice cream shops, zoos. We'll have plenty of time to visit the museums when they grow up and that's going to happen far to soon for me.
When our two oldest were 3 and 5, I started getting them to create a Journal of their trip. I bought them each a multimedia art book from a craft store (for the heavier, unlined paper) and took a small watercolour palette, markers, glue, scissors, stickers and one of those Instax mini polaroid cameras. I had them draw pictures of the things they were experiencing and special memories and helped them glue pictures from the polaroid camera, brochures, postcards, etc into the journals. Now that my oldest is starting to write, he writes a sentence or two in his journal each day. This was THE BEST thing I started doing with them on our trips. It gave them something to do when waiting in a restaurant, on a particularly rainy day or on the airplane. It gave them something to show grandparents when we got back home and now they love when I bring out their Trip Journals for them to look at. It has really helped them retain the memories and details from our trips in a way they wouldn't have otherwise.
Have a wonderful time, traveling with your kids is one of the best things you will do!
