We have traveled to Europe 6 times with our kids who are now 4 and 2. I would not call any of our trips 'difficult' but that is because, like you, we recognized that it would be a different kind of trip than when we travel on our own. Not better or worse, just different. You're right, you won't see many museums on the other hand kids easily open doors for conversations with the locals (I can't tell you how many nuns stopped us in Florence to admire the kids).
Most of our travel has been by car but we have taken the train a few times. I think the train vs. car decision really comes down to a personal choice on what you want to see and how you want to see it. Both train and car come with their good points and bad when traveling with such young children. If you are planning to rent a car, I recommend taking your own car seats for two reasons: 1. You can't know if a rental car seat has been damaged in an accident, 2. I have heard horror stories of people arriving to pick up their car and no car seat was available. It can be a bit of a hassle but I think the peace of mind is worth it. We purchased a backpack style bag to carry our car seats (on Amazon for $35CDN). It protects the seat on the airplane and the backpack makes carting the seats through airports and train stations so much easier. Note: not all of our rental cars have had the tethers so be sure you double check that the car seat can be installed by seatbelt, too.
I highly recommend a baby backpack/carrier (we take an Ergo) for the 2 year old. It made it so much easier to keep track of her in airports when we might be distracted (i.e. security). It also made getting on and off planes, trains, busses, etc MUCH easier. It allowed me to have my hands free to hold on to our other little one or to help my husband with our bags. We could visit museums without fear of her touching something and she napped well in it. I used a sling on our first trip and don't recommend it. I didn't use it for anything other than holding our little guy on the flight over.
We have a Phil and Ted's double stroller that has been invaluable on our trips. It is a one-up, one-down stroller rather and a side-by-side stroller, so it is narrow and makes getting on and off busses and trains or through narrow doorways easy. Sometimes it takes up more room than we would like in a rental car or hotel room but I think the benefits outweigh the negatives. We've used the same stroller for every trip and it is still going strong. Some say not to take a stroller to Venice because of all the stairs but we didn't have an issue picking it up and carrying it over the bridges - again that is a personal choice. You may find the combination of baby backpack and umbrella stroller works best for you. Either way, I highly recommend having a place you can buckle them into if things get crowded and confusing or little feet get tired of chasing pigeons. If its comfortable, they will probably nap in it, too!
I would recommend a little collapsing cooler bag. We always travel with one with our kids and have a sippy cup of milk, a sandwich (although our kids eat everything, we pick up a jar of peanut butter early in the trip to make sure there is always a familiar snack ready for them the moment crankiness starts to set in) , and other snacks. I loved having an excuse to shop at all the amazing fruit and veggie stands!
I encourage you to consider renting apartments at least a few times. Just one or two nights eating a familiar, homemade meal and being allowed to spread out and play as much as they want (instead of being on best behaviour at a restaurant) can have a hugely positive impact on your kids behaviour over the course of the whole trip.
If you ask around you will find some pretty amazing playgrounds - our favourite so far was in a suburb of Vienna.
Feel free to PM me anytime you want a second opinion as you plan your trip.
