Layering is the key. During March and April you are likely to see some really nice days (especially farther South) and some cold, wet days. Go with light-weight clothing that can be layered to keep you warm rather than warm clothing (useless and heavy to carry when it's nice).
If you can, start South and work North. It's better to be in Greece in March and Germany in April.
Find waterproof walking shoes that can dress up or down. This time of year I add some darker-colored light-weight trail runners as backup shoes. Women probably want a very light-weight pair of dress up shoes.
Get a hooded but unlined rain/wind parka to stay dry. Layer lightweight sweaters and shirts underneath for warmth.
You will probably get more use out of pants this time of year than shorts, skirts or capris.
The crowds should be relatively light so you don't need reservations for everything. The cost for lodging and airfare will also be cheaper than high season.