Hi Kathy,
You are correct, just shoulders and knees (and midriffs :-). If you are sleeveless, you can use a scarf to cover your shoulders, though it can be hard to keep the scarf on while you're taking photos. Short sleeves are fine. Pants or a skirt (often cooler than pants) that reach just below the knees are fine. There are some Russian Orthodox churches (and maybe the Western Wall) that require women to wear a skirt. They will give you a large scarf to wrap around you for your visit if you're wearing pants.
In warm weather, nearly everyone wears sandals, usually open-toed. More important is that they are comfortable with thick soles, since there are lots of cobblestones. You should bring a pair of closed shoes (hikers, sneakers) for sites than aren't paved like at archeological sites and national parks.
You should bring a hat (against the sun), sunglasses and good sunscreen too. If you need to cover your head in a mosque (I can only think of one or two that are possible on the typical tourist's itinerary), your hat will do fine.
Edit: I see from another post that you are in fact taking a Palestinian tour that is mostly in the West Bank. I think you should ask your tour organizers (friends?) what would be acceptable - it's outside of my experience.