Google or use Bing and go to a map of London. The Bing map I know has all the major sites marked.
Hampton Court ~ haven't been yet, but personally, I'm planning to set aside a day for that to tour the palace and gardens. It might take less time, but I'm not sure.
British Museum ~ depends on how much of it you want to see. I used the museum to combat jetlag. I was there for about 4 hours and saw nearly everything. I skipped over the Native American exhibits. It's not far from the British Library which is worth seeing. It's also not far from Covent Garden market.
Buckingham Palace ~ Unless you're going in August and take the tour (book ahead by phone because the line was horrid when I went about 10 years ago) or watching the Changing of the Guard, looking at it and taking a picture takes all of 5 minutes.
Kensington Palace ~ haven't been here yet either, but it's close to Buckingham Palace (K is on the west end of Hyde Park, B is on the east end). I don't know if there is any kind of tour or not. You'll want to visit them on the same day.
Westminster Abbey ~ I'd say give it a good 2 hours, depending on if you do a guided tour. If you're only going to take pictures of Buckingham Palace and Kensington Palace, visit the gardens, stroll through Hyde Park, etc., then you could probably walk the 20 minutes to Westminster Abbey that same day. Other sights near Westminster Abbey are the Houses of Parliament with the clock tower that houses Big Ben. The War Rooms (Churchill Museum). The London Eye is just across the river. The Abbey is a 15 minute walk to Trafalgar Square.
Tower of London ~ at the east end of the city. You'll probably want at least 2 hours for this, maybe more. The crown jewels are kept here and even if the line looks long, it moves along pretty quick. Tower Bridge is nearby and you can go to the top of one of the towers. St. Paul's Cathedral and Tate Museum are not far from the Tower.