Kayak doesn't actually sell the airline tickets. Instead, it searches multiple sites and returns the various fares for a route and sites that offer them. You will notice that it shows you airline sites (e.g. united.com, continental.com) and consolidators (e.g. cheaptickets.com, vayama.com, etc.) You then pick the site that you want to buy from. I often go directly to the airline itself as I prefer to buy from the airline itself.
Typically you get an "e-ticket". That means you print a copy of your receipt/itinerary and then when you go to the airport and check-in you get your boarding pass ... either by using a machine where you insert the credit card which you purchased the ticket with or by showing your itinerary to a person at the check-in counter. Alternatively, many airlines now let you check-in over the internet and print your boarding pass at home.
The benefit of a travel agency is that they can help you if something goes wrong (you need to change, your flight is cancelled, etc.). When you buy from a website, you have to deal with their customer service if you need to change. I prefer to buy directly from the airline's website as then you deal directly with the airline if something goes wrong.