All depends on how you travel and how in depth a knowledge you require. Our first trip to Rome was also for two days only, not sure if we'd ever make it back. On one of the days we took a tour, including a lunch, and it was pretty rubbishy (none of those mentioned above, more of a K-mart Special kind of tour).
We returned to Rome since and did it all on our own. I find that most people skip through the Imperial Forum, and are not as interested in or impressed by ruins as they thought they may have been. The Pantheon does not take long. The Colosseum is a top notch visit and should include the underground.
Don't know what time of year you are going but try for an early entry at the Vatican. I'm no holy roller but I found the Scavi tour to be interesting and very much worth it (booked in advance, you have to provide passport information etc).