What Suki said plus ...
Take a boat around the harbor.
Cheapest option is to catch a public ferry from Circular Quay on what is known as the inner harbour route. It's a ferry route that goes in a circle and takes around 40 minutes and stops at places like Mosman and Neutral Bay.
Also you can catch a public ferry to Manly to go to the beach. It's a longer ferry ride and a great way to see the harbour.
Sydney has a famous beach walk from Bondi to Coogee which follows the cliffs and takes you past a lot of beaches and cafes. It's easy to get public transport to and from the walk.
Sydney also has some really lovely harbour beaches in the eastern suburbs which are great for swimming - Google Redleaf, Parsley Bay and Nielsen Park. Beautiful places to relax and enjoy life.
If you are into hiking then a day trip to the Blue Mountains for a bushwalk is another great thing to do. There are tour companies but you can also easily do it yourself by catching a train to Katoomba, Leura or Wentworth Falls.
For great seafood go to Doyles restaurant on the waterfront at Watsons Bay. Excellent fish and chips. They have been in business since 1885.