Take a ferry ride to Cockatoo Island. Easiest is to catch the ferry from Circular Quay or Barangaroo Ferry Wharf. You can use an Opal Card or debit/credit card to access Sydney ferries.
Cockatoo Island has a lot of history. There are plaques to read all over the island! (but not so many cockatoos anymore...) These days it is home to many events, formally Underbelly Arts Festival and The Sydney Biennale.
Self-guided audio tours of Cockatoo Island are available from the Information Centre. Card payments only accepted.
I hope you enjoy your visit to this unique place.
