On a nice day this can be a high muggle area, so use stealth and pretend you are just a sightseeing tourist.
From the bridges you get great views of the dam and waterfall upstream and rapids downstream, and you can access the riverbed for rockhopping and exploring. This is a popular picture-taking and fishing spot. Be careful lest the cache get muggled. There is a public access area nearby where you can park and swim or launch a car-top boat for paddling on the pond behind the dam.
I travel frequently travel through Forestport on my way between the Black River Valley and Adirondacks, and until a few years ago always came 'the back way' from Boonville and drove across the 110 year old one lane bridges and through Forestport village rather than across the modern bridge on route 28.
Unfortunately the ravages of time and the elements took a toll on the old bridges. They were closed to vehiclular traffic for several years, and were replaced in the summer of 2013
The bridge beams and other laterals for the old bridges were brought in by canal boat on the Black River Canal and the feeder canal between Boonville and Forestport. Bridge history, information, and pictures are available at forestporttwinbridges.com.
Although the replacement bridges are just nondescript modern structures, the location is still picturesque, and the up side is that local traffic no longer has to detour on to the state highway which can be really busy at times.
Black River Canal history at blackrivercanalmuseum.com. The museum in Boonville is very interesting. Visit it if you are geocaching in the area.