The closest parking is at gate ES05 by permit only and the description in "ES05 El Sereno OSP Secret Trailhead" (GC250NA) details how to get a permit.
While I was hiking back from "A View for Shamar" (GCZCDJ) I noticed a hill and asked myself if there was a way up to that hill when I spotted what I was looking for, coming across the remnants of a shed, which enticed me further, and crawling along a game trail through the brush, to find an opening with a nice vista of Silicon Valley from the top of this hill.
This should be enough for you to find your way. Don't give away anymore than that. Good luck.
BTW, I didn't see any poison oak on the route I took, but there's plenty all around if you take a different way.