Panoche Hills is an interesting piece of BLM land! http://www.blm.gov/ca/st/en/fo/hollister/recreation/panoche.html When I visited the last week of October 2013, the gates had just been opened for the winter season to vehicles and there were at least 10 vehicles out here doing target shooting inthe hills. With DRY roads, during the winter months, this was an easy 2WD drive (take the main road P1 to P2, then look for a ridgetop left on P8).
Panoche Hills is CLOSED to vehicles, without a permit, from May to October. Call the Hollister BLM office to apply for a permit 1-831-630-5000. If you get a permit, I believe you'll only be able to drive about half-way to the cache (the observation point), but I'm not sure.
...or, you can leave your car down at the paved road any time of year and hike or bicycle up here. Bring water and tell someone where you're going. There's no phone signal out here!