Yes this is one of our favorite places on earth! We started bringing the kids up here in 1989 and DH has been coming up since he was a little tike. Silver Lake is in the June Lake Loop which is just north of Mammoth.
We have stayed in the cabins, rented a trailer that met us at the trailer park, camped in the campground, we have done it all! The campground is state run, and you can camp right next to Rush Creek, which runs from Silver Lake to Grant Lake. Great fishing is to be had there.
Here are some pics of my kids over the years:
here. This year it is just the 3 of us, this is the only request DD had before going off to college. DS has to work and isn't able to come. We leave next week and will be gone for 10 days. I can't wait. If you ever get the chance to visit or stay at Silver it will be a memorable time!