Definitely worth the hike. I felt that I had not found solitude from other hikers before this hike, but going up to Cliff Lake after Labor Day weekend provided it in spades. It is a beautifully secluded spot with tons of rock to scramble around on, so if you have time, take 3 days to explore the whole area. Given the orientation of the valley it gets a bit cold pretty early in the fall, so be prepared for a good chill. Also, campfires are allowed with a permit! Woo!
Road in takes a bit of slow going, but the trail is really well-maintained, albeit covered in cow pies. Give it a go.