From an interview published today in The Atlantic.
Stevens said he retired because, while he still loved the job of judging, he had no desire to linger beyond his physical prime. He had witnessed the final years on the bench of [William O.] Douglas, Thurgood Marshall and others who should have retired earlier for health reasons. A few years ago, he secretly asked Associate Justice David Souter to tell him when it was time for him to go. But Souter left first, in 2009.The suggestion, as I read it, is that Stevens had to judge himself strictly because he didn't have Souter to reassure him that the time to go had not yet arrived. (How can you tell if you've lost your mental powers?)
"When he retired, I knew I didn't have any safety valve anymore."
Why Souter was a unique confidante, the interviewer did not ask.