The justices did not comment on their order, which leaves in place the earlier ruling by a panel of three federal judges requiring California to reduce its prison population to improve medical and mental health treatment.
The case now goes back to the three judges, who repeatedly have ruled against Brown and at one point threatened to hold him in contempt if he didn't adhere to their order.
It's the second time Brown has been rebuffed by the nation's highest court. In 2011, Supreme Court justices ruled that the lower court panel had the authority to order California to reduce inmate overcrowding as the key condition for improving conditions.
At the heart of the case is a 2001 lawsuit filed on behalf of inmates who claimed medical treatment in the prisons was so poor it was leading to a death a week through neglect or malpractice. The federal courts agreed, saying conditions violated inmates' constitutional rights against cruel and unusual punishment.
In addition to spending billions of dollars on new medical facilities and staff, the state was ordered to reduce overcrowding, which was seen as a main obstacle to providing better health care. The state has also been forced to take similar steps to improve inmate mental health treatment, the subject of a separate lawsuit.
Deborah Hoffman, spokeswoman for the California Department of Corrections and Rehabilitation, expressed disappointment with the high court and said the state will continue working to improve prison conditions. She noted that in the past two years, the prison population has been reduced by 25,000 inmates.
Brown signed a bill in September allowing the state to spend $315 million this fiscal year to house thousands of inmates in private prisons and county jails if the lower court doesn't postpone its January deadline for reducing the prison population. If the court agrees to a three-year delay called for in the legislation, the state must spend part of the money on rehabilitation programs intended to reduce the inmate count.
Last month, the court ordered the state to negotiate with inmates' attorneys toward a possible compromise. But the attorneys say there have been no face-to-face talks with administration officials, though a court-appointed mediator has been active. The judges ordered the mediator to update them next Monday on the progress of negotiations.
"Now that the court has ruled and the governor's out of legal options, it would be in their interest to try and resolve this in a way that is effective and productive," said Don Specter, director of the Prison Law Office that filed the lawsuit over prison medical care.
While the issue plays out in court, the state has been signing contracts with private prison operators to house inmates.
The Corrections Department and Corrections Corp. of America announced Tuesday that the state had signed a $28.5 million annual contract to lease the private California City Correctional Center, which will hold 2,381 inmates. The lease is for three years, with unlimited two-year renewals, and the annual cost will rise with inflation after the initial three-year contract expires.
Last month, the state signed a five-year, $30 million annual contract with Geo Group to lease two prison facilities in Kern and San Bernardino counties. The two facilities can hold 1,400 inmates.
The steps leave the state about 4,400 inmates above the population cap set by the courts. The state must reduce the population of its major prisons to about 110,000 inmates by the end of January under the current lower court order.