Born in Oak Park, Illinois, Schrieffer was educated at the Massachusetts Institute of Technology and the University of Illinois, where he obtained his PhD in 1957. After serving as a postdoctoral fellow in Europe at Birmingham and Copenhagen he worked at the University of Illinois from 1959 until 1962 when he moved to the University of Pennsylvania, Philadelphia, being appointed professor of physics there in 1964. He was professor of physics at the University of California, Santa Barbara (1980–91), moving to the Florida State University, Tallahassee, in 1992.

Schrieffer worked on superconductivity. In 1972 he was awarded the Nobel Prize for physics with John Bardeen and Leon N. Cooper for their formulation in 1957 of the first successful theory of superconductivity, the BCS theory.