Holley was born in Urbana, Illinois. After graduating in chemistry from Illinois University in 1942, he joined the team at Cornell Medical School that achieved the first artificial synthesis of penicillin. He remained at Cornell to receive his PhD in organic chemistry in 1947.

Two years (1955–56) spent at the California Institute of Technology marked the beginning of Holley's important research on the nucleic acids. He decided that to work out the structure of a nucleic acid he first needed a very pure specimen of the molecule. Back again at Cornell, his research team spent three years isolating one gram of alanine transfer RNA (alanine tRNA) from some 90 kilograms of yeast. In March 1965 he was able to announce that they had worked out the complete sequence of 77 nucleotides in alanine tRNA. For this work Holley received the 1968 Nobel Prize for physiology or medicine, an award he shared with Marshall Nirenberg and Har Gobind Khorana.