Chargaff, who was born at Czernowitz (now Chernovtsy in Ukraine), gained his PhD from the University of Vienna in 1928 and then spent two years at Yale University. He returned to Europe, working first in Berlin and then at the Pasteur Institute, Paris, before returning permanently to America in 1935.

Initially Chargaff's work covered a range of biochemical fields, including lipid metabolism and the process of blood coagulation. Later his attention became concentrated on the DNA molecule, following the announcement in 1944 by Oswald Avery that the factor causing the heritable transformation of bacteria is pure DNA. Chargaff reasoned that, if this were so, there must be many more different types of DNA molecules than people had believed. He examined DNA using the recently developed techniques of paper chromatography and ultraviolet spectroscopy and found the composition of DNA to be constant within a species but to differ widely between species. This led him to conclude that there must be as many different types of DNA as there are different species. However, some interesting and very important consistencies emerged. Firstly the number of purine bases (adenine and guanine) was always equal to the number of pyrimidine bases (cytosine and thymine), and secondly the number of adenine bases is equal to the number of thymine bases and the number of guanine bases equals the number of cytosine bases. This information, announced by Chargaff in 1950, was of crucial importance in constructing the Watson–Crick model of DNA.

From 1935 Chargaff has worked at Columbia University, as professor of biochemistry from 1952 and as emeritus professor from 1974.