Compton came from a distinguished intellectual family in Wooster, Ohio. His father, Elias, was a professor of philosophy at Wooster College while his brother, Karl, also a physicist, became president of the Massachusetts Institute of Technology. He was educated at Wooster College and at Princeton, where he obtained his PhD in 1916. He began his career by teaching at the University of Minnesota and, after two years with the Westinghouse Corporation in Pittsburgh, he returned to academic life when in 1920 he was appointed professor of physics at Washington University, St. Louis, Missouri. The main part of his career however was spent at the University of Chicago where he served as professor of physics from 1923 to 1945. Compton then returned to the University of Washington first as chancellor and then (1953–61) as professor of natural philosophy.

Compton is best remembered for the discovery and explanation in 1923 of the effect named for him for which, in 1927, he shared the Nobel Prize for physics with Charles T. R. Wilson. He was investigating the scattering of x-rays by light elements such as carbon, and found that some of the scattered radiation had an increased wavelength, an increase that varied with the angle of scattering. According to classical physics there should be no such change, for it is difficult to see how the scattering of a wave can increase its wavelength, and Compton was led to seek its explanation elsewhere.

He thus assumed that the x-rays also exhibited particle-like behavior. Hence they could collide with an electron, being scattered and losing some of their energy in the process. This would lead to a lowering of the frequency with a corresponding increase in the wavelength. Compton went on to work out the formula that would predict the change of wavelength produced in the secondary x-rays and found that his precise predictions were fully confirmed by measurements made of cloud–chamber tracks by Wilson. Significantly, this was to provide the first hard experimental evidence for the dual nature of electromagnetic radiation; that is, that it could behave both as a wave and a particle. This would be developed much further in the 1920s as one of the cornerstones of the new quantum physics.

In the 1930s Compton concentrated on a major investigation into the nature of cosmic rays. The crucial issue, following the work of Robert Millikan, was whether or not a variation in the distribution of cosmic rays with latitude could be detected. Such an effect would show that the rays were charged particles, deflected by the Earth's magnetic field, and not electromagnetic radiation. As a result of much travel and the organization of a considerable amount of the research and measurements of others Compton was by 1938 able to establish conclusively that there was a clearly marked latitude effect.

During the war Compton was an important figure in the manufacture of the atomic bomb as a member of the committee directing research on the Manhattan project. He also set up at Chicago the Metallurgical Laboratory, which acted as a cover for the construction of the first atomic pile under the direction of Enrico Fermi and took responsibility for the production of plutonium. Compton later wrote a full account of this work in his book Atomic Quest (1956).