Krogh, the son of a brewer from Grenaa in Denmark, was educated at the Aarhus gymnasium and the University of Copenhagen, where he obtained his PhD in 1903. He spent his whole career at the university, serving as professor of animal physiology from 1916 until his retirement in 1945.

Krogh first worked on problems of respiration. He argued, against his teacher Christian Bohr, that the absorption of oxygen in the lungs and the elimination of carbon dioxide took place by diffusion alone. He made precise measurements to show that the oxygen pressure was always higher in the air sacs than in the blood and, consequently, there was no need to assume any kind of nervous control.

It was, however, with his studies of the capillary system that Krogh achieved his most dramatic success. The simplest explanation of its action was to assume it was under the direct hydraulic control of the heart and arteries: the stronger the heart beat, the greater the amount of blood flowing through the capillaries. Krogh had little difficulty in showing the inadequacy of such a scheme by demonstrating that even among a group of capillaries fed by the same arteriole some were so narrow that they almost prevented the passage of red cells while others were quite dilated, allowing the free passage of the blood. Not content with this descriptive account Krogh went on to make a more quantitative demonstration. Working with frogs, which he injected with Indian ink shortly before killing, he showed that in sample areas of resting muscle the number of visible (stained) capillaries was about 5 per square millimeter; in stimulated muscle, however, the number was increased to 190 per square millimeter. From this he concluded that there must be a physiological mechanism to control the action of the capillaries in response to the needs of the body. It was for this work, fully described in The Anatomy and Physiology of Capillaries (1922), that Krogh was awarded the 1920 Nobel Prize for physiology or medicine.