Gram graduated in medicine from the university in his native city of Copenhagen in 1878 and from 1883 to 1885 traveled in Europe, studying pharmacology and bacteriology. While in Berlin (1884) he discovered the method of staining bacteria with which his name has become associated. He followed the method of Paul Ehrlich, using aniline-water and gentian violet solution. After further treatment with Lugol's solution (iodine in aqueous potassium iodide) and ethanol he found that some bacteria (such as pneumococcus) retained the stain (Gram positive) while others did not (Gram negative). This discovery is of great use in the identification and classification of bacteria. It is also useful in deciding the treatment of bacterial diseases, since penicillin is active only against Gram-positive bacteria; the cell walls of Gram-negative bacteria will not take up either penicillin or Gram's stain.

In 1891 Gram became professor of pharmacology at the University of Copenhagen, where he showed a keen interest in the clinical education of the students. During this time he had a large medical practice in the city. He was chairman of the Pharmacopoeia Commission from 1901 to 1921 and director of the medical department of Frederick's Hospital, Copenhagen, until he retired in 1923.