Gmelin, whose father and grandfather were botanists, was born at Göttingen (in Germany) and studied at the universities of Tübingen, Göttingen, and Vienna. In 1817 he was appointed to the first chair of chemistry at Heidelberg, where he remained until 1851. In 1817 he published the first edition of what was to become the major chemical textbook of the first half of the 19th century, Handbuch der Chemie (Handbook of Chemistry), in three volumes. By 1843 the book was in its fourth edition and had been expanded to nine volumes. In this edition Gmelin adopted the atomic theory and devoted much more space to the growing discipline of organic chemistry. The terms ester and ketone were introduced by him. His book was translated into English in 1848.

He also worked on the chemistry of digestion, discovering several of the constituents of bile, and introduced Gmelin's test for bile pigments. In 1822 he discovered potassium ferrocyanide.