Mohs was born at Gernrode in Saxony and studied at Halle and at the Freiberg Mining Academy under Abraham Werner. In 1812 he became curator of the mineral collection at the Johanneum in Graz. He succeeded Werner at Freiberg in 1818 and in 1826 he was appointed professor of mineralogy at Vienna.

In 1812 Mohs introduced the scale of mineral hardness – Mohs scale – named for him. Ten minerals whose hardness is known are ordered on a scale ranging from 1 (talc) to 10 (diamond), the general rule being that a higher number will scratch all lower numbers. The hardness of a mineral is judged by the ease with which its surface is scratched by these minerals whose values are known, and it can be given a numerical value.