Smith was born in Churchill, England, the son of the village blacksmith. He was educated at local schools and in 1787 began work with the surveyor who had been commissioned to make a survey of his parish. This was at the time of the great canal boom and Smith soon found himself fully employed conducting surveys for proposed canals. He was also regularly employed to report on coal deposits in different parts of the country while making his canal surveys. During this work he traveled considerable distances throughout Britain.

From his surveying and observation of rock strata Smith formulated very clearly two basic principles of geology for which he is often known as the father of British geology. As early as 1791 he had noted that certain strata, wherever they occurred, all contained the same invertebrate fossils and that different strata could be distinguished by a difference in their fossil content. Previously geologists had relied upon the nature of the rocks to identify and discriminate between strata. This could work well in some favored cases but, in general, was far from reliable. Smith's other major theory was the law of superposition, which simply states that if one layer of sedimentary rocks overlays another then it was formed later, unless it can be shown that the beds have been inverted.

Smith never managed to produce a major book on the geology of Britain although he did publish two small pamphlets: Strata Identified by Organized Fossils (1816) and Stratigraphical Systems of Organized Fossils (1819). His major productions were instead in the field of maps. In 1815 he produced the first geological map of England and Wales. This was published in 15 sheets at a scale of 5 miles to the inch. During the period 1819–24 he published a series of geological maps of 21 counties. The plates were still being used to produce maps as late as 1911.

Smith seems to have received little recognition and reward during his early life; he was forced to sell his collection of fossils to the British Museum to overcome his financial difficulties. However, in the 1830s he began to receive the recognition he deserved. In 1831 the Geological Society of London awarded him their first Wollaston medal and in 1835 he received an honorary LLD from Dublin.