Born in The Hague in the Netherlands, Goudsmit was educated at the universities of Amsterdam and Leiden, where he obtained his PhD in 1927. He emigrated to America shortly afterward, serving as professor of physics at the University of Michigan (1932–46) and North Western (1946–48). He then moved to the Brookhaven National Laboratory on Long Island, New York, where he remained until his retirement in 1970.

In 1925 Goudsmit, in collaboration with the Dutch–American physicist George Uhlenbeck (1900–1988), put forward the proposal of electron spin. They suggested that electrons rotate about an axis and, as they are charged, set up a magnetic field. This model was successful in clearing up a number of anomalies that were becoming apparent in the fine structure of atomic spectra. A theory of spin was later given by Paul Dirac.

During World War II Goudsmit worked on radar and then became head of a top secret mission codenamed Alsos in 1944. The mission was for Goudsmit to follow the front-line Allied troops in Europe, and even in some cases to precede them, looking for any evidence of German progress in the manufacture of an atomic bomb. He found that the German scientists had, in fact, made little progress and it was clear that Hitler would not be presented with such a weapon before the end of the war. For this war service Goudsmit was awarded the Medal of Freedom from the US Department of Defense and he published his account of the mission in his book Alsos (1947).