Gibbs came from an academic family in New Haven, Connecticut. He entered Yale in 1854, graduated in 1858, and in 1863 received a PhD for research on the design of gears. The same year he traveled to Europe, returning in 1869 to Yale where he remained until his death. In 1871 he was appointed professor of mathematical physics.

His initial work on the theory of James Watt's steam-engine governor led him into a study of the thermodynamics of chemical systems. In a series of long papers published between 1873 and 1876 he developed, and indeed virtually completed, the theory of chemical thermodynamics. Gibbs's most famous paper, On the Equilibrium of Heterogeneous Substances (1876), contains the celebrated Gibbs phase rule, describing the equilibrium of heterogeneous systems. His name is also associated with the Gibbs free energy – a function that determines the conditions in which a chemical reaction will occur – and with several other equations in thermodynamics.

Gibbs was also active in mathematics and physics. He worked on the theory of William Hamilton's quaternions and introduced the simpler, widely used, vector notation. Between 1882 and 1889 he published a series of papers on the electromagnetic theory of light. He also made important contributions to statistical mechanics, introducing the fundamental concept of Gibbsian ensembles – collections of large numbers of macroscopic systems with the same thermodynamic properties, used in relating thermodynamic properties to statistical properties.

Gibbs, who never married, lived a quiet retiring life at Yale; he was a poor teacher but a brilliant and productive theorist. His work, carried out far from the European mainstream of science, was largely published in the obscure Transactions of the Connecticut Academy of Sciences. However, James Clerk Maxwell understood the importance of his ideas as early as 1875 and in later life Gibbs was widely recognized. Many regard him as the greatest native-born American scientist.