Linde, who was born in Berndorf (now in Austria), studied engineering under Rudolf Clausius in Zurich. He taught in Munich at the Polytechnic as assistant professor of machine design from 1868 before moving into the refrigeration business in the 1870s.

His first breakthrough came in 1876 when he produced an ammonia refrigerator. This was a much more efficient cooler than the compression machine introduced by Jacob Perkins in 1834. By 1908 the Linde Company had sold 2600 machines, of which just over half were bought by breweries. Linde also developed an equally successful domestic version.

Linde was also the first industrialist to use the new developments in low-temperature physics. Oxygen had been first liquefied by Raoul Pictet and Louis Cailletet in 1877. In 1895 Linde set up the first large-scale plant for the manufacture of liquid air using the Joule–Kelvin effect. Within six years he also developed a method for separating liquid oxygen from liquid air on a large scale. New industrial processes needed oxygen, and consequently Linde's process was rapidly taken up.