Hess, the son of a forester, was born at Waldstein in Austria and educated at the University of Graz where he obtained his doctorate in 1906. He worked at the Institute for Radium Research, Vienna, from 1910 to 1920 and then took up an appointment at the University of Graz where he became professor in 1925. In 1931 he set up a cosmic-ray observatory near Innsbruck but in 1938 he was dismissed from all his official positions as he was a Roman Catholic. Leaving Nazi Austria, he emigrated to America where he served as professor of physics at Fordham University, New York, from 1938 to 1956.

In 1911–12 Hess made the fundamental discovery of cosmic rays, as they were later called by Robert Millikan in 1925. For this work he shared the Nobel Prize for physics with Carl Anderson in 1936. The work stemmed from an attempt to explain why gases are always slightly ionized; thus a gold-leaf electroscope, however well insulated it might be, will discharge itself over a period of time. Radiation was clearly coming from somewhere and the most likely source was the Earth itself. To test this, attempts were made to see if the rate of discharge decreased with altitude. But both T. Wulf, who took an electroscope to the top of the Eiffel Tower in 1910, and A. Gockel, who took one up in a balloon in 1912, failed to obtain any clear results.

However when Hess ascended in a balloon to a height of 16,000 feet (4880 m) he found that although the electroscope's rate of discharge decreased initially up to about 2000 feet (610 m), thereafter it increased considerably, being four times faster at 16,000 feet than at sea level. He concluded that his results were best explained by the assumption that a radiation of very great penetrating power enters our atmosphere from above.

He was able to eliminate the Sun as the sole cause for he found that the effect was produced both by day and at night. Further, in 1912, he made a balloon ascent during a total eclipse of the Sun and found that during the period when the Sun was completely obscured there was no significant effect on the rate of discharge. Hess however failed to convince everyone that cosmic rays came from outside the Earth's atmosphere as it could still be argued that the source of the radiation was such atmospheric disturbances as thunderstorms. It was left to Millikan in 1925 finally to refute this objection.