Born in Heidelberg, Germany, Ruska was educated at the Munich Technical University and at Berlin University, where he obtained his PhD in 1934. He worked in industry until 1955 when he became professor of electron microscopy at the Haber Institute, Berlin, a post he held until his retirement in 1972.

It had long been known that optical microscopes are limited by the wavelength of light to a magnifying power of about 2000, and the ability to resolve images no closer together than 2–3000 angstroms (1 angstrom = 10^–10 meter). In 1927, however, G. P. Thomson first demonstrated that electrons can behave like waves as well as like particles. The wavelength of the electron depends on its momentum according to de Broglie's equation λ = h/p. The higher the momentum of the electron, the shorter the wavelength. It should be possible to focus short-wavelength electrons and obtain better resolving powers.

In 1928 Ruska attempted to focus an electron beam with an electromagnetic lens. He went on to add a second lens and thus produced the first electron microscope; it had a magnifying power of about seventeen. Improvements, however, came quickly and by 1933 the magnifying power had been increased to 7000. Soon after he joined the firm of Siemens and began to work on the production of commercial models. The first such model appeared on the market in 1939. It had a resolution of about 250–500 angstroms.

For his work in this field Ruska shared the 1986 Nobel Prize for physics with Binnig and Rohrer.