Goldhaber, who was born at Lemberg (now Lvov in Ukraine), was educated at the universities of Berlin and Cambridge, where he obtained his PhD in 1936. He emigrated to America in 1938 where he first taught at the University of Illinois, becoming professor there in 1945. He moved to the Brookhaven National Laboratory in 1950, serving as its director from 1961 until 1973.

In 1934, while at the Cavendish Laboratory of Cambridge University, Goldhaber codiscovered the nuclear photoelectric effect with James Chadwick. This is the disintegration of a nucleus by high-energy x-rays or gamma rays. From this it was later established that the neutron is slightly heavier than the proton. Following Enrico Fermi's discovery of slow neutrons, Chadwick and Goldhaber also discovered (1934–35) the neutron disintegration reactions for lithium, boron, and nitrogen. The nitrogen reaction is the major source of radioactive carbon–14 on Earth.

At the University of Illinois (1938) Goldhaber and his wife, Gertrude Scharff-Goldhaber, demonstrated that electrons and beta particles are the same. In 1940 he discovered that beryllium is a good moderator, i.e., it slows down fast neutrons so that they more readily split uranium atoms.

He also proposed a cosmological theory in which an initial ‘universon’ broke up into a ‘cosmon’ (matter) and an ‘anticosmon’ (antimatter), with the anticosmon forming a second universe made of antimatter.