Born at Gorizia, Trieste, Rubbia was educated at the University of Pisa, where he obtained his PhD in 1958. After spending a year each at Columbia, New York, and Rome, he took up an appointment in 1960 at the European Laboratory for Particle Physics (CERN), Geneva, becoming its director-general in 1989. He has also held since 1972 a professorship of physics at Harvard.

Rubbia is noted for his work in high-energy physics using the considerable accelerator capacity of CERN. He set himself an ambitious target in the mid 1970s, namely, the discovery of the intermediate vector bosons. Forces operate by interchanging particles. Thus the electromagnetic force works by exchange of virtual photons. The weak interaction would, therefore, require a comparable particle; in actual fact three such particles would be needed, W⁺, W^–, and Z⁰. Further, as the weak force acts at distances below about 10^–13 centimeters, and as the shorter the distance the larger the particle would have to be, the bosons would have to be massive, some 80 times bigger than a proton.

To produce such particles in an accelerator requires enormous energies and it was not expected that CERN would be able to obtain such energies for more than a decade. Rubbia proposed in 1976 that the existing super proton synchroton should be changed from a fixed-target accelerator to one producing collisions between beams of protons and antiprotons traveling in opposite directions. If feasible, and given that a particle's kinetic energy increases as the square of its velocity, much higher energies would be attained. As redesigned by his CERN colleague, Simon van der Meer, the SPS produced energies of 540 billion electronvolts (540 GeV) – the equivalent of the 155,000 GeV achieved by striking a stationary target.

Rubbia faced two further problems: how to produce enough antiprotons, and how to recognize the W and Z particles. Antiprotons were produced by accelerating protons in the SPS and firing them at a metal target. A new detector, designed by Charpak, was built. To detect a W particle the experimenters looked for its characteristic interactions. They should see antiprotons collide with protons and emit a W particle, which in 10^–20 second should decay into an electron and a neutrino. The experiment began in September 1982 and ran until December 6, leaving millions of collisions to analyze. Among them they found six possible W particles, five of which were accepted as genuine. The Z particle was subsequently discovered in May 1983.

Rubbia published the discovery of the W particle in January 1983 in a paper listing 130 coauthors. For their part in the discovery of the W and Z particles Rubbia and van der Meer shared the 1984 Nobel Prize for physics.