Löffler was born at Frankfurt in Germany, the son of an army surgeon. He was educated at the University of Würzburg and the Berlin Institute of Military Medicine, where – after serving in the Franco–Prussian War – he obtained his MD in 1874. After various official positions Löffler worked with Georg Gaffky as an assistant to Robert Koch from 1884 to 1888. He later served as professor of hygiene at the University of Griefswald (1888–1913), after which he succeeded Gaffky as director of the Koch Institute in Berlin, where he remained until his death.

Löffler's major contribution to the new field of bacteriology was the isolation and cultivation in 1884 of the bacillus responsible for diphtheria, which had first been observed by the German physiologist Theodor Klebs in the throats of diphtheria patients (the organism became known as the Klebs–Löffler bacillus). The isolation and cultivation of pure cultures involved a number of formidable technical problems; Löffler found it necessary to develop a new medium, thickened serum, as the conventional gelatin used by Koch required temperatures far too low for the diphtheria pathogen.

Earlier (in 1882) Löffler had discovered the organism responsible for glanders (a contagious disease, especially of horses) and in 1898, in collaboration with the pathologist Paul Frosch, he succeeded in demonstrating for the first time that viruses could cause diseases in animals. This was achieved by passing foot-and-mouth disease from one cow to another by inoculation with cell-free filtrates taken from lesions. Löffler's work, together with Dmitri Ivanovsky's demonstration in 1892 that plants were susceptible to viral infections, constituted the start of modern virology.