Milne, who was born in Liverpool, was educated at King's College, London, and at the Royal School of Mines. After fieldwork in Newfoundland and Labrador (1872–74) he was appointed, in 1875, professor of geology and mining at the Imperial College of Engineering, Tokyo.

Milne developed a passion for seismology and became known as ‘Earthquake Milne’ in Tokyo. He was instrumental in forming the Seismological Society of Japan in 1880. His first priority was to organize the recording, collecting, and distribution of data. He asked the postal authority in each town throughout Japan to return to him a weekly record of the numbers of earthquakes experienced and he also set up over 900 stations for more detailed recording of seismic activity. Milne also invented, in 1880, a seismograph and he spent much time devising simple and hardy seismographs, which could be used by the relatively unskilled in a wide variety of conditions.

Milne returned to England in 1894 and made his home at Shide, on the Isle of Wight. This became the center of an international system for the collection and distribution of seismological data. His publications included Earthquakes (1883) and Seismology (1898).