Kjeldahl, the son of a physician, was born at Jagerpris in Denmark and educated at the Roskilde Gymnasium and the Technical University of Denmark, Copenhagen. After working briefly at the Royal Veterinary and Agricultural University he joined the laboratory set up by the brewer Carl Jacobsen in 1876 to introduce scientific methods into his Carlsberg brewery founded the previous year. Kjeldahl directed the chemistry department of the laboratory from 1876 until his fatal heart attack in 1900.

Kjeldahl is still widely known to chemists for the method named for him, first described in 1883, for the estimation of the nitrogen content of compounds. It was much quicker, more accurate, and capable of being operated on a larger scale than the earlier combustion-tube method dating back to Jean Dumas. It utilized the fact that the nitrogen in a nitrogenous organic compound heated with concentrated sulfuric acid will be converted into ammonium sulfate. The ammonia can then be released by introducing an alkaline solution, and then distilled into a standard acid, its amount being determined by titration.

His name is also remembered with the Kjeldahl flask, the round-bottomed long-necked flask used by him in the operation of his method.