The son of a farmer from Lenthe in Germany, Siemens was the eldest of fourteen highly talented children, who included William Siemens. In 1834 he joined the Prussian army and spent three years in Berlin receiving a thorough training in science and mathematics. Afterward duties were so light as to allow him to pursue his growing interest in chemistry and electricity. From this work he derived his first invention, a new system of electroplating sold by his brother William in London for £1600 in 1843.

In 1847 Siemens founded, with Johann Halske (1814–90), the firm of Siemens and Halske, later to become one of the major industrial concerns in Europe. Initially Siemens hoped to move into the rapidly growing telegraphy business. In 1847 he built the Berlin–Frankfurt line, insulating the underground cables with the newly introduced gutta-percha. Unfortunately for Siemens the gutta-percha had been vulcanized and the copper wire and sulfur destroyed the insulation. Contracts were canceled and, for a time, Siemens found it necessary to work outside Germany. The period, nonetheless, gave Siemens time to experiment, refine, and improve on the basic principles of telegraphy. He was consequently selected to construct the telegraph line connecting London to Calcutta, a distance of 11,000 kilometers, which was completed in 1870.

Siemens's other main interest lay in power generation. The early electric generators were cumbersome machines using large steel permanent magnets and delivering very little power. In 1867 Siemens proposed to replace them with the self-activating dynamo. The permanent magnets were replaced by electromagnets and these were fed by a current obtained from an armature and commutator. Once the dynamo had been perfected it made possible the many manifestations of electric power – lighting, both domestic and public, transport, heating, cooling, and so on. The Siemens companies were well placed to take advantage of the commercial and industrial revolution created by the dynamo. Perhaps the clearest measure of the achievements of Siemens could once have been found in Berlin, where the suburb in which the firm's factories were located and where 120,000 men were employed was named Siemensstadt.