Backus was born in Philadelphia. After graduating from Columbia University, New York, he joined the staff of IBM in 1952 and remained with them until his retirement in 1991. From 1959 until 1963 he worked at the IBM Research Center, Yorktown Heights, New York, and thereafter as an IBM Fellow at the IBM Research Laboratory, San Jose, California.

Backus has reported on the state of programming when he joined IBM. It was, he noted, “a black art, a private arcane matter.” All programming was done using machine or assembly language. There were no compilers, no index registers, and the programmer spent most of his time debugging the program and feeding it into the computer. The programmers actually cost more than the computer. Backus commented, “They dismissed as foolish plans to make programming accessible to a larger population,” it was inconceivable “that any mechanical process could possibly perform the mysterious feats of invention required to write an efficient program.”

In 1954 Backus led an IBM team determined to free computer programming from the professional élite. As the speed of computers increased it made no sense to have them standing idle while a programmer struggled to operate them. The problem was made more pressing by the development of the new and more powerful IBM 704. By late 1954 some of the main details of the high level language fortran (from Formula Translation) had been established. Backus defined his aim as “to design a language which would make it possible for engineers and scientists to write programs for the 704.” The language itself was available in 1957 and soon became the most widely used programming language.