Born the son of a counselor-at-law and judge in Germany, Follen was in 1818 awarded a Doctor of Civil Law degree and was appointed to a lectureship on jurisprudence at the University of Giessen in Germany.

He was suspected, though never convicted, of intrigue when a fanatical friend assassinated a Russophile diplomat. Facing possible imprisonment, he escaped to Switzerland before fleeing to the United States in 1824. Letters of introduction by General Lafayette helped his acceptance in Boston.

In 1828 he married Eliza Lee Cabot, daughter of Samuel and Sarah Cabot of Boston. In 1829 Follen taught German at Harvard and wrote a German Reader for Beginners and a German Grammar. He introduced the first gymnasium equipment to Harvard University. He was appointed as a professor in 1830, but he was not reappointed in 1835, partly due to his antislavery agitation.

He introduced a decorated Christmas tree—a custom now widespread across the continent.

In 1836, with the assistance of William Ellery Channing, he was ordained a Unitarian minister and served a new congregation in East Lexington, whose octagonal church building designed by him still stands and is known as the Follen Church.

On return from delivering a lecture in New York, he embarked on the steamer Lexington, which took fire while at sea. Follen, along with other passengers, perished in the tragedy.

A five-volume set of his papers and his biography was prepared by his wife.