He was the son of William Connelly (1807–1890) and Margaret Ann (Terpenning) Connelly (1810–1882). The family removed to Esopus when Henry was still an infant. He attended the common schools, and for one year Charlotteville Seminary in Summit. Then he became a clerk in a store owned by his father and a cousin of his in Rondout. On April 12, 1854, he married Cornelia A. Aldrich (died 1857). In 1856, he became a partner in the store, and later engaged extensively in the manufacture of Rosendale cement. In 1858, he married Lucinda Manning, and they had nine children. He was also President of the Kingston Savings Bank and the Kingston National Bank.
He was Supervisor of the Town of Esopus from 1867 to 1870, and a member of the New York State Senate (14th D.) in 1874 and 1875; and again in 1886 and 1887.
Chief Justice Charles Evans Hughes was his nephew.