In April 1837 Kelly was baptized and joined a missionary church in Columbia. He was born to the Catholicism of his father, but was convinced by the Baptist teachings. On May 19, 1842, he was licensed to preach[2] from the Mission Baptist Church at Columbia,[1] and on October 1 of that year he was ordained by Rev. R. B. C. Harell and the First Baptist Church of Nashville to be an evangelist. His first posting was at the Mt. Lebanon Baptist Church in 1843, which at that time had only six members.[2] This posting made him the first black Baptist preacher ordained in Tennessee.[5] He is also credited for organizing the First Negro Baptist Church in Columbia in 1843.[1]
Around this time, he escaped from slavery on the underground railroad to Massachusetts.[6] He then purchased the freedom of his wife and four children for twenty-eight hundred dollars.[2] For this purpose, he collected money in New England and in England. While in England, it was recommended to him that he purchase not only his family's freedom, but also his own, so that he would not be captured under the Fugitive Slave Act.[7] His children included J. H. Kelly, a teacher in Columbia and W. D. Kelly who was a member of the 54th Massachusetts Infantry Regiment in the US Civil War.[2] In 1848, he organized the 12th Baptist Church in Boston, Massachusetts.[1]
Free, Kelly moved to New Bedford, Massachusetts, where he was a prominent leader in the local church and the national American Baptist Conventions.[8] In 1845 along with Thomas T. Allen he became leader of the newly founded Second Baptist Church of New Bedford, Massachusetts which was dedicated in January of that year. Allen served as the head pastor for three years, followed by William Jackson and then Cummings Bray. In 1855, Jackson returned to lead the church, and in 1858 left to form the Salem Baptist Church. Kelly again took charge, but only for a short time.[9] In 1857, Kelly was a preacher in Philadelphia. That year, he preached at the American Convention of Colored Baptists in Boston[10] and served as its president.[11] He preached at the 1864 convention as well. That year, he was still preaching in New Bedford.[12] On May 10, 1864, he founded the Shiloh Baptist Church in Newport, Rhode Island in a house at 73 Levin Street owned by Esther Brinley. William James Barnett was installed as the first pastor, followed shortly by Theodore Valentine.[1]
After the civil war, Kelly returned to the south. He first moved to Arlington, Virginia where, in 1866, he organized the Zion Baptist Church.[1] He then moved back to Columbia, Tennessee where he preached and taught.[6] Kelly, in 1869, was assaulted in Columbia in a political dispute.[13] In 1870 he organized a school for blacks at D. T. Chappell's, who supported the endeavor.[14] He did not receive regular pay for his work. Among Kelly's students was a young nephew of his, who was the father of Lyman T. Johnson.[6]
After about six years in the South, Kelly faced a dispute in the church over the use of alcohol, and in 1873 he returned to New England.[15] He moved to Portsmouth, New Hampshire and organized the Mount Olivet Baptist Church.[16] Around that time, he also organized three other churches, Calvary Baptist Church in Haverhill, Massachusetts in 1873; Calvary Baptist Church in Hartford, Connecticut in 1874, and the Myrtle Baptist Church in West Newton, Massachusetts in 1874.[1]
Kelly attended the 1876 Conference of Baptist Ministers in Philadelphia.[17]
Kelly wrote numerous pamphlets, which were praised by Bishop Daniel Payne.[2] By 1887, he had returned to New Bedford, Massachusetts, where he continued to teach and preach, although he had become aged and blind.[2]
