Arnold was one of three children born on a farm in Cranston, Rhode Island, to James Lincoln Arnold and Amey Underwood. From the time he was a child, he suffered from a form of lameness that required him to use crutches later in life. As a child he became aware of his descent from early Rhode Island settlers, and his interest in documenting that descent ensued. As a young adult he lived with his parents, and moved with them to North Kingstown, Rhode Island, in 1869, settling near the Gilbert Stuart birthplace. It was then that he began transcribing the inscriptions from tombstones, while also collecting local family lore and accounts of local Indians. He continued to live with his parents, and in 1880 was living with them and his sister, Amey Franklin, and her daughter, also named Amey Franklin. Following years of collecting records, he became the editor of the Narraganset Historical Register in 1882. It was also about this time that he embarked on his most ambitious project, which was to compile all of the known vital records of Rhode Island. He collected vital records from city and town archives, from churches, and from newspapers. By the time he finished, he had filled 21 volumes with these records.
Arnold eventually left his parents’ home in 1884, and moved to Providence. Despite his disability, he traveled all over the state in pursuit of genealogical records, and laboriously transcribed these records by hand. This occupation led to his being called Rhode Island's "unofficial statistician of the dead". He was also called "Old Mortality" after the character in Sir Walter Scott's novel of the same name. By the time of his death, he had assembled a library of over 4,000 genealogical and historical works, and nearly 10,000 pamphlets. Following his death in 1927, this collection was given to the Knight Memorial Library in the Elmwood section of Providence.
