Carnwath was first constructed in 1850 by William Henry and Lydia Willis. Willis was a retired hardware merchant from New York City,[1] and related to the Mesier family of nearby Wappingers Falls. The house was originally designed in the Italianate style by Andrew Jackson Downing.
In 1855, the estate was sold to George Barclay, son of the British consul in New York. He was a partner in the commercial firm Barclays and Livingston, the agents for Lloyd's for the City of New York.[2] Barclay died at the estate on 28 July 1869.
Barclay's daughter, Matilda Antonia married New York City attorney Francis Robert Rives,[3] son of diplomat William Cabell Rives of Virginia. Francis Rives was secretary of the American legation at London under ambassador Edward Everett during the William Henry Harrison administration.[4] For a time, the road leading up to the estate was called Rives Avenue (now known as Wheeler Hill Road).
Rives was a member of both the Knickerbocker Club and the Coaching Club and made many modifications to Carnwath Manor and also constructed the carriage house, cow barn and ice house. He died at the age of sixty-nine at Carnwath on July 16, 1891 and is buried in the Wappingers Rural Cemetery. His son Reginald inherited Carnwath. Reginald Rives was an accomplished coachman and vice-president of the Coaching Club.[5][6] He also had a home in Montecito, California, but would return east for club events. In 1894, Rives' coach took first prize at the Dutchess County Fair horse show, competing against that of John Jacob Astor.[7] He was an authority on coach horses and often served as a judge at horse shows. In 1914 he sold Carnwath to Isaac Untermyer, famous for defending Boss Tweed. In 1925, it was sold to the Augustinian Friars and was known as the Novitiate of Our Mother of the Good Counsel. The friary constructed the chapel in 1950.
In 1980 the property was sold to Greystone Programs, Inc. The Town of Wappinger purchased Carnwath Farms Historic Site & Park in 1999, and welcomed its first tenant in 2005, the Sports Museum of Dutchess County.[8] Carnwath Farms Historic Site & Park is located within the Wheeler Hill Historic District but is a separate historic site.
