The house was built in 1860 for James Ashworth, who left his native Kentucky to take part in the California Gold Rush.[2] He subsequently settled in San Jose on 250 acres of land, where he became a farmer and lived with his wife and seven children, until his died in 1895.[2][3][4]
The property was purchased in 1891 by Peter Remillard, an immigrant from Canada.[2] Remillard had also participated in the California Gold Rush.[2] By 1861, he founded the Remillard Brick Company. After he purchased Ashworth's house in 1891, he established another brick-manufacturing business on the grounds, with several outbuildings.[2] The bricks were used to build many houses and buildings in San Jose until it closed down in 1968. Peter Remillard remodeled and enlarged the farmhouse while maintaining the Victorian style.[4]
The house was inherited by Countess Lillian Remillard Dandini.[4] However, the countess had spent only her summers as a child in the house, and she resided at Carolands from 1950 to 1973.[2]
In 1968, the house was transferred to Dandini's longtime friend and neighbor, Joseph Covey and his partner, architect, Dick Gilbert.[4] After Gilbert's death in 1997, Covey's health declined and the house was used by transients and homeless people until it was boarded up 10 years later.[4][5] In 2007, Sue Cucuzza, a brick collector, found the house in disrepair.[4] Cucuzza decided to work with Covey, in order to remove the transient people and to raise money to clean, restore and preserve the home.[4][5]The house was owned by Cucuzza as of 2024.[6][7] The records of Remillard Brick Company had been found in the house and were donated to History San Jose Research Library.[8]
