In 1854, the earliest portion of the house, a prefabricated timber structure imported from Hamburg (present-day Germany), was constructed on the site, for the Geelong-based businessman, Charles Ibbotson.[3] Containing 14 rooms, it was modified and extended during the mid-1850s up until the 1970s, with alterations including the addition of servants' rooms in 1856, a billiard room in the 1870s, and external alterations by the architect Harold Bartlett around 1938-1939.[1]
Charles Ibbotson died at the house on 20 October 1883, aged 70. Charles' wife, Maryanne, died the previous year, aged 69. Ownership of the house passed to their daughter Minna.[4] Minna married amateur tennis player Louis Australia Whyte in 1889.
On 3 April, Louis Whyte, who had been suffering from insomnia and neurasthenia and subsequently depression, killed himself by shooting himself in the head.[5][6] Upon Minna's death in 1938, ownership passed on to Louis Melville Whyte, Minna and Louis' son.[7]
Both Louis Melville Whyte and his father Louis Australia Whyte studied at Geelong College, with Louis Melville Whyte studying from 1900–1908. In 1945, he was approached by the school for the purchase of 15 acres of land to the north of the house. In 1946, a further 19.2 acres were purchased, with Louis donating 15 acres of river-facing land. This land would make up the present-day Preparatory School of The Geelong College.[7] Louis Melville Whyte liked to collect cars, and a 1957 Daimler is parked at the property.[3]
As he and his wife Ella 'Marnie' Layton had no children, the house was bequeathed to the National Trust in 1975 upon his death.[8]
Within the grounds of the property are several other buildings. The original timber stables were constructed in 1855, composed of gabled roofs and dormer windows, and were likely designed by the architects Prowse and Snell. The adjacent groom's cottage, of ashlar bluestone, was built around 1856–1857. The property also contains a unique water tower, built around 1860, with a timber lookout tower sitting atop. The bluestone stables and dovecote were constructed in the 1860s, to the designs of Geelong architect Joseph Shaw.[2]
Since the acquisition of the property by the National Trust in 1975, the house has been opened to the public for guided tours, run by volunteers.[1]
Since 2022, The Heights has been the venue for the annual "Le Festival Français" (English: The French Festival), a festival containing a market with a variety of stalls, food court, and live music, which is typically held across a weekend in October. It is jointly organised by Alliance Française de Geelong and the National Trust of Australia (Victoria).[9]