Abinger Manor is a grade II listed manor house located south of the motte.[4] The manor was recorded in the Domesday Book as land belonging to William fitz Ansculf.[5] The house was re-established in 1688 by John Evelyn and was rebuilt in c. 1872 using some old materials following a new design by Alfred Waterhouse.[4]
St James' Church was first established in the 12th century and remains due east of the motte. The earliest extant sections of the church are its chancel and north chapel, rebuilt c. 1220. Bombed during World War II, it was restored in 1950 by Frederick Etchells. The church is a grade II* listed building.[6]
There are remains of a Mesolithic pit-dwelling near the motte, representing one of the oldest known and best preserved man-made dwelling remains in Britain. The site is also notable for containing many examples of microlith assemblages. The site is a scheduled monument.[7]