Sir William married three wives and had children with all of them. He married firstly Catharine, daughter and coheir to Thomas Sterne (died 1610), of Melbourn, Cambridgeshire. They had three sons and four daughters.
The sons were:
- William, the eldest, died in the West Indies within the life-time of his father;
- Benjamin, the second son, succeeded in the title and estate;
- James, the third son, had his mother's estate, at Melbourn, Cambridgeshire. He married twice and had children with both wives. His first wife was Jane, daughter to Sir William Herris, of Shenfield, in Essex. His second wife was Elizabeth, daughter to Thomas Penyston, of Rochester.
The daughters were:
- Mary, wife of Sir Anthony Thomas, of Cobham, in Surrey;
- Elizabeth, wife of Gervase Lee, of Norwell, in Nottinghamshire;
- Anne, who died without issue;
- Jane, wife of Edward Kighley, of Grays, in Essex.
Sir William married secondly Barbara, daughter and heir of Thomas Sexton. She died and was buried at Hornchurch on 16 December 1617. They had two sons who survived him:
- Thomas, of Gray's Inn, married Elizabeth, daughter of Edward Wentworth, of Bocking in Essex, and was father to Benjamin Ayloffe of Gray's Inn who married Victoria, daughter to Alexander Erskine, son to John, Earl of Mar, in Scotland.
- John, second son, was a Colonel in the West Indies, and died, leaving no children.
Sir William married thirdly Alice, daughter of James Stokes of Stoke near Coventry on 4 December 1624. She died and was buried on 17 April 1652 at Hornchurch. They had a son Joseph, who was also of Gray's Inn, and married Frances, daughter of Henry Ayscough, of Yorkshire and left children, first, Joseph Ayloffe, of Gray's Inn, Barrister at Law; second, William; and five daughters, Alice, Barbara, Anne, Margaret, and Mary.
