Colby was the son of John Colby of Beccles and his wife, Ursula née Rede, daughter of Edward Rede of Norwich. Ursula had been widowed three times before: by Thomas Garneys of Beccles, by Thomas Browne of Attleborough, Norfolk and by Sir John Brende of Beccles. Thomas Colby married Beatrice Felton, daughter of Thomas Felton of Playford, Suffolk. They had six sons and three daughters.
