Drummond appears to have had at least five families.[5] It is unclear if any were part of a legal marriage and he had multiple illegitimate children. His marriages were:
- Name unknown, possibly a woman from the Westmoreland area
- Mary Drummond, a negro on Jamaica originally named Juno (met before 1775)
- A negro woman on Jamaica (met before 1796)
- Eleanor (met before 1797)
- Kitty (met before 1798)
He had at least four mulatto children by Mary Drummond, all of whom were given very British names: Thomas Drummond (1775–1847), Esther (or Hester) Drummond (died 1796), Adair Drummond and John Drummond. All these children were given full rights and privileges in his will of 1793. These were all children of Mary Drummond a "free negro" whom had originally been a slave on his plantation. In his will he bequeathed Mary seven slaves, and on her death these (and their children) were to pass to Thomas Drummond "his reputed son" by Mary Drummond. Mary also inherited the estate, house (Drummond Lodge) and cows and horses and £50 per annum. Thomas, their eldest son was given 20 acres of land in Cornwall, 6 cows and £500 to buy negroes to operate this land. Their three other children received £50 per annum each. All surplus estate was also left to Thomas. The same will left Peggy Bartlett, a free quadroon woman of Jamaica, 5 acres of land on his Westmoreland estate on the west side of the island, and a negro slave to be purchased for her. Peggy appears to have either been a further daughter or a mistress given his generosity.
The will also allows for the freedom of his personal slave Hannah Browning covering all legal expenses for this.
More curiously, two of Mary Drummond's slaves, William the son of Eleanor, and Maria the daughter of Kitty, are acknowledged by John Drummond as his own children, and the will asks that these two children also be given their freedom, plus £25 each per annum. William Drummond and Maria Drummond were both returned to England in 1807.
Elizabeth Wedderburn, daughter of John Wedderburn in London was left the very substantial sum of £500, again indicating a relationship of some kind. John Wedderburn of Wedderburn & Co appears to have been his solicitor.
William Murray, his assistant as a surgeon, was left all his books and surgical instruments.
Three further female names, Alice Drummond, Kathleen Ellis and Ann Ellis are also mentioned in the will. Whilst there is suggestion that these are his aunt and two cousins their ages belie this. The Ellis girls appear to be as daughters of Thomas Drummond his son and a Katherine Ellis of Carriden in Scotland (born 1756), John Drummond's first cousin (i.e. they appear to be illegitimate granddaughters). The will provides that if this portion is not used the funds be instead passed to the Royal College of Surgeons of Edinburgh. Alice Drummond (whether daughter of John or Thomas) appears to be under 16 (and therefore not John's aunt, also Alice Drummond) and her guardianship is given to Rev Charles Mitchell of Jamaica (indicating that she was certainly not the daughter of Mary Drummond). Alice was living in Great Britain from 1802. It is unclear if Alice is the same person as "Alicia Drummond, daughter of the late Dr John Drummond of Jamaica" who married Walter Mitchell Esq, surgeon to the Royal Lanarkshire Militia, in Edinburgh on 30 November 1808.[6] If not the same person, this is yet a further child.
Other children appear on the home estate of Savanna-la-Mar in Westmoreland Parish over the years. One John Drummond (1799–1829) son of Dr John Drummond (with great political incorrectness described as a "sambo") is listed on the estate in 1800.[7]
Through marriage to the Murray family most of Thomas Drummond's descendants in Jamaica were thereafter known as Murray Drummond.