Scarlett had emigrated from England and stated in a 1691 court that he had lived in Stafford County for more than 30 years, which record survived because the case had been appealed to the General Court in Williamsburg.[3]
His plantation, which he called "Deep Hole" because of an artesian spring, was near the confluence of the Potomac River and Occoquan Creek, near both Belmont Bay and Marumsco Creek, not far from where Captain John Smith landed on one of his Potomac expeditions and which was frequented by Native peoples.[4] That area split from Stafford County to become Prince William County in 1731, well after his death. Around 1655, Scarlett purchased 700 acres from the widow of Thomas Burbage of Nansemond County (who had remarried to Capt. Edward Streator).[5] Then, in 1666, Scarlett and Richard Normansell patented 2,550 acres on what later would be called Mason Neck north of that confluence, in what split from Prince William County and became Fairfax County, Virginia about a decade after the split from Stafford County.[6] In 1674, the Stafford County court divided the parcel (using Pohick Creek as the dividing line between Normanstone's and Scarlett's shares); Scarlett sold all but 320 acres of his share, along Pohick Creek, to George Mason on 10 March 1690.[7]
Around 1675/6, Scarlet had married a widow, Ann Green (widow of William Greene Jr., who was affiliated with Lawrence who rebelled in Bacon's Rebellion),[8] and in his lifetime had given half his Deep Hole farm to his stepson Joseph Green. Ann Green Scarlett also had two daughters: Anne who married Edward Barton, and Lettice married Edward Smith (and had 3 children before being widowed) then remarried around 1695 to Burr Harrison (1668–1715). Anne Green Scarlett bequeathed the 740 acre Deep Hole property to her son Joshua Green in October 1696, and the rest of the former Normansell property to Edward Barton.[9] When Joshua Green died childless, the property passed to Lettice Green Smith Harrison, who died probably in 1699, since Harrison petitioned the Stafford Court to become the guardian of his late wife's three children.[10] In 1765 Lettice's great-grandson John Hancock sold the property to Col. John Tayloe.[11]