Born in Lexington, Kentucky, Brown was sold at the age of seven to Robert A. Alexander, proprietor of the famous Woodburn Stud near Midway, Kentucky. He worked as a groom and grew up developing a keen understanding of horse breeding and how to condition horses for racing. He was nicknamed "Brown Dick" after a famous horse of that name, an apparent reference to Brown's own speed as a foot racer.[2]
His small boyhood stature and knowledge of horses afforded him the opportunity to become a jockey, considered a "privileged position" for a slave. At age 14, he rode his first race and won aboard a colt named Asteroid whose sire was another Alexander horse, Lexington. The following year, Brown was emancipated after the Civil War. He remained as an employee of Robert Alexander and rode a number of his horses to victory in important races. The most notable of these was Asteroid, who went on to win all twelve of his career starts including multiple stakes races and was considered one of the best American racehorses of the nineteenth century.[2]
Robert Alexander died in 1867, and two years later Woodburn Stud manager Daniel Swigert left to establish Stockwood Farm. Brown accepted an offer to ride for Swigert's new stable and in 1870 he won the Belmont Stakes aboard Kingfisher. However, as he developed into a young man his weight gain hampered his ability to successfully compete in flat racing and for a short time he switched to riding steeplechase horses. With his vast knowledge of thoroughbreds, in 1874 Brown turned to training Swigert's horses. In 1877, he conditioned Kentucky Derby winner Baden-Baden. Brown was also the original trainer of Belmont Stakes winner Spendthrift and future Hall of Fame colt Hindoo (the 1881 Kentucky Derby winner) before they were sold at age two by Swigert.[2]
Brown later started training for other owners, including Milton Young, who was the fifth leading owner in 1881. In 1886, he finished second in the Kentucky Derby with Blue Wing, beaten by just a nose. Brown became a respected fixture on the racetracks of central Kentucky. The Louisville Courier-Journal wrote, "You see one side of Brown Dick's character when questions of fact are disputed before the judges and men accept his word as weightier evidence than the affidavits of many men."[2]
Brown used his profits to build a quality racing stable that competed under the name Ed Brown & Co. His keen knowledge of horses and breeding saw him buy unraced horses that would be among some of the best racers during the final decade of the 19th century. In 1893 Brown won the Kentucky Oaks with his filly Monrovia, a feat he would accomplish again in 1900 with Etta. Because he lacked the necessary capital to compete with the millionaires who dominated the sport, Brown used his limited funds to buy horses he believed had great potential, then trained and raced them to the point where their success attracted purchase offers from other wealthy owners. Such was the case of Ben Brush, whom Brown bought in partnership as a weanling and trained into the U.S. Champion 2-year-old of 1895. Sold to the Dwyer Brothers Stable, Ben Brush won the 1896 Kentucky Derby. In a twist of fate, Ulysses – a two-year-old horse Brown was unable to sell – finished last to Ben Brush in the only Derby that Brown ever contested as an owner. Similarly, Brown purchased Plaudit from breeder Dr. J. D. Neet and trained the colt until reselling him to John E. Madden, who then won the 1898 Kentucky Derby.
Struggling with rheumatism and tuberculosis, Brown was forced to retire in 1903, reportedly one of the wealthiest African Americans in the state of Kentucky. He died three years later in Louisville.[2]
Years after African Americans had been driven out of the sport, they and Brown's stature in racing was recognized in a May 2, 1942 Daily Racing Form article titled "Colored Folk Play Big Part in Sport: Brown Dick Regarded as Most Famous of All Negro Trainers."[3]
In 1984, Brown's important role in thoroughbred racing was confirmed with his induction into the National Museum of Racing and Hall of Fame, as "a standout jockey and then one of the top trainers of the 19th century".[1]
Brown, Alexander, Swigert, and Asteroid are figures in the 2022 Geraldine Brooks best seller historic novel, Horse, based upon the life of the race horse Lexington.[4][5]
