Little is known of Smith's early life, though he was born in Emporia, Kansas, the child of Tennessee Exodusters.
His professional career began in childhood, performing in Nathaniel Clark Smith's Picaninny Band before moving into minstrel troupes, vaudeville and cabaret. In 1913–1914, he made several early recordings with James Reese Europe's group, the Clef Club Society Orchestra. Between 1914 and 1919, he performed in the Ford Dabney Orchestra, the resident band at Florenz Ziegfeld's Broadway cabaret, Midnight Frolics. Between 1917 and 1919, they produced several dozen phonographs.
In June 1919, Smith relocated to Paris, playing with Louis Mitchell's Jazz Kings from 1919 to 1924, a group that recorded for Pathe Records. He became the leader of Mitchell's group in 1923. He toured France, Spain and Russia with his own bands from 1925 to 1933. During the Depression, he spent nine years in Southeast Asia, working with Herb Flemming, Leon Abbey, and Teddy Weatherford, mostly in Bombay and Batavia. In 1936, he recorded with a group called the Symphonians. Around 1943 he returned to New York, and died in late 1944.