The gang was founded by Ralph Sheldon in the months following the passage of the Volstead Act and it quickly became one of major liquor suppliers in Chicago's Southwest Side as well as a main supplier to Al Capone. The gang maintained an uneasy truce with the other rival gangs, until 1923, when a brief gang war broke out between the Sheldon Gang and the Southside O'Donnells until O'Donnell leader Edward O'Donnell was forced to leave Chicago after being severely wounded, by Frank McErlane, during a drive-by shooting on September 25, 1925.
Once the O'Donnells were eliminated, the Saltis-McErlane Gang, former allies of the Sheldon Gang, continued attacking the Sheldon Gang and by April of the following year the gang had lost their two top distributors, Frank DeLaurentis and John Tucillo. In October, a general amnesty had been ordered between Al Capone's Chicago Outfit and Bugs Moran's North Side Gang. However on December 30 Sheldon Gang member Hillory Clements was killed by the Saltis-McErlane Gang and in retaliation the Sheldon Gang killed Saltis gunmen Charlie Hubacek and Frank Koncil on March 11, 1927.
Joe Saltis soon went to Capone who, in exchange for a large percentage of his profits, would negotiate peace between the two. During the later years of Prohibition the Sheldon Gang was suspected of supplying liquor to Capone rival Bugs Moran's North Side Gang. In the closing days of Prohibition and with the emergence of Lucky Luciano's National Crime Syndicate the Sheldon Gang, like most of the remaining bootlegging gangs, were absorbed into the syndicate by 1932.
Sometime in the 1930s, the Sheldons dropped a bomb from an airplane into an ice cream store, in Creve Coeur, Illinois.