Spencer was born in Richmond, London, England but emigrated to Australia aged just three-years-old. In 1928, a contingent of Australian riders arrived in Britain to demonstrate the new sport, where Spencer competed in various exhibition events. He was nicknamed the Baby Cyclone and on more than one occasion fell foul of the local police due to motor car offences.[2] He rode more events during 1929, while based in Exeter and rode for the Exeter team in challenge events, before being signed by Leicester Super, where he began his British leagues career during the 1930 Speedway Northern League season.[3]
In 1931, he was signed by Freddie Hore, the general manager of the Pennycross Stadium and a former room-mate of Spencers in Leicester. He rode for and captained Plymouth Tigers in the South Coast Championship.[4] In 1932, the northern and southern leagues merged to form the National League and he rode again for Plymouth.[5] He continued to represent Plymouth in 1933 and 1934 but broke his leg during June 1933, ending his 1933 season early.[6]
In October 1934, he joined Wimbledon Dons for the 1935 season but failed to make the team. In 1936, he was loaned to Bristol Bulldogs for the 1936 Provincial Speedway League,[7] before joining and captaining the Norwich Stars on loan in 1937. He finally left Wimbledon, when signing for Norwich permanently in May 1938.[8][9] He helped Norwich win the Provincial Trophy and National Trophy double in 1938 and became Norwich's star rider.[10]
Although his career was interrupted by World War II, he spent four more seasons with Norwich from 1946 to 1949, becoming a Norwich legend.[11]
He continued to ride in Australia until a crash in 1956 at the Brisbane Queensland Exhibition Ground ended his career.