Baker earned his living driving a laundry van whilst playing for Port Vale's reserves as an amateur.[3] His performances for Vale's second string caught the attention of Wolverhampton Wanderers and on one evening manager Frank Buckley was due to travel to Vale to sign Baker.[3] But Stoke City manager Bob McGrory caught wind of the potential transfer and arrived earlier and persuaded Baker to sign for Stoke instead. He spent the 1936–37 season in Stoke's reserves, making only the occasional first-team appearance due to England international Joe Johnson occupying the left-wing position.[3] But he managed to get his chance one match into the 1937–38 season as Johnson injured his ankle and Baker took full advantage.[3] He linked up well with his centre forward Freddie Steele, providing him with many crosses and he also chipped in with 12 goals and scored a further 10 in 1938–39.[3] He was called up for army duty in 1939 and therefore played few matches for Stoke in the War League, although he did guest for Sunderland and Northern Irish club Linfield.[3] He played for Stoke against Bolton Wanderers during the Burnden Park disaster on 9 March 1946, and broke down in tears on the final whistle.[5]
When League football resumed in 1946–47, Baker had competition in the form of the hunchback Alexander Ormston.[3] McGrory decided to move Baker to inside left to replace the retired Tommy Sale, and although his goalscoring suffered the pair formed one of the best left wing combinations in the country which nearly helped Stoke win the First Division title – the "Potters" their must win match against Sheffield United 2–1.[3] Baker then suffered a crop of serious injuries which ended his career, in August 1947 he broke his arm against Liverpool and missed most of the 1947–48 season. He broke his leg against Manchester United in October 1948 and then fractured it again at Wolves five games into his comeback in April 1949. In all, Baker broke bones five times in two years.[3] After spending a year in rehabilitation, he retired in the Summer of 1951 on doctors' advice. He then ran a fish and chips shop in Fenton for many years, coached Stoke's 'A' team and managed non-League Foley until he died in 1989 at the age of 71.[3]