Richardson was raised in country Victoria. His junior and senior club rowing was from the Nagambie Rowing Club in Victoria's Goulburn Valley starting as a 14 year old in a beginners crew in 1957. .[2] Richardson began rowing and racing in a pair with Neil Lodding when he was just 18 and they won a state junior title and a state country title in the 1961 season. Richardson career's was guided and coached by his pair-partner Neil Lodding's father Alf, the club captain and senior coach at Nagambie.[2] Alf had represented for Victoria in the Kings Cup eights of 1934 and 1935.[3]
Richardson and Lodding aimed for selection for the 1962 Commonwealth Games but were beaten out by the Mosman Rowing Club pair of Maurie Grace and Bill Hatfield.[2] They continued to win Victorian state titles in coxed and coxless pairs throughout 1962, 63 and 64.[2] At the inaugural Australian Rowing Championships in 1962 they took the bronze in the national coxless pair championship,[4] then in 1964 they took silver as a coxless pair and won the gold as a coxed pair with Wayne Gammon on the rudder.[5]