Frazer won an exhibition scholarship to read natural sciences at Balliol College, Oxford; he stayed at Oxford for four years from 1920 to 1924, completing his degree in three years and gaining a first-class honours, and then staying on to do chemistry research for a further year.[2] His sporting record at Oxford was patchy. He played in the two non-first-class trial matches at the start of the 1921 season, but was not selected for the first-class team, instead making his first-class debut against the university side for Somerset: this was not an unusual arrangement for Somerset's matches with the universities in the first third of the 20th century, giving the county the chance to try new players (and also filling gaps in its largely amateur team) and allowing the university cricket club the opportunity to see potential players of its own. In the single match that Frazer played for Somerset, scoring 4 and 14, a second Oxford freshman (and chemist), Guy Blaikie, also played for the county side; unlike Frazer, Blaikie returned to Somerset for further matches in subsequent seasons.[4] Frazer did not play for Oxford that summer or the next, instead turning out for Sussex, for whom he was qualified by residence, in a couple of County Championship matches in 1921, and a further five games in 1922. He had no great success in these matches, with a highest score of just 40, made in the match against Warwickshire at Hove in 1922.[5] In 1923, he captained one of the teams of "seniors" in a trial match and saved the game for his side by making an unbeaten 81 in the second innings: "his driving was powerful and his hitting on the leg side was very clean," said Wisden in its report in the 1924 Almanack.[6] However, he again failed to appear in any first-class games for the university first team and he once more played in a single match against the university cricket team, this time for Free Foresters in a first-class match in June; he then appeared in nine Sussex matches at the end of the university term, raising his personal highest score marginally to 43 in the game against Gloucestershire at Bristol.[7][8] In September 1923, before starting his postgraduate year, Frazer was a member of a Free Foresters side that toured Canada, playing seven two-day matches.
While Frazer did not play first-class cricket during his undergraduate time at Oxford, he did play for the university's soccer team, winning a blue in the 1922/23 match against Cambridge at outside-left. He won a second blue in his postgraduate year when, in the report in The Times, he was "the most dangerous forward on the Oxford side".[9]
In 1924, as a postgraduate student with more time, Frazer finally played first-class cricket for Oxford University. He played in most of the team matches in the 1924 season, batting mainly in the middle order, but his best score and his only total ever of more than 50 was made as an opening batsman, with 81 in the match against H. D. G. Leveson-Gower's XI at Eastbourne, a rather weak team composed entirely of amateurs, many of them not in the first flush of youth (Ernest Smith, the captain, was 54 and had played for Oxford University 36 years earlier in 1888).[10] Frazer's timing for a big innings was immaculate: the next Oxford game was the University Match and he duly won a cricket blue to go with his two soccer blues, scoring 5 and 20 in his two innings in a nine-wicket defeat by Cambridge.[11] Frazer played seven further matches for Sussex when the university term was over, without success, and appeared for two amateur teams in first-class matches against Oxford University in 1925, but that was the end of his first-class cricket career.