Brown went to Manual Arts High School in Los Angeles, then spent his first two years of college at Loyola Marymount University.[2] As a freshman in 1981–82 he averaged seven points and six rebounds, and as a sophomore he averaged 12.5 points and led the team with 8.7 rebounds a game.[3] LMU did not offer the major that Brown wanted to pursue, so he decided to transfer. New Mexico coach Gary Colson saw him play in a summer league in Los Angeles and offered him a scholarship. Brown then sat out the 1983–84 season per NCAA transfer rules.
Brown was a 6' 6" forward, yet he was most comfortable playing inside the lane, using muscle and position to score and rebound over taller players. He thrived on contact and scoring in traffic, especially on bank shots off the glass, calling that "maybe my best attribute."[4] He learned to muscle inside by playing streetball with bigger players, including some who later played professionally.
Brown missed the start of his junior season, 1984–85, with a foot injury, but he had an immediate impact when he joined the New Mexico line-up, scoring 26 to lead the team to a win at rival New Mexico State.[4] The Lobos then beat NCAA tournament-bound Arizona before facing #1 Georgetown, led by Patrick Ewing. The Lobos lost in a competitive game, with Brown and front-court mate George Scott keeping the game close despite a marked size disadvantage.[5] Brown led Lobo scorers on the season with 18.8 points a game, adding 6.3 rebounds a game.[6] Brown and Scott both placed among Lobo all-time leaders in field goal percentage in a season.[7] The Lobos built a 14–6 record but then lost five of six down the stretch. They received a bid to the NIT, where they beat Texas A&M and then lost to Fresno State, finishing the season 19–13.[8]
The Lobos in 1985–86 suffered a key injury to guard Hunter Greene in the preseason, and Scott was lost to graduation. Senior Brown led an inexperienced squad with 20.9 points and 7.5 rebounds a game.[6][9] He also led the team in field goal percentage and minutes played. The Lobos played well at home but struggled badly on the road. Despite a fifth-place finish in the WAC, the Lobos again received an invitation to the NIT, where they lost to Texas and finished 17–14.[10] Brown was named first team All-WAC.
In his Lobo career, Brown scored 1,157 points, averaging 19.9 points a game.[6] He is one of four Lobo players to score over a thousand points while playing only two seasons, and his career average is fifth best in school history.[11] He also finished with the fifth best career field goal percentage in Lobo history (.569) and tied for first in single-game field goal percentage, going 11-for-11 in a game against Air Force.[12] Brown graduated from UNM with a degree in University Studies, Sociology, and Speech Communication.[13]
Brown was selected in the seventh round of the 1986 NBA draft by the Los Angeles Clippers,[14] though he never played in the NBA. He played for five seasons in the World Basketball League (WBL), most notably with the Las Vegas Silver Streaks. He was named to the WBL All-Defensive Team in 1988, as Las Vegas won the league championship,[15] and he was a three-time WBL All-Star.