In 1973, at age 17, Walker became the youngest-ever college quarterback.[1][2] In Walker's first start of his freshman season, Texas A&M defeated the TCU Horned Frogs, 35–16. The victory broke a four-game losing streak against the Frogs and was the first of 24 straight the Aggies have won in the series. The Aggies finished the season as the third-highest-scoring team in A&M history, and Walker was named the U.P.I. Southwest Conference Freshman of the Year.
Walker was the quarterback his sophomore season in 1974 and led A&M through an 8-3 campaign, A&M's first winning season since the victorious Cotton Bowl team of 1967. Their 21–14 win against the 5th ranked LSU Tigers, the Aggies' second in 15 tries at LSU's Tiger Stadium, moved the previously unranked squad to #9 in the country. In the game, the Aggies had three ball carriers break the century mark in rushing; Skip Walker with 130, Bubba Bean with 127, and fullback Bucky Sams who finished with 107 yards and scored the winning touchdown late in the 4th quarter. A final game loss at Texas kept them from the Cotton Bowl and the SWC championship, thus allowing Grant Teaff's Baylor team, a 20-0 victim of the Aggies earlier in the season, to claim the crown. The Aggies' Top 15 ranking marked the first time since 1957 the Aggies had been ranked at season's end and only the sixth since their 1939 National Championship.
Walker was replaced by Texas All-Stater David Shipman during the 1975 season and did not play. The Aggies rose to #2 and the country's top-ranked defense before ending the season with one-sided losses to Arkansas and Liberty Bowl opponent Southern California.
In 1976, he regained the starting QB job for the final six games, all lop-sided victories, including the first win in 20 years on the University of Texas campus in a decisive 27–3 victory in Darryl Royal's final A&M game as UT's head coach. Walker capped off the season by leading the Aggies to a 37–14 victory over Florida in the Sun Bowl, applying the finishing touches to A&M's second consecutive 10 win season, a first in A&M history and a feat accomplished only 3 times since. In the game Walker was 11 of 18 for 122 yards, including a touchdown pass to fullback George Woodard. Walker also scored in the 2nd quarter on a 9-yard keeper. The Sporting News ranked Texas A&M #3 and the A.P. had them 7th in the country following the season. It would be 1985 before the Aggies would be ranked again in a final poll.
Walker led Texas A&M to a 37-14 Sun Bowl victory over Florida to cap off the 1976 season and had the Aggies ranked No. 5 in the AP poll after a 33–17 win over No. 7 Texas Tech in 1977.
Walker set a Texas A&M single-game for rushing yards by a quarterback when he carried 27 times for 182 yards in a come-from-behind 38–21 win against SMU. The single-game yardage record stood for more than 30 years until Johnny Manziel rushed for 229 yards against Oklahoma in the 2013 Cotton Bowl Classic.[3]
Walker was injured for a game against TCU in the 1977 season, but returned for the final regular season games against top-ranked Texas and the University of Houston to complete the 8–3 season. He then sat out the 47-28 1977 Bluebonnet Bowl loss to USC until the final minutes.