Wolffe was graded by the Penrith Panthers in the 1978 season. Penrith coach Len Stacker gave Wolffe his first grade debut on the wing in his sides' 33-7 loss to the Newtown Jets at Penrith Park in round 14 of the 1979 season. He scored his first try in his sides' 19-8 loss to the Eastern Suburbs Roosters at Penrith Park in round 20 of the 1979 season.[3]
The early 1980s was a troubled period for the Panthers. They went on to finish with the wooden spoon in the 1980 season after winning only 2 games. In 1981, the Panthers won eight games and went on to finish 11th. Panthers coach Len Stacker was subsequently sacked from the coaching role at the end of the 1981 season. In 1982, John Peard replaced Stacker as head coach. Wolffe had his best season at the Panthers in the 1982 season when he finished the season with 8 tries, second only to Panthers newcomer Brad Izzard with nine. Despite these efforts, the Panthers went on to finish the 1982 season in 12th position. Peard's stint as coach ended at the conclusion of the 1983 season.
In 1984, former Panthers player Tim Sheens took over the coaching role. The Panthers were a much more formidable outfit under Sheens' coaching. For the first time the club's history, they looked like making their first ever finals appearance in the 1984 season, when with just two rounds to go, they'd won 12 games, and were in 5th position, but they lost their last two games to the Canberra Raiders and Parramatta Eels in round 25 and 26 respectively, and subsequently missed out on finals qualification.
The Panthers would eventually be finalists in the 1985 season. They defeated the Manly Sea Eagles 10−7 in the playoff for fifth spot, but a few days later, they lost 38−6 to the Parramatta Eels at the SCG. This was Wolffe's lone appearance in a finals match. Wolffe's stint with Panthers ended at the conclusion of the 1986 season.[4]
In 1987, Wolffe shifted to arch rivals and defending premiers the Parramatta Eels. He made only eight appearances with the Eels, before announcing his retirement at season's end. He finished his career having played 111 games, and scoring 31 tries.