He started out playing for Walkerburn.[2]
Duffy then played for Selkirk. He became the second player to captain the side for three consecutive seasons; from 1963 to 1966. The first Selkirk player to achieve this feat was George Downie (1951–54).[3]
He had to give up playing rugby union after a bad car accident in 1970.[4]
In 1978, as Secretary of Selkirk he tried to get Northern Ireland rugby union sides to play friendlies against Selkirk prior to the Scotland versus Ireland match at Murrayfield Stadium.
He was a selector for the South of Scotland District.[5]
He became a Vice-President of Selkirk rugby club in 1989.[5]
Duffy then became a President of Selkirk.[6]
In 1996, he was elected the South of Scotland District representative on the SRU board.[7]
Just before his SRU presidency he became involved in trying to save Walkerburn who were struggling at the time. He stated: "Rugby has always been my life. I couldn't play after I was in a bad car accident 35 years ago but I have always been involved with clubs. It gives me something to live for. I believe that many sports clubs are the heart of small communities and without them the world would be a sadder place."[4]
Duffy became the 117th President of the Scottish Rugby Union. He served one year in office from 2003 to 2004.[1]
When Edinburgh Rugby reached the final of the inaugural Celtic Cup in 2003, Duffy asked clubs to postpone their weekend fixtures to support the Edinburgh side. He stated: "Because of the magnitude of this event, I would like to encourage all clubs in Scotland not to play any fixtures in conflict to the Celtic Cup Final so that everyone in Scotland can support Edinburgh Rugby in the final of a major competition."[8]
Duffy had to state that the speculation that the Scottish Rugby Union would sell off Murrayfield Stadium to pay off their debt was unfounded.[9]
It was under Duffy's presidency that work began to merge the Scottish Rugby Union and the Scottish Women's Rugby Union although the bodies didn't officially merge until 2009. He stated: "When I first took over as president of the SRU, I said that I definitely wanted everyone involved in rugby in Scotland to work together to support Scottish rugby. This proposed integration is a prime example of how we can all work together for the good of the game."[10]