He married twice; his first wife, Anne, died from cancer in 1979, he married his second wife, Elisabeth, in 1980. He had three children, from his first marriage, John, Andrew and Alison and five grandchildren. His son, John, died of motor neuron disease in 2013.[2]
Gemmill was a keen golfer, and liked travel and music.[12] In his younger days he was a keen scout and attended the 1947 International Jamboree in France; a trip that he later recounted to the Scouts of St. Gilbert's Church for their own jamboree.[2]
He was treasurer of his local church, Sherbrooke St Gilbert’s Church in Pollokshields, for eight years. His period of treasurer was notable for his successful organising of a fund raising campaign to pay for church repairs after a disastrous fire.[2]
He gained a first class honours degree in economics from Glasgow University.[12] After graduation he served on the business committee of the general council of the University and also served on the University court. In recognition of his services to the university he was given an honorary doctorate in 2000.[2]
Gemmill worked for Procter & Gamble then PA Management Consultants. His move to PA Management Consultants meant that he could return home to Glasgow, and he resumed his rugby with Glasgow HSFP.[12]