Kao was born in modern Yicheng, Zaozhuang, Shandong in 1907. He entered the Whampoa Military Academy in 1926. After graduating from the academy's 4th class, he participated in the National Revolutionary Army's Northern Expedition. After the end of the Northern Expedition, Kao served as the commander of the 3rd Battalion and the 593rd Regiment of the 18th Army and participated in several campaigns against the Chinese Red Army.[1]
After the beginning of the Second-Sino Japanese War, Kao was sent to participate in the Battle of Shanghai. In 1938, he was promoted to major general and served as the commander of the 295th Brigade of the 99th Division. The following year, he was promoted to division commander and participated in the Battle of Kunlun Pass, the Third Battle of Changsha and the Battle of West Hunan.[1]
After the end of the war with Japan, Kao was given command of the 18th Brigade of the 11th Division, and later became the deputy commander of the 11th Division. In 1949 Kao became the commander of the 18th Army and led the army to withdraw to Kinmen. He commanded the 18th Army during the Battle of Guningtou.[2]
Kao became director of the Political Warfare Bureau in 1961. In 1965, Chiang Kai-shek promoted Kao to Commander of the Army. He was then promoted to colonel general in 1967. In 1973, Kao was about to retire, however newly appointed Premier Chiang Ching-kuo convinced Kao to stay in service and appointed him became the Minister of National Defense in July 1973. When he retired from that position in November 1981, he was the last graduate of the Whampoa Military Academy still in active duty. Kao's position was taken over by Sung Chang-chih.[3] After retiring, Kao became a strategic advisor to the Presidential Office, a position he held until his death in 2012.[4]
