Challis was born in England, the son of John Henry Challis, sergeant in the 9th Regiment, and his first wife. He was educated at several schools and trained as a clerk. He then migrated to Sydney, New South Wales, arriving on the Pyramis on 9 May 1829 as a steerage passenger. The business dealt in wool, whale oil other commodities and became very prosperous. Challis was a full or part owner of nine vessels that made 26 whaling voyages between 1840 and 1852.[1] He acquired several properties, including a large holding at Potts Point, pastoral licenses of over 12,000 sq. miles (31,080 km²) in southern New South Wales, more than 3,500 cattle and 11,000 sheep. In 1855 he sold his business interests and returned to England.