Carollo was born in Leland, Mississippi.[1] He was the lead singer of The Bobcats in 1959 when he merged the band with another group, The Rollons, and became the rock group Joe Frank and the Knights.[2] They soon became one of the most popular groups in Mississippi,[2] and toured until late 1965 as according to Joe Frank they had "been together since the eighth or ninth grade" and were starting to get most of the band were married and wanted to do new things.[2]
In 1966, The T-Bones, which was actually a collective of members from The Wrecking Crew, had a hit with "No Matter What Shape (Your Stomach's In)" but none of the Crew wanted to tour in the band, and a new lineup including Carollo was made to tour. The lineup included Carollo on bass, Dan Hamilton on guitar, and Tommy Reynolds on keys.[3] The lineup did not play on the hit record and all but one studio album, as they were allowed to record their 1966 album Everyone's Gone To The Moon (And Other Trips).
After The T-Bones disbanded, Carollo teamed up with Hamilton and Reynolds and formed Hamilton, Joe Frank & Reynolds.[4] The group had two hits in the US in the early 1970s, "Don't Pull Your Love" (US #4) and "Fallin' in Love" (UK #1).[5] They disbanded in 1976 and returned for a few years in the 1980s.