The song was inspired by the retired film director John S. Robertson who lived in the small town near San Diego where the Byrds' bassist Chris Hillman grew up. Robertson was an aberrant figure around the rural area, frequently being seen wearing a Stetson hat, and sporting a white handlebar mustache, which gave him the appearance of an American frontiersman out of the Wild West.[4] In the song's lyrics, Hillman recalls how the town's children cruelly laughed at Robertson's appearance, and the combination of shock and awe that he provoked in the townspeople.[4]
Hillman attempted to capture the visual eccentricity of Robertson by having the band record the song as a country-western two-step. On the album version of the song, an unnamed session musician underscores the visual representation by playing a fiddle, mixed deep within the track.[1] Writer Ric Menck has speculated that the fiddle player is Byron Berline, who had recently moved to Los Angeles and mingled with the Byrds' crowd, as well as appearing on various other country rock albums of the period.[1] The country-tinged nature of "Old John Robertson", like a handful of other Byrds' songs from the period, foresaw the Byrds' later experimentation with country rock.[2][5]
During the recording of the song, Hillman switched instruments with band member David Crosby to play rhythm guitar, instead of his usual bass. In press conferences, Hillman explained that he was beginning to improve upon his guitar playing, and since he wrote "Old John Robertson" with the instrument, he wanted to carry it over to the recording, even though Crosby had not played bass since the early days of the group.[1][4]
The album version of the song makes broad use of the audio effects known as phasing and flanging, particularly during the orchestral middle section and third verse.[6]