On 31 January 1854, he married Sophie van der Mark (1818–1856), widow of the Count of Panisse-Passis and daughter of Jean Baptiste Agapit van der Mark and Marie Françoise Amélie Lanchère de la Glanderie. Before her death in 1856, they were the parents of:[1]
- Maria Wilhelmine von Quadt-Wykradt-Isny (1854–1914)
- Otto Gebhard Lothar von Quadt-Wykradt-Isny (1856–1928)
He married, secondly, to Marianne, Countess of Rechberg and Rothenlöwen (1834–1910), daughter of Count Albert von Rechberg (president of the First Chamber of the Estates of Württemberg) and his wife and cousin, Countess Walpurga Marie Julie von Rechberg (1809-1883), in Donzdorf on 1 June 1858. Together, they were the parents of:[1]
- Julie von Quadt-Wykradt-Isny [de] (1859–1925), a writer.[1]
- Elisabeth von Quadt-Wykradt-Isny (1862–1940), who married Carl Ernst Fürst Fugger von Glött, President of the Bavarian Reichsrätekammer.[1]
- Albert Wilhelm Otto von Quadt-Wykradt-Isny (1864–1930), a diplomat who married Amedea de Martino (1874–1933).[1][4]
- Aloysia von Quadt-Wykradt-Isny (1869–1952), who married Franz von Brühl (1852–1928).[1]
Count Quadt died on 24 October 1892 at Schloss Moos in Lindau on Lake Constance.[1] His widow died there in 1919.
Through his daughter Elisabeth, he was a grandfather of Joseph-Ernst Graf Fugger von Glött (1895–1981), a representative of the Christian Social Union of Bavaria who was a member of the Bundestag of Germany between 1949 and 1953 and a member of the Landtag of Bavaria from 1954 to 1962.[5]