Born in Tholen, Zeeland, Kousemaker started playing football at a young age at amateur club Tholense Boys. In 2000, he was brought to the NAC Breda youth academy at the age of 15. There he went to play with the under-17 team.
On 18 April 2004, Kousemaker made his senior debut against Volendam, replacing the injured Robbie Haemhouts,[1] who had substituted the similarly injured Nebojsa Gudelj earlier in the match. NAC lost this game 4–2. In January 2005, he made a few more appearances for NAC, also due to injuries to starters. Because Mark Schenning suffered an injury, Kousemaker was added to the first team squad. He played a few games and at the end of the season his amateur status was turned into a two-year professional contract. During the 2005–06 season, NAC struggled and head coach Ton Lokhoff was fired as a result.[2] His replacement Cees Lok soon announced that he did not see future prospects for Kousemaker. He was removed from the first-team squad and transfer-listed. A few months later, Lok was also dismissed and Kousemaker's future remained uncertain.[3]
Before the start of the 2006–07 season, Kousemaker made multiple appearances in pre-season games and left a good impression with new manager Ernie Brandts. However, this did not immediately result in playing time during the season, and Kousemaker opted ahead of the 2007–08 season to move to HFC Haarlem in the second-tier Eerste Divisie.[4]
On 31 December 2009, it was announced that Kousemaker would finish the ongoing season with Den Bosch as the successor to Adnan Barakat who had moved to Azerbaijani club FC Baku.[5] In May 2011, Kousemaker moved to Dordrecht.[6] In 2013, he was sidelined for a substantial period due to infectious mononucleosis.[7] At the end of the 2013–14 season, he won promotion with Dordrecht to the Eredivisie.
On 1 August 2014, he signed with Topklasse club VV Capelle.[8] Kousemaker retired from football in May 2020 as part of his first club, Tholense Boys.[9]