Veiel joined the army 1904, and was commissioned as an officer in 1905, serving in the Württemberg cavalry during World War I. After the war, Veiel was a member of the Freikorps in Württemberg (1919). From October 1920, he served in the Reichswehr. In 1938 he was given command of the 2nd Panzer Division. Veiel was promoted to oberst in 1933, a Generalmajor in January 1937, and a Generalleutnant in 1938.
In the early stages of World War II, Veiel commanded the division during the Invasion of Poland in 1939, the Battle of France in 1940, Operation Marita, invasion of Yugoslavia and Greece, and Operation Barbarossa in 1941. On 3 June 1940, Veiel was awarded the Knight's Cross of the Iron Cross. From 1 April 1942 to 28 September 1942, after the German defeat in the Battle of Moscow, Veiel was promoted to full general (General der Panzertruppe). He later commanded the XLVIII Panzer Corps. After 28 September, Veiel had a staff position with the Army Group Center.
From September 1943 to 20 July 1944, Veiel was Commanding General of Battle Sector V (Wehrkreis V) in Stuttgart. He was relieved of command because of complicity in the 20 July Conspiracy to assassinate Adolf Hitler. On 16 April 1945 he was put in the Officer Reserve (Führerreserve) of the Army High Command (Oberkommando des Heeres or OKH). After World War II, Rudolf Veiel spent two years in American captivity. He was released in 1947 and died in 1956.
