A reality show cast member and former politician who is under investigation for multiple rape allegations is quitting the show. The AP reports that Thomas Ravenel says he will exit the Bravo series “Southern Charm,” where he has been one of the show’s main cast members since it debuted in 2014.

Ravenel tweeted about his decision on Tuesday, writing: “I’m not doing the show anymore. In the contract they have the right to fictionalize your story. They took advantage of me. I decided I’ve got too much to lose and informed them I’m not coming back.”

Ravenel, 56, is the former South Carolina State Treasurer. He reportedly resigned from the post in 2007 after being indicted on federal drug charges.

“He has denied the [rape] allegations and is being investigated by Charleston police and the show’s production company, Haymaker,” the AP reports.