A department of Los Angeles County came down hard on actor Charlie Sheen, dealing the star of FX’s "Anger Management" a setback in his struggle with his ex, Brooke Mueller, over their 4-year-old twin sons, Bob and Max.

TMZ.com reports that a "very punitive" Department of Children and Family Services denied Sheen the right to spend time with the twins.

"Sources familiar with the situation tell TMZ … the department contacted Denise Richards — who has temporary guardianship over twins Bob and Max — and they told her she was not allowed to take the kids over to Charlie’s house today for a visit," the website reported Saturday. "We’re told department officials told Richards they were pissed off that Charlie dared to tell TMZ Bob and Max were not being protected by the department."

Sheen complained Friday about Mueller and about the role of the Department of Children and Family Services, as TMZ reports separately. The interview includes some of Sheen’s typically colorful language, with the actor describing Mueller’s home as "a house full of creeps and cretins" and quoting one of his sons saying of the house: "Oh no — I’m back at the crazy house."

Sheen also says in the interview that he believes Mueller is using drugs again, and accuses the DCFS of failing to accurately track the situation involving Mueller and the twins.

Commenting on the latest county action, TMZ notes: "DCFS’ actions are curious … because it seems the decision to deny Charlie visitation was not based on his fitness to be around his kids but rather on the embarrassment he caused the department."

Charlie Sheen