Comedy is known to work at the expense of the most visible celebrities, but sometimes said celebrities get tired of being the butt of jokes.

Joel McHale, host of the former E! comedy series The Soup, spoke to Variety ahead of the debut of his upcoming Netflix show The Joel McHale Show With Joel McHale. According to the veteran comedian/TV host, the two shows are like night and day...and that's primarily because his restrictions are no longer dictated by the Kardashians. "When Ted Harbert [started at E!], he basically said the sky is the limit and do what you want to do," McHale said of the former E! chief. "Way back when, Kris Kardashian would complain about our jokes, and Ted would literally go, 'Hey man, Kris called, can you just lay off of her for a week?'"

McHale explained that the muzzle tightened even more once new executives took over the network. "The next administration, the president said to my face, 'Don’t make fun of the Kardashians. We don’t want you to make fun of the Kardashians anymore.' So I was like, oh this show is doomed because that’s why the show worked, because we would make fun of ourselves. It’s like when Letterman made fun of GE in the ‘80s. You have to bite the hand that feeds you. The network really did not like when the Kardashians first came out and then we just said Kim Kardashian was only famous for having a big ass and a sex tape."

The Soup ended its 22-season run in December 2015. McHale even poked fun at the Kardashians in a statement marking the end of his pop culture-themed show. “I loved doing The Soup for all of these years — 86 to be exact — but am excited to solely focus on my acting career now,” McHale told Variety, adding, “Thanks to all who watched, and thanks to Kim Kardashian’s ass for all that it’s done for me and my family.”

Now that he's shifted networks, McHale says there's an opportunity to cover the Kardashians, though that won't be his sole focus. "It’s not a thing where I’m like, “Finally, I get to say what I want to say about the Kardashians!” I’m also a tiny little pilot fish on a huge whale shark that is the Kardashian empire. I don’t think they’ve thought about me much."

Head to Variety to read the entire interview.