Fox Broadcasting Entertainment President Gail Berman recalled that this past fall, when her network was faltering in the ratings, she was tempted to add another edition of “American Idol” to the network’s schedule. Instead, showing discipline rare for a network executive, she suffered through those dark days and looked ahead to the first quarter of this year.

“It was very hard to do,” she said with a laugh last week at a party held in Los Angeles to cele-brate the success of “American Idol” in its fourth season. “In November it was really hard to do. But it’s the thing that keeps the show coming back year after year, and people waiting and wanting it.”

Doing it only once a year “keeps the show fresh and alive,” said Tom Gutteridge, CEO of FremantleMedia North America, which produces the U.S. edition. The pop music competition has expanded to versions in 38 other countries around the world.

That tantalizing wait, added Mr. Gutteridge, is what makes it “a national institution. Just look at how America switched on to it this year. They were waiting for the show. It had instant success. And it’s extraordinary, because America needs the show. It’s now part of American culture, and that’s exciting.”

It is also the dream of every other producer, writer and network executive-to have a water-cooler show that year after year commands ratings, respect and big bucks.

I went to the party for “American Idol” seeking to understand why this show remains at the top after four years. The nature of TV is that most shows wear out. Fox was prepared for that to happen this year. When Ms. Berman met with TV critics in January, on the eve of the premiere of the new “Idol” season, she tried to lower expectations. “I certainly think in the fourth [season] of a show one can anticipate seeing some audience dispersion,” she said. “That’s only natural.”

Then premiere night attracted about 54 million viewers, making it the third-most-watched night in Fox Network history. It was the biggest thing on TV since the “Friends” finale on NBC. Ms. Berman admitted to the media, “No one in their right mind would have predicted a 9 percent increase. I’ve never seen numbers like this. I’m stunned. We are all stunned.”

So what is it that creates “Idol” worship? At the party I ran into Jonathan Taylor, a former trade and consumer press editor who is now an executive with a PR firm that isn’t associated with “Idol.” “All great television is about stories,” he told me. “It doesn’t matter whether you’re doing news, drama, sitcoms or reality. You have to find something that engages, that has some drama to it. … You have to have somebody to root for and somebody to root against. This show has incredible storytelling.”

You do feel like you get to know the contestants on “Idol” as the elimination process narrows the field. Is that it? I asked Ms. Berman.

“There are two reasons it works,” she said. “One, it is an incredibly aspirational program, and people want to see that. And two, the people get to pick. It’s like getting behind your racehorse, your contestant, your rocker, your singer. And you really get to see the fruits of your efforts when voting.”

Ed Wilson, president of the Fox Television Network, told me “Idol” is that rare show that appeals to everyone. “It’s a phenomenon because it brings families together,” he told me at the party. “It’s one of those unique television shows that allows you to sit down and watch with your children, whether they are 3 or 4 years old, or 17 or 18 years old. There aren’t many shows like that on television.”

That made me wonder why there aren’t more copycat programs. After all, the highest form of flattery in TV is usually imitation. Yes, there was “Star Search,” now canceled, and a young people’s edition of “Idol,” and a few other wannabes. But none hit the same high notes in terms of cultural interest or ratings. Is there something unique about the show?

It is something unique about Fox, TVWeek Senior Reporter Christopher Lisotta suggested as we watched a rerun of “Idol” on a bank of TV screens at the party. “Fox represents in-your-face programming,” Mr. Lisotta explained. “It’s youthful. It’s an upstart. It’s often rough. It’s rarely sugar-coated.”

“Oh, you mean Simon Cowell,” I said, referring to the controversial judge who never hesitates to say what he really thinks. Chris answered: “[Fox] was the first to put someone like Simon on the air. Who else would say [to a contestant], ‘You’re horrible, terrible. Do you have a teacher? You should fire her’? He said that the first season and [Fox] put it in a commercial. Would NBC have done that? I don’t think so.”

Then I ran into a cynical middle-aged journalist from New York who didn’t want to be quoted but had his own take on “Idol.” He said it is the humiliation factor. People say they hate it but in fact love to watch those contestants being put to the test. And it’s not just Simon doing the humiliating. It is the whole way the show is set up to drag out the competition-to have host Ryan Seacrest play games about who is staying and who is going.

“It is painful to watch at times, but you can’t take your eyes off of it,” the journalist added.

Back in January, the New York Daily News ran a column item calling the way applicants were weeded out on the first shows “cruel” and in some cases “mean.” The newspaper quoted a fan who had written on a chat board: “To crush them like that was abominable.” There were even calls on Internet sites to boycott the show.

The result? The ratings went up again.

That reminded me of something else Mr. Taylor said: “There is no reason this thing should have worked. It’s such a fraud. They say they are going to find the next great singer, but it’s never the next great singer. It’s some talented singers … but people don’t watch it for their ability to sell records. It’s the drama.”

Whatever the secret, Fox and Fremantle hope to go on confounding expectations. No one can manufacture this kind of magic, but they are doing a pretty good job of keeping it going. And no matter how good the marketing, the public eventually catches on to false idols. So far, this one has proved to be the real thing (and I don’t mean because Coke is a sponsor).