At a conference with TV critics today, Cowell had this to say about the most recent season of X Factor, with former judges Britney Spears and L.A. Reid: "It's like having a dinner party," Cowell said. "You invite people for dinner and sometimes it's a fun night and other times it's not as fun as you hoped it would be. This [season] is a fun dinner party."
So when did Simon know it wasn't working? "Day one," he said. "I didn't say that!" He then explained: "I felt we were doing what everybody else was doing and I had this impatience to get on to season three so we could do what we wanted to do."
Cowell, who was joined on stage by returning judge Demi Lovato and new judges Kelly Rowland and Paulina Rubio, explained that the new judging panel has so much more energy, and feels "like family." Producers also revealed that the show is going back to the original categories: boys, girls, over-25s and groups.
Another tough (and intriguing) Q came for Simon during the panel: If X Factor doesn't stick around, would he ever consider a return to his original TV home, American Idol?
"I can't imagine either scenario!" Simon said. "I can't. I just can't do that. No. Somebody well-known once said to me, ‘Know when to leave the fair.' And this [X Factor] is a new generation type of show. It's got to build. And most importantly, if you make great TV shows, people will watch them. I feel really comfortable with the girls and I think this is going to be the best show."