In a recent interview with E! News, the 'America's Got Talent' judge revealed that he has no plans to host his own daytime talk show - and even backed out of a previous offer to do so, reports people.com.

"I got to the point where they built the set and I started to get anxious and then really stressed out, and I just walked out the meeting," Cowell, 63, told the outlet after being asked if he would consider hosting a daytime talk show like his American Idol discoveries Kelly Clarkson and Jennifer Hudson.

He continued, "I just said I literally couldn't do this. I just couldn't talk to people all day long. I'm not very good at talking."

While he may not be the right person for the gig, Cowell continues to be great at identifying talents and is resuming his role judging alongside Howie Mandel and Heidi Klum on AGT: All-Stars, a new extension of the beloved series on NBC, which premiered on Monday.

Speaking previously to 'PEOPLE' about the show, Cowell said, "It was terrific and it was really fascinating to see people who've competed on the American show and done really well, competing with people who won the show in other countries."

"The dynamic was fascinating ... and what really was great and cool about the show was that so many people decided to come back and compete," he added.

Much like the original series, AGT: All-Stars - which is hosted by AGT emcee Terry Crews - sees dozens of variety acts compete for the winning title. However, this show is different in that it features 60 of Got Talent franchise's past winners, finalists, fan-favourite acts and viral sensations as they compete to be the best of the All-Stars.

"We were reunited with so many great people, and then met so many great people, and there were a lot of surprises on the show as well," Cowell told 'PEOPLE' in November. "I really, really enjoyed it."