'X Factor' judge Demi Lovato has been through a lot, but if she's irritable these days it's because of one man: Simon Cowell.

"Simon's great, but he nags on me so much that I don't know if I can take it any longer," the 19-year-old said, according to Contact Music. Lovato and fellow pop star Britney Spears officially joined the cast of the hit singing competition series in May.

However, while Cowell can be pushy, Lovato knows how to handle his abrasive ways. ''I'm a brat," she admitted. "I'm annoying. But I give it back to him."

As for her judging style, Lovato isn't exactly a pushover. The 'Skyscraper' singer insists she calls it like it is when it comes to performances. ''It's really hard for me to smile if they're performing and they're horrible," she said. "'I have to be super super honest and sometimes I feel bad about it."

Meanwhile, Cowell's take on his new panel stars is that both Lovato and Spears are "catty" when it comes to female contestants. With auditions in full swing and the new season set to premiere this fall, it's clear that there's going to be plenty of drama with this crew.

Watch Demi Lovato + the 'X Factor' Judges Make Their Entrance