According to Radar Online, a source revealed that "Kelsey hasn't even seen Mason, 12, or Jude, 9, in over six months."
"The children remained in Southern California while Camille underwent a radical hysterectomy for endometrial cancer in Houston," the source told the website. "She wanted the kids to remain in school, and they were well cared for. Kelsey didn't see the children while Camille was in Houston for her cancer surgery. He didn't make any attempt to visit them, or have the kids come stay with him."
The source added that while he "calls the kids," it's "been a very long time since they actually saw their father."
"The Radar story is wrong," Kelsey's rep told E! News on Friday, Nov. 1. "Kelsey has been out of state and out of the country filming, so he was not able to have the kids with him on location."
The rep added he has been filming for the past two months.
A rep for Camille had no comment.
Earlier this week, the former Real Housewives of Beverly Hills star obtained and was granted a temporary restraining order against her boyfriend Dimitri Charalambopoulos, after claiming that he abused her in a Houston hotel room where she was recovering from surgery.
The 45-year-old was diagnosed with an early stage of endometrial cancer last month.
—Additional reporting by Maureen Heaton