Simon Cowell Baby Mama Lauren Silverman Visits With Husband Andrew: They're "Trying to Keep Things Normal for Their Son"

Andrew Silverman told paparazzi to please keep their distance at this "difficult time"

Looks like Andrew Silverman has a new constant companion.

"This is a very difficult time, please keep your distance," the husband of Lauren Silverman, who's now pregnant with Simon Cowell's child, told photographers who caught up with him outside of the family's house in Bridgehampton, N.Y., where Andrew went to meet with Lauren on Friday afternoon.

Earlier in the day, officers from the South Hampton Town Police Department had stopped by the house after being notified that the homeowners wanted an extra patrol, apparently due to the increased media presence hanging around near the house.

A police spokesman told E! News that they had received calls from neighbors and passing motorists about strange vehicles and all the press in the area, and that someone from the Silverman household had called to say "this is what is going on and giving us a heads-up."

Andrew filed for divorce two weeks ago, naming Cowell as a co-respondent alongside his wife and citing "adultery" as a reason for the breakdown of their 10-year marriage.

Asked about the estranged couple's rendezvous in the Hamptons (Andrew also reportedly was there yesterday), a friend of the couple tells E! News, "They are spending time together trying to keep things normal for their son."

They have even spent the night in the same house, the friend adds.

Andrew and Lauren are parents of a 7-year-old boy, who was spotted out walking in town with his mom on Thursday, the first sighting of Lauren since news broke that she was going to be making a father out of Simon Cowell, a longtime friend of both her and her husband.

Aside from what's in the divorce filing, there hasn't been any public mud-slinging between the Silvermans, with Andrew saying in a statement yesterday, "My top priority is protecting my 7-year-old son while working to resolve all outstanding issues with my wife. Hopefully people can respect the interest of our child during this process."

Lauren issued a statement via her attorney, saying, "I am committed to sorting things out with Andrew as amicably as possible to ensure the well-being of our son. I would hope for some space and privacy in order to work through this."

We hear that Cowell, meanwhile, is probably going to address the media all at once regarding his sudden expectant-dad status. He sort-of-broke his silence yesterday at the Fox panel at the Television Critics Association press tour (after being primped at home by makeup-artist friend and ex-fiancée Mezhgan Hussainy) when a reporter offered congratulations.

"I haven't read the newspapers. Am I missing something?" he pretended to inquire, adding, "Sorry. Unfortunately, I have to keep this private for the moment. It's just one of those things, but thank you anyway."

—Reporting by Holly Passalaqua & Claudia Rosenbaum