The X Factor judge and Silverman were spotted walking closely together Friday during their vacation in St. Tropez.
Clad in his signature t-shirt and jeans, Cowell palmed a beverage as he took a stroll to a go-kart track with his New York socialite ladylove, whose growing baby bump was visible in the multicolored top she was wearing.
The duo was later spotted Saturday walking on the beach hand-in-hand and smiling.
"They are definitely a couple," the source said. "The relationship is loving and they're very happy together."
Cowell recently broke his silence about becoming a dad, telling the BBC, "Things are changing in my life right now, for the better."
"I'm proud to be a dad. It's something I haven't thought of before, and then, now I know I feel good about it. And she [Silverman] is a very special girl."