The father of three highlights this very point in a clip from Thursday, March 18's season 20 premiere of Keeping Up With the Kardashians. The Flip It Like Disick star shares this revelation after Kourtney asks how Scott is doing amid his break from Sofia Richie.
"I don't know, I just think people come into our lives and think it's maybe easier than they think," the Talentless founder starts off. "To be with you or to be with me, but it's like, very true that we do come with a lot of baggage."
As he continues, he acknowledges that it can't be easy for a significant other since he and Kourtney "see each other, work together and are friends."
"When you first get with someone everybody is like, making changes and 'Oh, I'll do anything 'cause I'm so happy or so in love and so excited,'" Mason Disick's dad relays. "But then, when it sinks in and it becomes real life, it's a lot different."
Reflecting on the situation, Scott admits he wouldn't be able to do it if the roles were reversed. He adds, "But I've always been clear that my priority has been my children, my life with them. I even put it out there that taking care of you is one of my priorities."
In a confessional, Scott calls Sofia "an absolute trooper." However, he hints that Sofia was feeling like a second fiddle to his other priorities. He explains to the KUWTK camera, "The truth is, anybody dating somebody is gonna feel neglected when their significant other is spending more time with their ex than with them."
As the conversation continues, Scott ponders aloud about his future with Kourtney: "I just don't know like, are we just gonna grow old just traveling the world with the kids, living one house down from each other or together at some point?"
While the Poosh founder doesn't directly answer Scott's query, she does applaud their co-parenting relationship.
"Well, I think that it's great that we can do that with the kids and the kids like, love it," Kourtney concludes. "And I do think we have to be respectful of our relationship and make sacrifices for that, if it means a lot to you."
Catch the candid conversation in the clip above.
Binge past episodes of KUWTK on Peacock before Thursday's premiere.
(E! and Peacock are both part of the NBCUniversal family.)