When news broke recently that the couple had apparently called it quits after nine years of marriage, rumors began running rampant that the hoofer's relationship with Spencer was to blame.
The two competed on the Australian version of Dancing With the Stars earlier this year and the pair was later photographed getting cozy together in June.
"I'm not the other man, never have been, never will be," Whitewood said during an interview on the Australian program Today Tonight.
And while Whitewood pointed out "you can't stop a friendship forming when you are doing a job like this," he was quick to note that "we were just great friends, great partners on the dance floor, that's about it."
As for whether Spencer conveyed there was trouble in paradise between her and Crowe, Whitewood said that when they began dancing together, "she never showed one bit there was problems whatsoever."
But when the season came to an end, Whitewood claimed Spencer began to open up about her marriage.
"Once we'd wrapped and finished, she said things weren't great. And that's pretty much as far as the conversation went."
As for the questionable pics of them reportedly together at Sydney's Star Casino, Whitewood stated they were actually taken in Melbourne.
"She lives in Sydney. Those photos were taken in Melbourne. That's false straight away," he said. "Staying the night with her—that's another false accusation. We all actually stay in the same hotel, everyone's there.
"We console each other - we do that on the show as well. You don't know what we were saying to each other. It's a photo that says I had my arm around her, but I had my arm around her every day when I danced with her. I was probably in worse situations in front of millions of viewers."
Meanwhile, on Monday, Whitewood tweeted: "Wow! Sad and crazy news day full of rumours I didn't ask for!"