"Apart from them stealing money from my family, we would probably still be together now," the One Wish performer claimed during an explosive tell-all interview.
Ray J was referencing a 2008 lawsuit filed by his mother, Sonja Norwood, and said it was the final straw.
"When that happened, I said I don't want to be sleeping around with you no more – you stole money from my family. So that's why we stopped speaking," he alleged to Daily Mail.
Norwood sued Kim, who used to work as a stylist for Ray J's sister Brandy, claiming that $120,000 was charged to an Amex credit card without permission.
Norwood said Kim and her siblings Khloé, Kourtney, and Rob racked up the charges in 2006 and 2007 at the Kardashian family-owned clothing stores, DASH and Smooch.
Kim denied the claims and when it was all said and done, the lawsuit was dismissed in an apparent settlement in June 2009.
Ray J not only spoke about the lawsuit but also went into more detail about his meetup with Kanye West, Kim's estranged husband, in the newly released part of his interview.
Reflecting on the four hours they spent talking at LAX, Ray J said he wanted to move forward and keep the conversation they had between them.
"I asked him to keep this private. Father to father, I didn't want anyone to know that I did this because I was genuinely trying to make peace," the Love & Hip Hop star alleged.
Ray J also claimed Kim already had the only version of the unreleased second sex tape that West was supposed to recover from him. The Sexy Can I rapper said all that West collected were old pictures and texts.
During Ye's own sit-down interview with Hollywood Unlocked in January, West said that Kim cried when he returned from the meetup.
"You know why she cried when she seen it on the laptop? Because it represents how much she's been used. It represents how much people didn't love her and they just saw her as a commodity," the Yeezy fashion designer said at the time.
Not long after, Ray J seemingly replied via Twitter, writing: "This needs to stop. I also have kids."