Watch the video on RadarOnline.com (WARNING: Content Contains Explicit Humor)
The premise of the bit, which aired on Wednesday's Chelsea Lately, was that the redheaded comic was back at the studio where he hosted the Tonight Show -- a studio now inhabited by the feisty blonde comedienne.
Conan, stark nude, complained to the equally-bare Handler that she stole his studio, parking space and back massager.
"I do use that back massager -- although I had no idea it was for my back!" Chelsea said.
Later on, tensions rose as Chelsea bragged, "Everybody knows I am the first woman in late night."
Conan shot back, "No, that was Joan Rivers!"
Chelsea countered, "Have you taken a good look at her recently?"
"I know," Conan replied. "I'm still convinced, whatever that is, it's female."
The two then mock fought, with both ending up with with black eyes.
Chelsea Lately on weeknights at 11/10c on E!, while Conan airs weeknights at 11/10c on TBS.