'Teen Mom' Custody Battle: Amber Portwood's Ex Gary Shirley Vows Their Daughter Will Never Live With Her Ex-Convict Mom

The last few months of 2013 were perhaps the best years of Teen Mom star Amber Portwood's life: Sober at last, she was released from jail after 17 months, and began the long process of rebuilding her relationship with her five-year-old daughter, Leah. But Leah's dad, Gary Shirley, is speaking out to shatter any hope that 2014 will be the year Portwood regains custody. In a new interview with OK! magazine, he insists, "It's going to stay the way it is, to protect Leah."

Shirley won full custody of the tot in December 2011, amid Portwood's struggles with drug addiction and anger management. And though he has forgiven her for how she treated him during that difficult time, he hasn't forgotten.

"Well, everyone knows that she went to jail for drugs and stuff," he explains. "I'm not saying that she'll ever get back into that, but I hear stories of how easy it is to relapse because that pressure is always there."

With that in mind, he says he plans to keep a tight lock on Leah's custody situation. "I don't want to screw myself over by agreeing to joint custody and then have to fight for Leah," Shirley tells OK! "So I think it's going to stay the way it is, just to protect Leah."

In addition, he says, "It's because I've grown, and I have a bond with my daughter that's special, and I don't want that to ever go away. Amber being away gave me a chance to get to know my daughter in a way most guys don't."

But that doesn't mean that there hasn't been some major mother-daughter bonding since Portwood's release.

"Leah's excited!" Shirley reveals. "She's glad that Mommy's home. … Sometimes I let Leah stay the night over with Portwood at her grandmother's before school so her hair is all nicely done for school."

And slowly, he says, the formerly tumultuous Teen Mom is learning how to be a good parent. "I feel like she has her daughter in mind more," Shirley admits. "But I thinks he's still adjusting to being out of prison. She needs to accept Leah at the age she is, versus the age she was when Amber went away."

He continues, "Leah's smarter and more independent now, and it's hard for Amber to say, 'Wow, you're not a baby any more. I don't have to hold the cup for you!'"

For more on the MTV stars -- including rumors of a rekindled romance and whether Portwood will fight for her daughter, pick up the latest issue of OK! magazine, on stands now!