"Everything that Derick has said has caused the family bad press," said the insider.
The entire family, especially the famous matriarch and patriarch, have become "pissed off," the insider continued.
Most recently, Dillard praised a homophobic baker who refused to make a gay wedding cake.
The insider said that Dillard's famous in-laws have tried to respect the fact that Derick is married to Jill, but they recently decided to confront him because "they don't like what he is doing."
The source continued to RadarOnline.com that the Duggars are concerned about "what he says and how he says it."
Ultimately, Dillard's insensitive tweets have died down in recent weeks. Jill's husband has also changed his Twitter account to private.
What do you think of Michelle and Jim Bob Duggar stepping in to confront Dillard? Let us know in the comments.
We pay for juicy info! Do you have a story for RadarOnline.com? Email us at firstname.lastname@example.org, or call us at (866) ON-RADAR (667-2327) any time, day or night.
For the biggest celebrity news stories of the day, controversies, crime, and other hot topics listen to our new podcast ‘All Rise’ below!