Your husband, (The Pastor) is threatening to ruin other families, if you don't agree to his terms and then says it will be your fault. How God like of him. Please do not buy into these threats. If he talks about your friends, and other parishioners, it is on HIM and no one else. And may God forgive him.
What ever has been built up in your two decades of marriage is 50/50, that includes his$97,000 IRA, bills, assets of any kind. And yes, he has to give support to his chlldren. But please run to a lawyer and get all the facts and make sure you know as much about your assets as possible. Income statement, should be photocopied, bank statements, and given to the lawyer. Your husband is not calling the shots here, there are laws.
Now I know you don't want this divorce and all you can do is offer to go for help. But if a man is determined to end the marriage, there is not a whole lot you can do. But you can fight to get as much as you are entitled to, for the sake of your own future, and for your children. This should be your goal now, and do not lose focus on this one issue.
Pastor or not, I am smelling a rat here, and I wouldn't be surprised if there is someone waiting in the wings.
You have a long journey ahead of you. We have all been there. I am now divorced alittle over a year and still it is no picnic, but it is so much better than being with someone that doesn't want to be with you. This forum has been a life saver for me over the years and I hope you take advantage of it. Just venting, reading other posts can give you knowledge and strength and comfort that you are not alone.