If you are seriously concerned about his mental stability and think he could harm your children, then you must act. If he is just this way toward you, then you can decide whether or not to take action. My ex acted insane for the first 4 months after I told him I wanted a divorce. From stalking me, calling people i knew and reading/deleting my emails to harassing me on the phone and threatening to hurt people I loved. He was very controlling in our relationship so losing the control over me made him lose control over himself in a way.
Chances are, he could just be mad at you and want to hurt you. If you don't think that your kids are seriously in danger, then I think you should weigh the options of taking action. As far as your summer visitation issues, he has to abide by the courts decision. If he's unreasonable and you can't work things out, then he MUST abide by the courts ruling, no ifs, ands or buts.
I'm sorry you are going through what you are going through. Having a controlling hot-headed ex myself, I can relate to your experience and it is super hard to deal with. Hang in there. Your kids will grow up and you will have to deal with him less and less. I hope hope things get better for you.
When Jerk would call or talk to me face-to-face, anytime no one else could hear, he would threaten me. Not physically, but I'll do this...I'll do that. I sent him a letter and cc'd it to my attorney requesting that all communication regarding the divorce be sent via e-mail or through the attorneys. We went back and forth by e-mail regarding summer visitation. I requested that if he is working or not around I can have the kids. At first he refused to answer this question, stating it is none of my business. I kept posing it. I simply did not respond to questions he had asked before and I had responded to. He became frustrated trying to communicate this way and called one night. His tone changed completely and for the last several weeks we have been able to work a lot of things out regarding the kids and the divorce settlement. He was smart enough not to threaten me by e-mail. At one point he slipped up during a phone conversation. I said, "This is why I refuse to talk to you. I refuse to sit here and be threatened by you. If you want to discuss this further, talk to your attorney." He apologized (I wonder when the last time is he's apologized for anything) and his whole tone changed.
Good luck. It really sucks! Let us know how it turns out.