It is never too late to reclaim your life. Those are easy words to say, and in your case with special needs children, it would not be easy, but not impossible.
Sounds like you think you are all alone. You are not. There is much for you to learn, though. If you have never asked for help before, you need to know that emotionally, it can be hard to get a grasp on what you need to do if you decide to leave. Are you sure you have tried everything that you feel you need to do before you make this decision. It is important for you to be sure for your own sake that this is the right thing for you.
I wonder if you are getting help with your children from one of the many service agencies available that serve SN kids. There may be a support group for you to lean on with regard to them.
In your area, is there a domestic abuse center for those who have experienced this kind of physical, emotional or verbal abuse. Making a call to that center can make a big difference in your life.
It sounds like maybe your kids father has been abusive to them in the past, and you are worried that you would not be there to protect them if he gets partial custody, as you said. There are many options both legal and custodial that can be explored in order to find the safest way for you to proceed.
Ask for help from your local center. Calls are confidential, and they may have a lawyer on staff who could help you navigate through the legal process.
Come back here if you have more questions, but I believe the real help is face-to-face with someone who knows your story and can come up with a strategy that fits your situation.