He's just manipulating you. He is not capable of being happy himself, he is not capable of making anyone happy. If his children do not want to know about him, it is because his attitude was always as rude as he was. If you have enough money to live alone, do it, do not think twice, it is time to act. Listen to your friends and family and get all the help you can from them. Talk to them about your decision, you don't need to feel alone, you need the support of your loved ones. My ex-husband is like your husband, he abandoned me with my 3 children and now he is trying to come back into our lives with the excuse that he did nothing, said that I was the one who disappeared and he was looking for us. These types of men are sociopaths and they only hurt people. I'm not in the best position right now, but I'm trying to get on with my kids. Life isn't fair, but we can improve our lives for our loved ones if stop living with this kind of men.