I agree talk to a family atty and find out what needs to be done .. honestly it doesn't sound like much if he doesn't show .. he doesn't show and that's a good thing. ALL states can be REALLY backwards when it comes to terminating parental rights .. I would suggest strongly that you document everything .. what I do is I have something on my computer that says Kids Visitation .. I document if he picks them up .. if he doesn't .. what time he drops them off .. if the kids were emotionally upset and disturbed when they come home .. I stick to the facts .. use words like so and so appeared to me blah blah blah .. my understanding was .. blah blah blah .. so and so expressed that they felt .. blah blah blah .. be as emotionally detached in description as possible. The police showed up on whatever date .. I called CPS on whatever date and time and spoke to Mr/s blah blah .. I think you get my drift I would go back as far as you could NOW and that way when you talk to an atty you have some back up and it's not just well he did this and he did that you have actual vested time into documenting this is what happened and times and dates.
Best of luck these situations are icky sticky .. Lisa ;)
Here's my thoughts on reading what you wrote -- know that I know nothing about this category of law -- sounds like it would be GREAT if he removed himself parentally. Just think of all the pain/arguments you'll be spared if he never needs to be consulted about anything to do with the kids forever. What a load off you that would be! You would never have to be in contact with him about holidays, travel plans, moving, school plays, etc.
YES it would stink for the kids but what really stinks is that they don't have a good dad. But they wouldn't have a good dad if he kept his parental rights. What they would have is a lot of broken promises, fear and stress.
However, does HIM deciding to do this mean he doesn't pay child support? That I don't know. Are you dependent upon it to pay bills/live out of poverty? I'm hoping not because, as sad as it is, sounds like you'll all be better off without him in your lives.