Do things legally this way he cannot come back and fight you. If you think he won't...he will. Mine came back when he sobered up and now I have 50/50 custody. Had I done it earlier it would have been easier for the courts to decide I was the responsible parent but because he's sobered up and did AA they found no fault in him. I worry every time my kids are with him theat he"ll slip. He's done it inthe past so I know he"ll do it again.
Get sole custody. DO it now while you can prove that he is unfit. then like susan said if he sobers up and is ready to be a parent then you can go from there. You and your daughter's safety is the only thing that matters at this point.