Your ex sounds mixed up. He should never have full custody. You have the right to be with another man before the kids turn 18 and he needs to let go. Like the other poster said, just speak with him when it concerns the kids. When the kids turn 18, cut off contact with him unless it is urgent or concerns the kids, i.e., one of them is having a serious issue, getting married, etc.