Have you tried marriage counseling? It might help to sit down with a third party to work things out. It might help you to go on your own. Going on my own helped me alot!
Even if counseling doesn't resolve the issues, at least you will know that you tried. If he refuses to go, then go by yourself, and that will also tell you something about his willingness to work on your marriage.
There are two ways you can deal with it. Number one you have to be completely unaware of it. He is in his fun and you are in your fun. Keeping them cool is not the only way to become like this in each other's lives. . . If he is in the midst of another marriage and he is doing all this on purpose