You are both strong women and I agree that the guilt belongs to your exes. Only God can judge the heart, but I know he would not want you to bare the shame of others. I felt ashamed of the things my ex did with this other woman and I had to learn that that was his shame, not mine. He made a fool of himself, not me...I can no way compare this to the two of you and how it must feel to have a husband who betrayed you, your family and others in a deeply sexual way.
So rid yourself of any shame these exes have brought about. You are not to bare the shame of others...that is what Christ came to do for those who truly wish for forgiveness. Judgement is for God...When asked what it was that we should be doing, the Lord said the greatest commandment of all is to love one another. If we truly love one another we would not have to worry about betrayal, but our exes really don't understand love at all. Don't ever let your ex steal your compassion for others and also for yourself. Wrongful guilt can do that to a person.