You know that once the kids reach a certain age that you two can start a life together. And his wife is thinking the same outcome for herself. You can work on conserving your marriage through open communication, therapy, and joint efforts to rebuild trust and intimacy. Professional help could possibly be necessary. Marriage is a big