Edith, I'm sorry for the pain you must be experiencing. Whether this is (as you put it) just about sex, or whether this is an intimate relationship, both are painful. From the description of the relationship, it sounds like this may be about more than just sex.
Your husband could be in what is referred to as "limerence" - basically the head-over-heels feeling of falling in love. It actually starts to affect brain chemistry (there's a great article on limerence at
www.joebeam.com/blog), which may be one explanation for why it seems he just can't break away.
I would agree with del88 - some open and honest communication does need to take place. But make sure that you go into this communication calmly; no one wants to feel like they are being attacked. If you ask questions, be sure you are prepared to hear the true answer. Above all, make it a safe place for him to speak. if he opens up just once, but is immediately shut down, he will be less likely to be honest in the future.
I would also recommend that you borrow or buy a copy of the Your LovePath book. It does a great job of explaining how affairs start, build, and even end. If at all possible, you might want to consider attending a LovePath 911 seminar for marriages in crisis (more information is at the top of the page in this forum).
My prayers for your peace.