+1 for counselling.
Relationships, no matter how great they are, are a rollercoaster. It seems that yours keeps going down. Yes, there is a clear reason why (cheating/emotionally cheating) but your partner was pulling away before this happened (not being affectionate) there are some obvious communication issues.
I think you should plan to have several counselling sessions as I don't think this is something that can be worked out with just the 2 of you.
Kudos for not just calling it quits and trying to work on it. So many people give up way too fast.
Best of luck
