Should i walk out or compromise?

are you there to work as a professional-career wise or you are there for some different reasons? if you are a professional you shouldn't let these things happen between you and any colleague. Warn him, and if he continues, leave there.