I'll get straight to the point: To me it sounds like he is rather controlling and manipulative, and I would advise to be really careful around this person. He might not realise it, but it won't help the situation if he does. If you can't be honest with him when you're hanging out with others, then to me that shows you cannot and do not have an open and caring relationship (even if he is affectionate when you do spend time together). 'Jeffery' should allow you to hang out with others.
Although have you ever tried to talk to him about it? Said it's bothering/worrying you, and that you really enjoy him as a friend, but don't like it when he does those things?
If you haven't, and he is willing to talk about it, then things can be worked on. If he isn't willing to talk about it and gets frustrated, I would suggest to get out as soon as possible.
And why?
Because this behaviour should not be accepted, and it could even get worse if it continues. It would be better for you in that case to break contact. It might be hard, but you have other friends, and I think it might be best you spend more time, effort and energy into those friendships than with 'Jeffrey'.
Hope this helps.