This is a big ass NO. NOPE. NO.
Let's break this down:
If your friends, all of your friends, are telling you the SAME THING about someone - relationship, friendship, family - whatever - LISTEN TO THEM. Don't romanticize someone simply because you WANT them to be good. It is hard to tell a friend something negative about someone they care about so if your friends have felt the need to express negative feelings towards this guy, they are real. They are valid.
HE TOLD YOU TO NEVER DO SOMETHING AGAIN? K.... Obviously I don't know your situation but yeah, that is manipulative behavior. Unless y'all were exclusive and you cheated on him or you in someway violated his personal boundaries etc... what the hell is he telling you not to do? I am thinking the absolute worst over here. If someone tells YOU how to behave, act, react, talk, not talk - NOPE. Unacceptable. If this guy is a friend or more, he owes you the respect to discuss feelings with you and come to a compromise. All I am hearing is "I did something he didn't like, he pitched a fit and told me not to do it again... or else" nah sweetie, we gots to do better in 2020.
You have feelings, okay, valid. Let's move the fuck on. Why do you have feelings for someone you know is a trash human? Let's reevaluate. You think he is attractive? Okay, cool. Look at him. Don't date him. Just because you have feelings for someone doesn't mean you should.
I hope you will reevaluate how this person treats you. How this person treats others. How this person communicates and reacts to "no".
2019 has been FULL of me seeing people accept a whole helluva lot less than they deserve and I am so damn tired of it.
There are WONDERFUL human beings on this Earth.
There are people who enjoy the things you do, hold their own, have their shit together, and want to share things WITH YOU.
Find those people and don't accept less than that.