There's no need for you to worry so much about him.
Tips
usage
犯不着 (also written 犯不上) means something isn't worth the effort, trouble, or emotional cost. It is typically followed by a verb phrase indicating the action that isn't worth doing. More emphatic than 不值得 (bù zhídé, not worth it).