He did something shameful and has been uneasy ever since.
Tips
culture
The proverb 不做亏心事,不怕鬼敲门 ('do no shameful deed, fear no ghost knocking at the door') is one of the most common usages — a guilty conscience is like inviting hauntings.
memory
亏心 literally = 'to shortchange the heart/conscience'. A 亏心事 is an act that owes a debt to your conscience.