忄 is the heart radical 心 in left-side form. It places 悔 in the inner-feeling family alongside 怕怀怒情 — all states of mind. Regret happens internally, so the heart header signals the emotional core.
每 supplies the sound (měi drifted to huǐ). The same phonetic powers 海梅莓霉. A subtle flavor of constancy and each-time fits regret — repeatedly, every moment, returning to the heart over what cannot be undone.