Tree radical on the left marks 柚 as a kind of tree - the pomelo or grapefruit tree. The same radical groups most fruit-bearing trees: 桃 peach, 李 plum, 梨 pear, 桔 tangerine.
Right side supplies the sound: yóu becomes yòu (a tone shift). 由 also visually resembles a fruit with a stem on top, which works as a memory hook even though it is not the historical source of the image.
