Right
夫 supplies the sound - fū exactly, no shift. A clean phonetic pairing; the 'man' meaning of
夫 contributes nothing semantically. Same phonetic family:
肤 skin (also an outer layer!),
扶 to support,
趺 instep. The
肤/
麸 pair is a nice mnemonic - both are outer husk layers, one of the body, one of grain.