Right component supplies the sound (xiàng → gǎng, with significant Old Chinese drift between velar initials, but a regular pattern).
巷 itself means 'alley between walls' and adds a beautiful semantic echo: a harbour is a water-alley between landmasses, a narrow passage for boats. So
港 is etymologically 'an alleyway made of water'. The Hong Kong (
香港 'fragrant harbour') reading uses this character.