The name literally means 'north of Mount Heng' (
衡山之阳) — in classical geography
阳 means the sunny north side of a mountain or the south side of a river.
衡山 is one of the Five Great Mountains (
五岳) of China, and the city takes its name from that relationship.