乌 here is not 'crow' or 'black' but the classical question word 'how, where', so
乌有 is literally 'where is there any?', that is, nothing at all. In
司马相如's rhapsody 《
子虚赋》 two invented speakers are called
子虚 'Sir Fictitious' and
乌有先生 'Mr. Nonexistent', which gave the idiom
子虚乌有 'pure fiction'.