破落 carries a strong sense of "former glory now lost" — used both for buildings (a once-grand house gone shabby) and for families (
破落户, the descendants of a declined gentry family). Compare with simple
破旧 (pòjiù, "worn-out"), which lacks the lost-grandeur connotation.