In everyday speech
甥 is bound — you almost never see it solo. The common compound is
外甥 (sister's son), with the female counterpart
外甥女 (sister's daughter). The 'wài' (outside) marks them as the children of a married-out daughter — the patriarchal kinship system treats them as belonging to another family. A brother's son, by contrast, is
侄子 — INSIDE the lineage.