I wouldn't say they're "from Ham," per se. It's not like Noah could give birth to three children of different races with the same mother. The Bible doesn't say whether or not all people on earth are descended from Noah and whether all life was wiped out in the flood. IIRC, it does say nephilim were on earth before the flood and after, but that doesn't say whether they survived or whether more were born.
