The Oxford English Dictionary says people in the 16th century first started using “boob” and “booby” as insults for being stupid. That likely sprang from the Latin balbus, meaning stammering, which then gave way to the Spanish bobo for stupid.