A partial truth

Jill Lepore: These Truths: A History of the United States https://books.google.com/books?id=v2ZSDwAAQBAJ: '“It is imagined our Governor has been tampering with the Slaves & that he has it in contemplation to make great Use of them in case of a civil war,” young James Madison reported from Virginia to his friend William Bradford in Philadelphia. Lord Dunmore, the royal governor of Virginia, intended to offer freedom to slaves who would join the British army. “To say the truth, that is the only part in which this colony is vulnerable,” Madison admitted, “and if we should be subdued, we shall fall like Achilles by the hand of one that knows that secret.”64 But the colonists’ vulnerability to slave rebellion, that Achilles’ heel, was hardly a secret: it defined them. Madison’s own grandfather, Ambrose Madison, who’d first settled Montpelier, had been murdered by slaves in 1732, apparently poisoned to death, when he was thirty-six...

We need to include the plight of impressed seaman, children sold into colonial america, forced religious migration, and all other aspects of primitive worker right in colonial times.  legal, in additional to customary racism did not rally begin until long after the constitution. The founders were protecting their authoritarian limits on labor.

Racism and slavery are semi-orthogonal, in many cases the same races enforces slavery within itself. For example, no one would suggest that the union farm workers movement as about absolute slavery. But if you took the same California worker right to primitive 1750, then it look much closer to legal, racist slavery.